Would you trust a non-human general intelligence agency with no integrity?

Some are fixated on expecting or achieving so-called Artificial General Intelligence (AGI), which they interpret as an AI system capable of performing a number of tasks – reducing what they call intelligence to merely completing tasks and define greater intelligence solely by evaluating performance. of the performance of those tasks compared to humans.

If this is a property of human intelligence that the so-called AGI is supposed to imitate, then we are either already close to achieving ‘AGI’ with the hallucination capabilities of large language models, or we have yet to discover how far away we are from something create that can – rationally – calculate this level of irrationality.

Here are four important, yet underestimated, challenges on the road to evolving AI into “AGI.”

1. Benchmarks of intelligence reveal our limits.

OpenAI publicly defines “AGI” as a “highly autonomous system that outperforms humans at the most economically valuable work – benefiting all humanity.”

Their latest model, o3, has achieved a major milestone by scoring 75.7% on the ARC-AGI benchmark under standard computing conditions, and 87.5% with higher computing resources.

The ARC-AGI benchmark, based on the Abstract Reasoning Corpus (ARC), evaluates an AI system’s ability to adapt to new tasks and demonstrate fluid intelligence through visual puzzles that require understanding of basic concepts such as objects , boundaries and spatial relations. Before o3, the highest score on ARC-AGI was 53%, achieved by a hybrid approach that combined Claude 3.5 Sonnet with genetic algorithms and a code interpreter.

Given these results, some already expect that o3 has probably achieved ‘AGI’.

However, if we stick to OpenAI’s definition of what would qualify as ‘AGI’, aside from confirming coding skills or logical reasoning, there is no doubt that the ARC-AGI benchmark in no way validates that o3 is the human exceeds economic performance. valuable work for the benefit of all humanity.

Nothing in the ARC-AGI allows for proof that o3, or any other AI model, could on its own perform economically valuable work that would benefit all humanity, surpassing human performance.

2. If intelligence is the measure, impact is the ultimate test.

According to The Information, we have learned that OpenAI and Microsoft have internally defined the achievement of ‘AGI’ as the development of a system capable of generating $100 billion in profits.

Objectively, this different “AGI” definition is not necessarily mutually exclusive from the public definition of OpenAI, depending on how “economically valuable work that benefits all humanity” is defined.

Research, such as Theodore Panayotou’s, has examined the relationship between economic growth and environmental degradation, often finding that increased economic activity without appropriate policies can lead to environmental damage.

We could say that the public definition of OpenAI focuses on the functional capabilities and expectations of AGI, while the internal benchmark adds a financial metric to measure its impact and success.

Leaving aside the question raised by the exclusive nature of measuring intelligence by an economic indicator, is this metric a means to an end, or an end in itself?

In the first case, it may well be a limited but nevertheless laudable ambition.

In the second case, this is where ‘AGI’, presented as supposedly surpassing human intelligence in economically valuable work that benefits all humanity, could in reality lead us further to the survival of a society more entrenched than ever is in prioritizing the short term over long term human imperatives, reducing the definition of what is valuable to a purely economic dimension. Ultimately, this might not be all that intelligent and certainly would not transcend our own human misconceptions in that regard, but rather exacerbate them – and therefore become the antithesis of being beneficial to humanity.

3. When integrity falters, intelligence becomes a problem.

In the search for intelligent machine performance that makes a meaningful contribution to society by performing economically valuable work that benefits all humanity while exceeding human capabilities, artificial integrity – and not just artificial intelligence – is essential.

If it were necessary, recent developments underline this reality:

The spread of deepfake technology has led to sophisticated scams and disinformation campaigns. There is no need to discuss the negative economic potential that the proliferation of deepfake technology has, given the direct financial losses that companies could face due to false communications with suppliers, fraudulent financial transactions or identity impersonation, which could potentially lead to significant monetary losses and operational disruptions. Microsoft has even called for legislative action to combat AI-generated deepfakes that mislead consumers and manipulate public opinion. They call for clear labeling of synthetic content and the development of provenance tools to build trust in information.

There is a growing call for individuals to have the right to have their legal cases handled by humans rather than AI. Concerns have been raised about the implications of AI in legal proceedings, highlighting the importance of human judgment in the legal system. We cannot rule out the possibility that each of us might think that any human being, regardless of who he or she is, is better able to demonstrate humanity and therefore more inclined to demonstrate integrity regarding human values ​​than a machine. But in reality, it is our human biases that impose this limitation on the machine, and not the other way around. Indeed, machines are limited by the data and design choices provided by humans, meaning that their “failures” in emulating humanity or integrity can often be traced back to human shortcomings in consistently exhibiting those qualities.

Experts advise against seeking medical advice from AI chatbots due to concerns about accuracy and confidentiality. Relying on AI systems that lack integrity to comply with health information privacy laws for medical guidance could lead to misinformation and potential harm. This is a no brainer. The negative economic consequences could include higher healthcare costs due to misdiagnoses or delayed treatment, as well as lawsuits against companies providing AI systems that lack integrity in their approach to medical advice.

Technology companies are facing legal challenges over the use of copyrighted material to train AI systems without permission or compensation. Authors, news media and musicians have accused companies like OpenAI, Anthropic and Meta Platforms of violating their rights, highlighting the need for clear guidelines on AI’s use and creation of intellectual property. This infringement on the integrity of intellectual property sets a precedent that weakens protections for creators across industries, undermines their financial stability, and ultimately infringes on the intellectual rights of content creators, devaluing the essence of intellectual property itself.

The economic value of human achievement, which benefits all humanity, does not lie in human intelligence without integrity.

4. Intelligence is not value neutral.

Would you trust a non-human so-called general intelligence that has no integrity towards human values?

Defining “AGI” as “highly autonomous systems that outperform humans at economically valuable work – benefiting all humanity” is the way to go. Executing it is the most difficult part, not because it requires greater intelligence, but greater integrity – something we must demonstrate that we are capable of, to ensure that the machine can imitate this quality, which, frankly, is even less evenly distributed. among people than intelligence.

If the goal is to benefit humanity, the question of greater artificial intelligence – whether it could be called general intelligence or superintelligence – cannot be answered separately from the question of creating something akin to what the behavioral capacity of machine integrity would define: artificial integrity.

A critical shortcoming of the ARC-AGI benchmark is the lack of emphasis on integrity considerations. Evaluating an AI’s ability to make ethically, morally and socially acceptable reasoning and decisions is of paramount importance. This involves presenting the system with scenarios in which competing values ​​such as fairness, security and privacy must be balanced.

There cannot be such a thing as (artificial) intelligence that benefits all humanity without (artificial) integrity.

This goes well beyond adaptation to new tasks validated by visual puzzles that require an understanding of basic concepts such as objects, boundaries and spatial relationships.

And this is the real challenge of any advance in artificial intelligence, especially for those that seek to benefit all humanity, as stated in OpenAI’s AGI definition.