On June 3, Yoshua Bengio, The World’s Most-Cited Computer Scientist, Announced the launch of Lawzero, a nonprofit that aims to create “safe by design” ai by pursuing Approach to Major Tech Companies.
Players like Openai and Google are investment heavily in ai agents – Systems that not only answers queries and generate images, but can craft plans and take action in the world. The goal of these companies is to create virtual employees that can do practically any job a human can, Known in the tech industry as artificial general intelligence, or AGI. Executives Like Google Deepmind’s Ceo Demis Hassabis Point to Agi’s Potential to Solve Climate Change or Cure Disease as a Motivator for its development.
Bengio, however, say we don’t need agenic systems to reap ai’s rewards – a false choice. He Says there’s a chance a system could make man control, with potential irreversible consortes. “If we get an ai that gives us the cure for cancer, but also maybe another version of that ai goes rogue and generates wave after wave after wave of bio-sopons that kill billions of people, this i done It, “He says. In 2023, bengio, along with others including openai’s ceo sam altman signed a statement declaring Risks such as pandemics and nuclear war. “
Now, Bengio, Through Lawzero, Aims to Sidestep the existent Perils by Focusing on Creating What He Calls “Scientist Ai” – A System Trained to Understand and Make Statistical Predicalls World, Crucially, without the agency to take independent actions. As he puts it: We could use ai to advance scientific programs without rolling the dice on agentic AI Systems.
Why Bengio Says We Need a New Approach to Ai
The current approach to giving ai agency is “dangerous,” bengio say. While Most Software Operates Through Rigid If-to Rules-IF the user clicks here, do this-TODAYI AI Systems Use Deep Learning. The technique, which bengio helped pioneer, trains artificial networks modeled loosly on the brain to find patterns in vast amounts of data. But recognizing patterns is just the first step. To turns these systems into useful applications like chatbots, engineers employs a training process called reinforcement learning. The AI Generates Thousands of Responses and receives feedback on Each One: A Virtual “Carrot” for Helpful Answers and a Virtual “Stick” for Responses that MISS The Mark. Through Millions of these trial-second-feedback cycles, the system gradually Learns to predict what responses are most likely to get a reward.
“It’s More Like Growing a Plant or Animal,” Bengio Says. “You don’t fully control what the animal is going to do. You provide it with the right conditions, and it grows and it becomes smarter. You can try to step in various directions.”
The same basic approach is now being used to imbue ai with greatness agency. Models are tasked with challenges with verifiable answers – Like Math Puzzles or Coding Problems – Ren Rewarded for taking the series of action that yields the solution. This Approach has Seen Ai Shatter Previous Benchmarks in Programming and Scientific Reasoning. For example, at the beginning of 2024, the best ai model scored only 2% on a standardized test for ai of sorts, consisting of real world software engineering problems; By December, an impressive 71.7%.
But with AI’s Greater Problem-Solving Ability Comes The Emergence of New Decept Skills, Bengio Says. The last few months have borne witness to ai systems learning to Mislead, Cheat, and Try to Evade Shutdown – Even Resorting to BlackMail. These have almost exclusively been in carefully contrared experiences that almost beg the Ai to Misbehave – For Example Example, by Asking It To Pursue Its Goal At All Costs. Reports of such behavior in the real-wind, thought, have begun to surface. Popular Ai Coding Startup Replit’s agent ignored explicit institution not to edit a system file Cognitive Revolution Podcast in May. The company’s enginers intertned, cutting the agent’s access by moving the file to a secure Sandbox, only for the ai agent to Attempt to ” Engineer ”The user to regain access.
The Quest to Build Human-Level Ai Agents Using Techniques Known to Produce Deceptive Tendencies, Bengio Says, is comparable to a car speeding down a narrow mountain road, with Eiter Side, and Thick Fog Obscurning The Path Ahead. “We need to set up the car with headlights and put some guardrails on the road,” he says.
What is “Scientist Ai”?
Lawzero’s focus is on developing “Scientist ai” which, as bengio describes, would be fundamentally non -gentic, trustworthy, and focused onwarding and trustfulness, rathfulness Its oven goals or merely imitating human behavior. The aim is creating a powerful tool that, while lacking the same autonomy other models have, is capable of generating hypotheses and accelerating scientific process to “Help us SOLVE CHLVE CHALEVE CHALEVE CHALEVE CHALLENGEGES of Humanity, Bengio says.
Lawzero has raised Nearly $ 30 Million Alredy from Several Philantropic Backers Including from Schmidt Sciences and Open Philantropy. “We want to raise more trust we know that as we move forward, we’ll need significant computer,” Bengio says. But even ten times that Figure would pale in comparison to the roughly $ 200 billion spent last year by tech giants on aggressively pursuing ai. Bengio’s hope is that scientist ai could help ensure the safety of highly autonomous systems developed by other players. “We can use those who are non -gentic ais as guardrails that just need to predict where the action of an agentic ai is dangerous,” Bengio says. Technical Interventions will only ever be one part of the solution, he adds, noting the need for regulations to ensure that safe practices are adopted.
Lawzero, Named after Science Fiction Author isaac asimov’s zeroth law of robotics – “A Robot May Not Harm Humanity, Or, by Inaction, Allow Humanity to Comel Harm” Founded to chart a safer path for ai development. Openai was founded as a nonprofit in 2015 with the goal of “Ensuring agi benefits all of humanity,” and intended to serve a counterbalance to industry players gued by Profit Motives. Since opening a for-protrfit arm in 2019, the organization has become one of the most Valuable Private Companies in the World, and has decided criticism, include critic, including from its founding ideals. “Well, the good news is we have the hindsight of maybe what not to do,” Bengio say, adding that he wants to avoid profits incentives and “brings governance of lawzero.”
“I Think Everyone Should Ask Themselves, ‘What can I do to make sure my children will have a future,” “Bengio Says. In March, He Stepped Down as Scientific Director of Mila, The Academic Lab He Co-Founded in the Early Nineties, In an effort to reorient his work towards tackling ai risk more directly. “Because i’m a researcher, my answer is, ‘Okay, i’m going to work on this scientific problem where maybe I can make a difference,’ But other people may have been different answers.”
