Amazon is using its massive collection of internal services and applications as “reinforcement learning gyms” to train its next generation of artificial intelligence, according to the executive leading the company’s centralized AI development efforts.
The strategy is key to building “more general intelligence systems that can just break out of the box and can learn a new task with minimum input,” said Rohit Prasad, Amazon’s senior vice president and head scientist for artificial general intelligence, during the opening session Wednesday at Madrona’s IA Summit in Seattle.
“I strongly believe that the way we get the learnings fast is by having this model learn in real-world environments with the applications that are built across Amazon,” Prasad said in response to a question from Madrona’s S. “Soma” Somasegar at the event.
The concept mirrors the way that Amazon originally took lessons from its own infrastructure development to create and launch what became its market-leading AWS cloud platform.
It illustrates one of the key advantages that tech giants such as Microsoft, Amazon, and Google have over smaller companies in the AI race, leveraging their own business operations in addition to their technology infrastructure.
Prasad, who was previously the senior vice president and head scientist of Amazon’s Alexa personal assistant, was named to the broader role in 2023, reporting to Amazon CEO Andy Jassy, as part of a larger effort by the company to catch up in generative AI at the time.
His comments at the event gave a window into his mindset today, and how the company is approaching its efforts to develop its own AI technology, including its in-house Nova models.
Amazon is building a “model factory”: Prasad said his team is moving away from a waterfall-style process of building one model at a time. Instead, they are focused on creating a “model factory” designed to “release a lot of models at a fast cadence.”
This mindset is key to improving the models faster, he said. It requires making strategic trade-offs for each release, deciding which properties — like the ability to call software tools or excel at software engineering — are key for a particular launch timeline.
Shifting focus to AI agents: A central theme of Prasad’s comments was the evolution from conversational AI to autonomous systems. “We are now moving from chatbots that just tell you things to agents that can actually do things,” he said.
This new era of agentic AI requires models that can break down a high-level task, integrate different sources of knowledge, and execute actions reliably, he said. As an example, he cited Amazon’s Nova Act model and toolkit for creating autonomous agents in web browsers
Using AI to automate “the muck”: Prasad highlighted the value of applying AI to internal productivity, particularly for unglamorous work such as automating the upgrade of Java versions. Practical business challenges are helping to drive Amazon’s internal AI adoption.
“I want AI to do the muck for me,” he said, “not the creative work.”
