At its annual AWS Re:Invent conference earlier this month, Amazon unsurprisingly focused on AI, and placed more emphasis on the building and managing of agents.
In his keynote, AWS CEO Matt Garman argued that “the true value of AI has not yet been unlocked,” but said we’ll one day live in a world with “billions of agents inside every company and across every imaginable field.”
“Getting to a future of billions of agents, where every organization is getting real-world value and results from AI is going to require us to push the limits of what’s possible with the infrastructure,” Garman added. “We want to reimage every single process in the way that all of us work.”
A New Frontier
Indeed, the bulk of his keynote focused on the infrastructure for AI—new instances with chips from AMD, Intel, and Nvidia; new versions of Amazon’s own chips; and new AI models.
On the hardware side, Garman talked about setting up “AI factories” for building agents and applications, whether running Nvidia hardware or AWS’s own Trainium chips. The majority of inference in Amazon’s Bedrock AI development platform is now done on Trainium, including all use of Anthropic’s Claude, with over 1 million Trainium chips now in use, he said.
At the show, Amazon announced the general availability of Trainium 3 UltraServers, which use a new chip built on TSMC’s 3nm technology. Amazon says it delivers up to 4.4x more compute performance, 4x greater energy efficiency, and almost 4x more memory bandwidth than Trainium2 UltraServers. The new servers are available with up to 144 Trainium3 chips, which can deliver up to 362 FP8 PFLOPs with 4x lower latency.
(Credit: Amazon)
Separately, the company also announced Graviton 5, the new generation of its Arm-based chip. This version offers 192 cores in a package with a 5x larger L3 cache.
On the model side, the company announced a series of new AI models, including Nova 2 Pro, Lite, Sonic (a speech-to-speech model), and Omni, a multimodal model that supports text, images, video, and speech inputs while generating both text and image outputs.
Garman described these as “frontier models”—meant to be on the leading edge but still “cost-optimized.” In other words, he didn’t claim they had the absolute best performance, but said they gave similar performance at a lower cost.
Get Our Best Stories!
Your Daily Dose of Our Top Tech News
By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy
Policy.
Thanks for signing up!
Your subscription has been confirmed. Keep an eye on your inbox!
The most interesting announcement in this area to me was Nova Forge, which would allow organizations to create their own models by picking an early checkpoint from another version of Nova and then adding their own data sets as part of the pre-training process.
This is an alternative to taking fully pretrained models and fine-tuning them with custom data, which eventually causes them to “forget” knowledge acquired in training, or to RAG (retrieval augmented generation), which can improve results in a specific domain by connecting to a proprietary knowledgebase but doesn’t enhance the model’s underlying reasoning abilities. It’s an interesting concept; I’m curious to see if this really improves domain-specific models companies use in the real world.
Agents Everywhere
It seems like everyone is talking about agents these days, and AWS is no exception, with a lot of talk about “frontier agents,” which Amazon defines as those that are autonomous, can run for hours or days without intervention, and deliver complete outcomes rather than just tasks.
At the event, the company introduced a bunch of new features for Bedrock AgentCore, its platform for building, deploying, and managing agents. The big new thing here is Policy control, which Garman compared with guardrails or boundaries you’d give a teenager who just got a driver’s license. Other new features include memory, so an agent can remember previous conversations.
Recommended by Our Editors
Much of this is still in preview, and like the other major agent platforms—Google Agent Space (now part of Gemini Enterprise) and Microsoft Agent365—it all seems a little early to me. In other words, the concepts are good, but I think it will take some more use in the real world before people even know how to make them right. Only then will these platforms be ready for large-scale deployment.
Similarly, the company announced general availability of Nova Act, a service that helps developers build, deploy, and manage fleets of agents. But I was a bit taken aback by the company’s claim that such agents would be “over 90% reliable.” That sounds good, but it’s not good enough.
Meanwhile, Garman spent some time talking about the company’s Quick Suite agents, which connects a chatbot with access to company data, including both structured data from databases and unstructured data from applications like Microsoft 365. This can be used for things like research and data flows, he said. In addition, the company offers Amazon Connect, a self-service customer agent service.
But much of the discussion was on tools for developers, including new features for Kiro, the company’s AI-integrated development environment and Transform, a tool for migrating code from legacy languages to a more modern system. Most interesting were a Kiro agent to help development teams create projects; a DevOps Agent designed to help developers respond to incidents, identify root causes, and prevent future issues; and a security agent designed to protect applications through code analysis and penetration testing.
“The reality is, building and scaling these amazing agentic systems are harder than the problems they are trying to solve,” AWS VP of Agentic AI Swami Sivasubramanian said in a separate Re:Invent keynote. The new tools from Amazon and the others are steps in the right direction, but I think there’s still a lot of room for improvement.
About Our Expert
Michael J. Miller
Former Editor in Chief
Experience
Michael J. Miller is chief information officer at Ziff Brothers Investments, a private investment firm. From 1991 to 2005, Miller was editor-in-chief of PC Magazine,responsible for the editorial direction, quality, and presentation of the world’s largest computer publication. No investment advice is offered in this column. All duties are disclaimed. Miller works separately for a private investment firm which may at any time invest in companies whose products are discussed, and no disclosure of securities transactions will be made.
Until late 2006, Miller was the Chief Content Officer for Ziff Davis Media, responsible for overseeing the editorial positions of Ziff Davis’s magazines, websites, and events. As Editorial Director for Ziff Davis Publishing since 1997, Miller took an active role in helping to identify new editorial needs in the marketplace and in shaping the editorial positioning of every Ziff Davis title. Under Miller’s supervision, PC Magazine grew to have the largest readership of any technology publication in the world. PC Magazine evolved from its successful PCMagNet service on CompuServe to become one of the earliest and most successful web sites.
As an accomplished journalist, well versed in product testing and evaluating and writing about software issues, and as an experienced public speaker, Miller has become a leading commentator on the computer industry. He has participated as a speaker and panelist in industry conferences, has appeared on numerous business television and radio programs discussing technology issues, and is frequently quoted in major newspapers. His areas of special expertise include the Internet and its applications, desktop productivity tools, and the use of PCs in business applications. Prior to joining PC Magazine, Miller was editor-in-chief of InfoWorld, which he joined as executive editor in 1985. At InfoWorld, he was responsible for development of the magazine’s comparative reviews and oversaw the establishment of the InfoWorld Test Center. Previously, he was the west coast bureau chief for Popular Computing, and senior editor for Building Design & Construction. Miller earned a BS in computer science from Rensselaer Polytechnic Institute in Troy, New York and an MS in journalism from the Medill School of Journalism at Northwestern University in Evanston, Illinois. He has received several awards for his writing and editing, including being named to Medill’s Alumni Hall of Achievement
Latest By Michael J. Miller
Read Full Bio
