Amazon S3 anniversary marks 20 years of evolution – News

For Amazon Web Services Inc., this year’s Amazon S3 anniversary marks more than a milestone — it underscores S3’s rise from an internal utility to a pillar of cloud infrastructure and beyond.

Commemorating the 20th Amazon S3 anniversary on March 14, the company is using Pi Day — the annual celebration of the mathematical constant π on 3/14 — to spotlight the service’s transformation into a defining piece of the modern internet. That evolution is likely to continue in the years to come, especially as S3 takes on a larger role in AI systems, according to theCUBE Research’s Rob Strechay.

“I think S3 is absolutely critical as part of that fabric of AI going forward,” Strechay said. “I think that when you start to look at things like the S3 Vectors, the S3 Tables, [Apache] Iceberg and all of the different open table formats, it’s really becoming that data substrate for AI and analytics.”

Strechay spoke with Andy Warfield (pictured), vice president and distinguished engineer at AWS, as well as theCUBE’s Dave Vellante, for the AWS Pi Day 20th Year Celebration, during an exclusive broadcast on theCUBE, News Media’s livestreaming studio. They discussed the Amazon S3 anniversary and its evolution from a cloud storage service into foundational infrastructure for the internet and AI, as well as AWS’ push into new data-layer capabilities. (* Disclosure below.)

Reflecting on the Amazon S3 anniversary

S3’s origins were rooted in Amazon’s own infrastructure challenges. Internally, teams building internet-facing applications were repeatedly forced to set up and manage their own storage. That decentralized approach led to fragmented storage silos and a great deal of repetitive, undifferentiated work, according to Warfield.

“At the end of the day, the storage that they were building on top of wasn’t the best fit for the types of web-facing applications that, at the time, Amazon was really pioneering with,” Warfield said. “S3 was born out of both a really significant internal need around having an elastic, scalable web-facing storage service, but also the observation that what was good for Amazon was probably something that was really valuable for a lot of external builders.”

AWS now sees S3 moving closer to application builders as the pace of launches accelerates, Warfield explained. Many databases and services are built directly on top of S3, relying on it for durability and scale while focusing their own innovation on higher-performance application layers.

“I think that we will probably continue to move closer to applications, which is going to drive us into higher-performance domains. It’s probably going to tie us into deeper integration with computational tasks that you want to perform on your data,” Warfield said. “And it’s definitely, absolutely, going to bring us more tightly integrated with the kinds of development tools that are emerging today that are allowing a much broader population to build applications.”

Here’s the complete video interview with Andy Warfield:

AWS takes S3 beyond the bucket

In a special AnalystANGLE segment, Vellante and Strechay looked at Amazon S3’s 20-year evolution and what comes next as it takes on a bigger role in AI and data infrastructure. As part of that shift, many cloud-native file systems now sit on top of object storage platforms such as S3, making performance the next major battleground, according to Strechay.

“You still have a read penalty …  but that can be overcome,” Strechay explained, referencing the extra latency that can come with reading data from object storage. “I think what Andy and Andy’s team and the team at S3 is really working on is, how do you make it even more performant? How do you make it more scalable and give you more throughput? Not just to go into the file system space, but to offer something different?”

S3 has always had native capabilities built around the idea of a bucket. But as the platform takes on broader roles, that language may no longer fully capture what it does, according to Vellante.

“A bucket implies I’m going to put stuff in it and I’m going to get stuff out of it. I expect that parlance is going to change. I hesitate to call it a data platform, because it’s not a data platform like you would think of a data store or a database, but it is more than just a bucket,” Vellante said. “It’s becoming this kind of substrate. The conversation is not necessarily about storage. It’s about data, it’s about feeding accelerators, [keeping] them busy, making sure that the caches are most efficient. It’s much more than just a bucket.”

While S3 may not be a data platform in the traditional sense, it is increasingly becoming part of one. The storage and services layers are moving further up the stack into areas such as metadata, tables and vectors, according to Strechay.

“As they start to continue to innovate on top of that, they’re going to keep pushing up,” Strechay said. “One day it may be a full data platform in there as well — I think that entire market is evolving, as we know very well.”

Here’s the complete AnalystANGLE:

(* Disclosure: TheCUBE is a paid media partner for the AWS Pi Day 20th Year Celebration. Neither AWS, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or News.)

Image: News