At the recent AWS re:Invent conference in Las Vegas, Amazon announced the public preview of Aurora DSQL, a serverless, distributed SQL database featuring active-active high availability. This new PostgreSQL-compatible database option has generated significant excitement within the AWS community and was widely regarded by attendees as the standout announcement of the conference.
Aurora DSQL is a distributed SQL database that claims unlimited scalability, 99.99% single-region availability, and 99.999% multi-region active-active availability. The database handles horizontal scaling, multi-region active-active operations, connection management, and scale-to-zero, a long-awaited feature recently added to Aurora. Luc van Donkersgoed, principal engineer at PostNL, writes:
IT’S HERE! Amazon Aurora Distributed SQL (DSQL). It’s DynamoDB and Postgres merged into a truly serverless relational database. This is THE announcement of Re:Invent 2024. Everything else is irrelevant.
Aurora already offered PostgreSQL-compatible databases spanning multiple regions through Global Database, but the technology behind DSQL is significantly different. In DSQL, both regions operate in active-active mode, accepting writes and cross-replicating data. In contrast, Global Database can only replicate data from one region to another, with only one region allowed to write to the database. The secondary region acts as a read-only replica and can be promoted to primary if needed.
In a single-region configuration, Aurora DSQL commits all write transactions to a distributed transaction log and synchronously replicates the committed log data to user storage replicas across three availability zones. In a multi-region configuration, the clusters improve availability through two regional endpoints, one for each linked cluster region. Introducing the new service, Raluca Constantin, senior database engineer at AWS, and Arun Sankaranarayanan, specialist solution architect at AWS, write:
Both endpoints of a linked cluster present a single logical database and support concurrent read and write operations with strong data consistency. This allows you to balance applications and connections for geographic locations, performance, or resiliency purposes, making sure readers consistently see the same data.
A third region acts as the witness region, receiving data written to linked clusters and storing a limited window of encrypted transaction logs to ensure multi-region durability and availability.
Source: AWS blog
As explained in a separate article, Aurora DSQL employs optimistic concurrency control (OCC) instead of the traditional locking approach. Rajesh Kantamani, senior database specialist at AWS, and Prasad Matkar, database specialist at AWS, write:
Using OCC with Aurora DSQL is well-suited due to its distributed architecture, because it allows for higher throughput and system efficiency by avoiding the need for resource locking during transaction execution.
The cloud provider highlights that OCC not only prevents a single slow transaction from impacting or slowing down other transactions but also provides better scalability for query processing and a more robust cluster to realistic failures. This is achieved by avoiding reliance on locking mechanisms, which can lead to deadlocks or performance bottlenecks.
Marc Brooker, VP and distinguished engineer at AWS, has released a series of technical articles sharing insights into the implementation of Aurora DSQL. He describes how transaction latency remains constant relative to the transaction statement count, even across regions. Brooker also explains how writes, conflict detection, and durability are managed, as well as the implementation of transaction isolation and query performance. These articles have proven popular, and his deep-dive session at the conference is now available on YouTube.
Source: Brooker’s blog
Many in the community view the new offering as AWS challenging PostgreSQL-compatible Neon, Cloudflare D1, and Google Cloud Spanner. By separating transaction logic from the storage layer and processing transactions with strong consistency, Amazon’s CEO claimed on stage that the new database delivers reads and writes that are four times faster than Spanner.
The reaction on Hacker News has been mixed, with many developers highlighting a significant set of limitations. While the implementation relies on AWS precision time infrastructure which provides an accurate clock with strong error bounds, some experts see DSQL an inspiring project for open-source databases. User Edwinr95 writes instead:
No temporary tables, no foreign keys, no views, no more than 10k rows in a transaction. Except for some basic wire compatibility with the Postgres protocol, I’d hardly call this a “database”, and more a key-value store.
The new service aims to provide low latency and synchronous replication worldwide. However, the public preview is currently available only in three US regions: Northern Virginia, Ohio, and Oregon.
