John O’Hara, the originator of the Advanced Message Queuing Protocol (AMQP), took the audience on a journey through the creation and evolution of this foundational technology during the keynote of day 3 at QCon London.
His “Advanced Message Queuing Politics” talk was a compelling narrative of people, strong visions, collaborations, challenges, and the intricate dynamics of building technology at scale. O’Hara, whose extensive background includes Chief Architect roles at JP Morgan and Bank of America and CEO and board member positions, recounted the genesis of AMQP.
He explained that AMQP was born out of a necessity to solve specific business problems within JP Morgan. The bank faced significant limitations with existing middleware systems, prompting the development of an internal solution aimed at commoditizing middleware and reducing reliance on a single vendor. Early in his career at JP Morgan, while writing a Swift gateway for a derivatives trading system, O’Hara experienced a moment that vividly illustrated the high stakes involved. As he described:
And three years in, writing the Swift gateway for a derivatives trading system, and we put it live. And I watched as the first… message came up in the system. The first thing it did was make two billion dollars in collateral calls… and I just stared at the screen and thought, ‘Well, that’s the most interesting console message I’ll ever see.’ You know, really brought it home.
This anecdote underscores the critical nature of the systems he worked on and the environment that necessitated a robust solution like AMQP.
Navigating Collaboration and Overcoming Challenges
The development of AMQP was a multi-organizational effort, requiring careful negotiation with numerous internal and external stakeholders. O’Hara highlighted the challenges encountered, including navigating patent risks and aligning the diverse motivations of the participants. iMatics, under Peter Hintjens, was contracted to build the first AMQP broker, aiming to make middleware more accessible. O’Hara shared anecdotes about presenting AMQP at industry conferences and Microsoft’s early involvement.
The Path to Standardization and Wide Adoption
O’Hara detailed the evolution of AMQP through various versions, acknowledging the difficulties in maintaining compatibility and establishing clear standards—the transition from version 0.9 to the significant 1.0 release involved navigating differing opinions among stakeholders. Ultimately, AMQP achieved the status of an international standard (ISO 19464).
O’Hara highlighted the widespread adoption of AMQP, noting that even cloud platforms like Azure Service Bus rely on AMQP 1.0 as a primary communication protocol for their queuing and publish/subscribe messaging features. Furthermore, RabbitMQ emerged as the most widely deployed implementation with substantial adoption. Key figures like Alexis Richardson, a long-time contributor to the messaging space and a former QCon London track host, played a pivotal role in the popularization and success of RabbitMQ and, by extension, AMQP.
Reflections on the Human Element of Open Standards
Reflecting on the AMQP journey, O’Hara emphasized the critical human element in developing open standards. He stressed the importance of individual identity, clearly defined business cases, and the delicate balance between vendors’ and users’ needs.
While acknowledging that a product is not inherently a standard, he highlighted the necessity of comprehensive specification and careful selection of standards bodies. The success of AMQP as a standard is a powerful illustration of the complex yet essential process of collaboration in such endeavors.
O’Hara’s insightful and often humorous account provided valuable lessons for anyone involved in large-scale technology development. It emphasized that the “politics” of collaboration and shared vision are just as crucial as the technical foundations.
