At QCon London 2026, Lada Indra, Head of Data Platform at Pleo, shared insights from his experience across high-scale data systems. He illustrated both the risks of poorly aligned teams and the practical strategies that organizations can adopt to bridge the gap.
AI has fundamentally changed how engineering and data teams interact. Once clearly separate domains, engineers building production systems, data teams producing dashboards, the lines are now increasingly blurred. Senior engineers and leaders must confront this reality, data is no longer just backend fuel or a reporting artifact. It’s a first-class part of production, driving real-time decisions that impact customers and revenue.
In modern AI-driven systems, engineering and data responsibilities are no longer siloed. Engineers may deploy models, handle streaming predictions, and manage real-time features. Data teams are tasked with ensuring production-grade pipelines, monitoring model outputs, and maintaining data quality at scale.
The speaker outlined practical strategies for navigating the increasingly blurred boundaries between engineering and data teams, emphasizing the use of data contracts to treat data streams like APIs by defining ownership, schema, and service-level expectations in code, enforced through CI/CD pipelines or schema registries to ensure accountability and prevent bad data from reaching downstream consumers, alongside full-stack observability, which extends monitoring beyond uptime and latency to include data health by tracking semantic validity, freshness, and consistency, enabling teams to monitor and validate these critical metrics effectively.
Testing with production data is essential because staging environments rarely capture real-world edge cases, and using shadow environments, partial replicas of production pipelines, allows teams to safely validate models and data transformations under real traffic distributions, while graceful degradation patterns minimize user impact in case of failures.
The old divide between engineering and data is a fiction, as AI systems now rely on real-time data to make decisions that directly affect customers, requiring the same operational rigor for data as for software. Shared ownership of data quality is more important than tooling alone, demanding accountability and clarity between teams.
Bridging the engineering-data divide also requires both technical skills and mindset shifts, including technical literacy to understand storage patterns, big data processing, and analytical modeling to anticipate downstream needs, mindset shifts to embrace shared ownership of data quality, retries, idempotency, and the distinction between analytical and transactional data, and organizational alignment by including analytics engineers and data scientists in design and architecture meetings and implementing joint incident management across teams.
Indra emphasized that even small organizational changes, such as involving the right stakeholders in architecture discussions or enforcing contracts on key business events, can have an outsized impact on operational reliability and collaboration.
Key takeaways from the talk highlight that data is now a first-class production asset, and the old divide between engineering and data is a fiction, as AI systems demand the same operational rigor for data as for software.
Shared ownership of data quality is more important than tooling alone, requiring accountability and clarity between teams. Concrete patterns such as data contracts, schema registries, observability, shadow environments, and graceful degradation help manage the messy realities of production data, while investing in T-shaped skills, where engineers understand the data layer and data engineers understand production constraints, creates invaluable cross-functional expertise.
As Indra concluded, the AI era requires teams to think holistically, meaning one system, one team, shared responsibility, and only by dissolving silos and embracing joint accountability can organizations reliably deliver AI-powered features at scale.
