Adi Polak, a speaker with deep expertise in scaling machine learning and data streaming at Confluent, addressed a critical challenge in adopting generative AI: achieving precision in data retrieval.
According to Polak:
Achieving precision is one of the hardest things we need to do to operationalize AI, go from zero to one, from MVPs of prototypes to production, and see things that work.
In her talk, “Achieving Precision in AI: Retrieving the Right Data Using AI Agents,” at QCOn London 2025, Polak highlighted the limitations of current Retrieval-Augmented Generation (RAG) systems and introduced agenticRAG as a promising solution for delivering pinpoint accuracy.
Polak began by illustrating the challenges of AI precision with real-world examples, including the Air Canada chatbot incident that led to legal repercussions due to inaccurate information. She emphasized that precision becomes a non-negotiable differentiator impacting customer trust and business outcomes as organizations move beyond basic AI experimentation to production deployment.
The Quest for Measurable Precision in Generative AI
While traditional machine learning precision has established metrics, Polak pointed out the ongoing complexities of measuring precision in generative AI tasks like text and image generation. The lack of clear, automated evaluation methods remains a significant hurdle.
Data-Centric Optimization: RAG and Beyond
Polak outlined two primary data-centric optimization approaches: RAG and domain-specific fine-tuning. She focused on RAG, explaining its process of augmenting Large Language Models (LLM) responses with retrieved relevant data. While RAG improves grounding, it faces challenges such as retrieving outdated information, handling ambiguous queries, and dealing with latency. She explored various retrieval techniques like hybrid search, re-ranking, similarity search (including vector search), and graph search, noting that:
The challenge with term search is that you need to know the specific term you’re looking for. Today, with generative AI, we don’t always have that specific term.
Agentic RAG: Enhancing Precision Through Task Decomposition
The core of Polak’s presentation revolved around agentic RAG. She explained that in agentic systems, precision is significantly enhanced by breaking down complex tasks into smaller, more manageable sub-tasks handled by specific, intelligent agents. Examples include orchestrator agents managing workflows, worker agents executing specific retrieval or processing steps, and the use of feedback loops guided by LLMs acting as “judges” to refine the process. Polak also discussed architectural patterns in agent systems like blackboard, market-based, and hierarchical designs. She emphasized that:
As a judge, LLM has proved itself highly valuable as a feedback mechanism loop that goes beyond the human in the loop.
Actionable Insights for Implementing Precise AI
Polak concluded with actionable recommendations for attendees looking to improve AI precision:
- Investigate the integration of RAG to enhance AI accuracy in production environments.
- Explore domain-specific fine-tuning and hybrid search techniques to optimize data retrieval.
- Consider adopting agentic RAG systems for task-specific AI solutions requiring high accuracy.
- Evaluate the implementation of feedback loops and memory systems for continuous refinement of AI precision.
Polak’s talk provided a vision for the future of AI precision, emphasizing the potential of agentic RAG to move beyond generalized data retrieval towards intelligent, context-aware systems capable of delivering genuinely accurate and actionable insights.
