Another useful component are clearly documented workflows for common processes – such as multi-step actions linked to MCP servers or direct API calls. “It’s crucial that AI agents are coordinated through defined workflows. Then autonomy scales in a predictable, controlled manner rather than spiraling into chaos,” says Heath Ramsey, Group VP of AI at ServiceNow.
At this point, Jentic manager Kilcommins advises clear, machine-readable capability definitions – and refers to the Arazzo specification. This is an industry standard that emerged from the OpenAPI initiative.
Multi-agent orchestration
Companies that want to work with multi-agent systems also need AI agents that can be integrated with each other and fit well into continuous feedback loops. “Multi-agent systems only become necessary when they are sufficiently large. Then, instead of one generalist agent, a team of several specialized agents is used – for example for reasoning, retrieval and validation,” explains Anurag Gurtu, CEO of the AI startup AIRRIVED.
This reality requires a unifying element, as Amazon expert Kovi puts it: “At the core, you need an orchestration layer for the plan-do-evaluate cycle.”
According to the BI specialist, the common tools in this area include:
- LangGraph, a low-level orchestration framework,
- CrewAI, a Python framework for multi-agent orchestration, and
- Bedrock Agents, which helps AI agents automate multi-step tasks.
In addition, open standards and protocols such as A2A are likely to become more important in the future when it comes to enabling AI agents to work together effectively.
Security & Authorization
Because LLMs are notoriously prone to hallucinating and otherwise misbehaving, security is probably one of the most important elements to building secure agentic AI systems. “As soon as agents can change access rights, trigger workflows or resolve incidents, any unregulated decision becomes a potential control failure,” says AIRRIVED CEO Gurtu.
Because the potential scale of uncontrolled, chained executions is enormous, there is a need for clearly defined permissions for AI agents that prevent privilege escalation and the disclosure of sensitive data. However, in the case of agentic systems, differentiated security methods are appropriate, as Amazon expert Kovi notes: “An agent decides at runtime what should be queried and which tools should be called. That’s why you can’t define permissions in a conventional way.”
In order to secure the “Non-Human Internet” of the future, many experts see just-in-time authorization as crucial.
Human checkpoints
Even with advanced authentication and authorization, sensitive agent actions require human approval. At least they should. That’s why Shopify also relies on “Human-in-the-Loop by Design”, as McNamara explains: “We have introduced approval steps to prevent completely autonomous changes to production systems. This allows, for example, retailers to review AI-generated content before it goes live.”
Unsurprisingly, the financial service provider Block sees it similarly, as Jackie Brosamer, Head of Data and AI, explains: “Our general rule is: Everything that touches production systems needs human control points.”
Evaluation skills
Building agentic AI systems also requires extensive testing in advance. This is the only way to evaluate whether the actual results match the intended ones.
For this purpose, Shopify relies on both human testers and simulations with specialized, LLM-based “judges”: “Once this ‘judge’ reliably agrees with the human evaluators, you can trust the system,” says McNamara.
Behavioral observability
Observability is another essential layer for agentic systems. In the case of AI agents, however, this must go beyond traditional monitoring and error detection. Rather, it is important to capture extended signals – for example, why an agent preferred a certain action over another.
“Observability must be integrated into your agentic AI system from the start. You need transparency in every execution step – in prompts, tool calls, intermediate decisions and the final outputs,” warns Edgar Kussberg, Product Director for AI at dev tool provider Sonar.
If your AI agents are “observable,” you can also continuously optimize your system.
Context optimization strategies
Almost all the experts we spoke to agree on one point: It is far better to provide AI agents with minimal but relevant data – rather than overloading them with information. This is crucial in order to avoid overloading the context windows and a deterioration in output quality – as Block Manager Brosamer also states: “Thoughtful data curation is far more important than the data volume. The output quality of an agent depends directly on the quality of its context.”
For this purpose, the engineers at Block maintained clear README files and applied uniform documentation standards and well-structured project hierarchies, says Brosamer. In addition, they also adhered to other semantic conventions that helped the agents find relevant information.
“Effective systems give agents versatile search tools and allow them to go through retrieval loops until they have sufficient context,” says Sonar man Kussberg. The prevailing philosophy is gradual disclosure of information. Shopify also takes this to heart and relies on modular instruction delivery. McNamara explains: “Providing context just-in-time is critical. Instead of cluttering the system prompt, we deliver relevant context along with the tool data when it is needed.”
Amazon expert Kovi also points out that the context should also include semantic nuances: “If an AI agent doesn’t know that ‘active users’ means something different in the product area than in marketing, it will confidently give wrong answers – and that’s difficult to recognize.”
Architectural best practices
Agentic AI is trendy and can achieve a lot under the right conditions. However, it does not follow from this that everything has to be agent-based from now on. For example, the combination of LLMs and MCP integrations is ideal for situations in which highly scalable, situation-aware reasoning and reaction skills are required. However, for repetitive, deterministic automation, MCP can represent “overkill” – especially if the context is static and security requirements are strict.
Kilkommins therefore recommends first determining which behaviors are adaptive and which are deterministic – and then codifying the latter: “This allows agents to initiate defined programmed behaviors and ensures more stability.”
Determining the most important areas for agentic processes ultimately comes down to finding reusable use cases, ServiceNow manager Ramsey points out: “Companies that have successfully used agentic AI usually identify a process with a high level of friction in the first step. This could include service requests from employees or the response to customer incidents.”
Gurtu adds that AI agents work best when they are given specific business goals. “Start with decisions, not demos. What doesn’t work is treating agents like stateless chatbots or replacing humans with them overnight.”
Kussberg, on the other hand, is convinced that agents with limited autonomy deliver better results: “AI agents work best as specialists, not as generalists.”
Shopify, for example, sets clear limits on scaling tools – and has opted for a sub-agent architecture, as McNamara explains: “Our recommendation is actually to avoid multi-agent architectures initially. We are now going into sub-agents with the right approach. A key principle here is to build low-level tools and teach the system to translate natural language into that low-level language – rather than building tools for each scenario.”
Some further tips from our interviewees for designing and implementing agentic AI systems:
- Use open infrastructures: Open agents and vendor-independent frameworks make it possible to use the models that are best suited to the respective use case.
- Think API-first: Good API design and clear, machine-readable definitions are the best way to prepare for AI agents.
- Keep data in sync: Synchronizing shared data is another challenge. This can be mastered with event-controlled architectures.
- Creating a balance between access and control: Ensuring the security of agentic systems requires proactive security measures, comprehensive audit protocols and defensive data validation.
- Continuously improve: To avoid agent drift, agentic AI systems inevitably require ongoing maintenance.
The future of agentic systems
Looking ahead, experts expect that primarily multi-agent systems will be developed in the future. This should further encourage the need for more complex orchestration patterns and the adoption of open standards.
“I assume that in 2026 we will see experiments with frameworks to structure agent ‘factories’ that coordinate complex knowledge work,” says Block decision-maker Brosamer. The biggest challenge will then be to optimize existing information flows for agentic use cases.
Another aspect that could become more important in this future is alternative cloud offerings and edge-based inference. In the future, specific workloads could be outsourced from centralized cloud architectures and latency could be reduced. Akamai expert Weil explains: “In the future, competitive AI will require not only computing power, but also proximity. AI agents must be able to operate in the real world and interact with users, devices and data in almost real time.” (fm)
This article is im Original published by our sister publication Infoworld.com.
