Over the last year, I keep hearing the same statements in meetings, reviews, and architecture forums:
“We’re doing AI.” “We have a chatbot now.” “We’ve deployed an agent.”
When I look a little closer, what most organizations really have is not enterprise AI. They have a tool.
Usually it is a chatbot, or a search assistant, or a workflow automation, or a RAG system. All of these are useful. I have built many of them myself. But none of these, by themselves, represent enterprise AI architecture.
AI is not a feature. AI is not a product.
AI is a new enterprise capability layer. And in large organizations, capability layers must be architected.
That is exactly what Enterprise Architecture — and TOGAF in particular — was created for.
The Real Problem: AI Without Architecture
When I work with large enterprises, I see a very familiar pattern emerging. Teams build isolated LLM pilots inside business units. Different groups spin up their own vector databases. Shadow AI tools appear outside governance. There is no consistent data ownership model, no security architecture, no operating model, and no serious cost control.
This is not innovation.
This is architecture debt being created in real time.
Why TOGAF Fits AI So Naturally
TOGAF was never meant only for “IT projects.” It was designed for enterprise transformations, for introducing new capability layers, for driving cross-business change, and for enforcing governance at scale.
AI is exactly this kind of transformation.
What Architecting AI with TOGAF Actually Means
It starts, as it always should, with clarity of intent. Before any model is chosen or any platform is provisioned, leadership must be able to explain what business outcomes AI is meant to drive, what decisions are being augmented or automated, and where sustainable advantage is expected to come from. If this is not clear, it is better to pause than to build.
From there, the conversation must move to business architecture. Which business capabilities are changing? Which workflows will be redesigned? Where must humans remain in the loop? And which metrics will actually move? If AI is not mapped to business capabilities, it remains a science experiment, not an enterprise system.
Very quickly, the hard problems surface in data and application architecture. Where is the source of truth? What data feeds training, retrieval, and feature systems? How are lineage, quality, privacy, and compliance enforced? On the application side, where does AI integrate with core systems? Where do agents operate? Where do decision services live? How do workflows trigger actions? This is where many AI initiatives quietly break down.
Then comes the technology architecture, which is where most organizations spend their time — often too early. Model strategy, inference layers, vector databases, feature stores, orchestration, observability, GPU and CPU strategy, and cost controls all matter. But from an enterprise perspective, when you look from 10,000 feet, you do not see models. You see cost curves, reliability risks, and blast radius.
After that comes the unglamorous but essential work of migration and implementation. Which capabilities move first? How do you avoid big-bang failures? How do you coexist with existing platforms? This is how AI becomes real, not just impressive in demos.
And then there is governance — which, in my experience, is where serious programs either succeed or fail. Security, data protection, access control, auditability, risk management, and explainability are not optional in enterprises. You do not scale what you cannot govern.
Finally, we must accept a simple truth: AI is not a project. It is a living system. Models drift. Data drifts. Costs drift. Capabilities evolve. This requires continuous architectural stewardship, not one-time delivery.
What People Call “AI” vs What It Actually Is
There is still a lot of confusion in the market. Generative AI is mostly an interface layer. Decision AI is where the economic value is created. Agents are operators that execute within workflows.
If you only built a GenAI interface, you have built a front end — not an enterprise system.
The 10,000-Foot View
From altitude, real enterprise AI looks very different. You see decision engines embedded into workflows, agents orchestrating business processes, retrieval grounded in governed data, feature stores driving predictions, and observability tracking cost, accuracy, drift, and risk.
This is not a collection of tools. This is a new digital nervous system.
The Only Question That Really Matters
If you are building something and calling it “AI,” ask yourself one simple question:
Is this a tool, or is this an enterprise capability?
If it does not map to enterprise architecture, to business capabilities, to governance, and to an operating model, then it is not enterprise AI architecture.
Final Thought
AI is the biggest enterprise architecture shift since cloud.
If you don’t approach AI with TOGAF — or at least with the same level of architectural discipline — you will almost certainly end up with impressive demos and fragile systems.
Enterprise Architecture, AI Strategy, Digital Transformation, Technology Leadership, TOGAF
