The knowledge layer: giving AI the meaning behind the data
Table of Contents
The knowledge layer is where a lot is happening in the data & AI space. In my conceptual framework I described this as what enables data & analytics acceleration: the layer that lets people and agents answer from what the business actually means, and makes its answers traceable.
While you can definitely point AI at a database and let it generate queries successfully, the AI at that moment does not understand how the business actually operates. It has no reliable way of reasoning about what the data means in your context, because that meaning is not in the database. It is scattered across the business:
- It’s in dozens of systems with different definitions and different data models for the same concept.
- Business rules are hidden inside applications, ETL jobs and dashboard logic.
- There are undocumented local processes and workarounds.
- Knowledge sits in people’s heads, who eventually leave.
When you let an AI agent operate in this landscape, it ends up guessing what data means. Now you are not just dealing with hallucinations. Worse, ask the same question twice and you could very well get two different answers that are both right in their own way. Finance and marketing apply different rules for what an “active customer” is, so the agent picks one of them without saying which. You end up with implicit ambiguity, a cognitive split in the organization. At best this split leads to classic distrust in numbers and distracting discussions, at worst agents start acting on wrong information at machine speed.
One side of the knowledge layer is therefore a mechanism to make meaning explicit. Explicit, not necessarily agreed: it is fine that finance and marketing each keep their own definition of an active customer, as long as both are written down and the agent can tell which one is relevant for the question. Instead of forcing consensus, implicit disagreement is removed.
The other side is that the same model also captures how the business actually runs. The processes that act on those customers and their contracts, the steps the processes are made of, and the conditions and exceptions that decide which step applies. All of it linked to data in your organization. What you end up building is a digital twin of your business. Not of a machine or a production line, but of the concepts, definitions and processes that apply to your business: a model complete enough that an AI agent can reason and act on it.
None of this is a new idea. Twenty years ago the data warehouse promised a model of the enterprise too, and largely did not deliver one. So the interesting question is not what a knowledge layer is, but why it should work this time. That is what this post is about, followed by what the industry actually means by ontology, knowledge graph, semantic layer and context layer.
Why now and not in 2006 #
The parallels between the two are strong:
| Data warehousing | Knowledge layer |
|---|---|
| Conformed dimensions | One governed identity, reusable across domains |
| Single version of the truth | Explicit, attributed plurality |
| Model the enterprise first, then load | Model the enterprise first, then let agents query |
| MDM and the golden record | Entity resolution in the graph |
| Slowly changing dimensions, bi-temporal Data Vault | Bi-temporal assertions in the graph |
We also know why most data warehouse initiatives did not achieve their promises: the top-down modeling approach was slow and rigid. Incremental delivery proved to be more successful, leaving us with many domain siloed data warehouses. This is why in the current knowledge layer era people often talk about “don’t boil the ocean”, advocating for the incremental approach. Gartner makes the same point: “Many organizations seek to define an enterprisewide schema, ontology or taxonomy first. This is a mistake. Such endeavors are costly, time-consuming, filled with disagreement and, in many cases, stopped before any value can be shown or delivered” (Jaffri, How to Build Knowledge Graphs That Enable AI-Driven Enterprise Applications, 22 May 2025, G00768041).
Mutual business understanding is still a good idea, but what has changed is the way we achieve this mutual understanding. And probably more importantly: the reason why we need the mutual understanding has changed.
What has changed #
Before AI we could tolerate differences in meaning better. A human looking at a dashboard has the business context and can judge why numbers might be diverging or look odd. They discuss them among their peers, know who to talk to to get it right or know where else to look for the right insight.
With conversational analytics this is already changing. The answer is still judged by a human, but it is the AI that decides how to answer. Without traceable answers, it becomes difficult to track down why the number is off. Even more dangerous is when the number seems right, but actually is wrong.
An AI agent is not able to make these judgments as it lacks business context. It needs to be explicitly told about the context in order to interpret the information, or else it will act upon an assumed understanding.
On the other hand, there is also a similarity between the human and AI: both can reason. This means that instead of trying to create a single source of truth, we can now actually model the disagreement and let the consumer reason over it. That is the finance and marketing case from the start: the knowledge layer holds both definitions of an active customer, and the AI decides which one is relevant to the question.
This is neuro-symbolic AI at play: the symbolic side represents the disagreement, while the generative side makes a situational choice. I will explore modeling ambiguity in a future post.
What has not changed is where these initiatives fail. As with most things in data and IT, technology is not the blocker: what breaks a knowledge layer implementation is political disagreement on definitions, models without an owner, and subject matter experts who already have enough on their plate. They are why the model has to stay small enough for a human to review, why plurality gets modeled rather than resolved, and why the layer has to help maintain itself. I come back to what they cost in the costs.
Industry terminology demystified #
Everyone is now talking about ontologies, knowledge graphs, semantic layers, context layers and knowledge layers. They are less competing architectures and more a sequence, each one adding something to the previous. They line up like this.
| Term | Common definition | What it adds |
|---|---|---|
| Semantic layer | “A business-friendly translation interface that sits between complex data structures and the people (or agents) that need to query them” (Alation) | Consistent metric definitions, historically a BI concept |
| Ontology | A formal specification of concepts, properties and relationships in a domain, usually in OWL or RDF (timbr) | Classifications, constraints, hierarchies, rules. Supports inference |
| Knowledge graph | The ontology instantiated with actual data: entities, relationships, traversable connections (timbr) | The instances. Cross-domain traversal |
| Context layer / enterprise context layer | The ontology plus “the broader enterprise context that agents need to navigate real-world decisions” (Alation) | Operational state, policy and decision provenance. Assembled context, not necessarily stored |
| Context graph | The knowledge graph extended with what changes: user activity, permissions, recent events, decision traces (timbr) | Situational state and organizational memory: what is true now, and what was decided before |
| Knowledge layer | No settled definition. A synonym for the semantic layer (Stardog, 2025), a tier below the context layer (Atlan), or the governed superset of ontology, data and memory (Neo4j) | Depends entirely on who is speaking. I define my interpretation below |
While semantic layer, ontology and knowledge graph are well understood, the terms context graph and knowledge layer are not as settled.
Context graph is a recent term, coined by Foundation Capital in December 2025. timbr states that it “is an emerging and inconsistently used term. Definitions are still diverging across vendors, investors, analysts, and AI infrastructure teams.” I agree, and I think the term is actually pointing at something else: it is pointing at agent memory, scaled to an organization.
The agent field already converged on the taxonomy for that: semantic memory (facts and concepts), procedural memory (how things get done), episodic memory (what happened) and short-term memory (the task at hand). This is currently focused on individual agent memory, but conceptually can scale to the organization as well. The knowledge management literature defines organizational memory (Walsh & Ungson, 1991, doi:10.2307/258607) as “stored information from an organization’s history that can be brought to bear on present decisions”.
The concept in the knowledge literature does not line up perfectly with the memory types in the agent field. The agent memory types are borrowed from psychology, where the episodic/semantic split goes back to Endel Tulving (1972). At its core it is about human memory, which is very different from the organizational context. However, procedural, semantic and episodic memory all exist at the organizational level too. Only short-term memory does not scale: it always belongs to one agent working on one task. This is why I think organizational memory is the right term to define memory in the enterprise knowledge layer: it is a shared, governed memory for many agent instances.
Knowledge layer, the term I use in this post, is also not established. Neo4j treats it as the umbrella and the context layer as a subset of it, while Atlan does the reverse. I follow Neo4j, which calls it “a shared, governed substrate where an enterprise’s knowledge lives” built from an ontology, the enterprise data it grounds against, and memory, where “every run leaves a decision trace, so the next agent inherits what the last one learned.” This aligns with my architecture of the system.
Moreover, I think the term captures the system better. Calling it a context layer names it after the consumption pattern. Serving an agent the slice it needs is one function of the system, but not the point of it. The point is the digital twin I described at the start: what the business means by its terms, how its work actually gets done, and what happened each time it was done. This body of knowledge is the real asset, as it provides insights into the organization that go beyond enabling agents.
Knowledge layer vocabulary #
When considering all this, this is how I define and use these terms:
| Term | Definition |
|---|---|
| Knowledge layer | The shared, governed platform where an organization’s knowledge lives: the ontology, the data it points to, and the memory of what has been done |
| Ontology | The classes, relationships and rules. What could exist. Authored and governed |
| Knowledge graph | The ontology and its instances, with provenance and history |
| Organizational memory | What the organization knows, shared and governed across agents rather than per agent: semantic (facts and concepts), procedural (how work gets done) and episodic (what happened) |
| Context layer | The runtime that works out what a question needs and delivers it |
| Semantic layer | Only the metric calculation |
Consensus #
In broad strokes the industry is agreeing on what the knowledge layer is:
- A layer between data and AI is necessary; prompting over raw schemas does not scale in an enterprise.
- The layer describes data rather than replacing it: meaning is defined once and is reused, instead of each tool re-implementing it.
- It carries identity through to the sources, so an agent sees only what its principal may see.
- Answers must carry lineage.
- Start with a use case, not with the model. Gartner’s prescription is a minimum viable ontology and minimum viable graph: only as many concepts and relationships as are needed to deliver a defined capability, then expanded iteratively. Separately, practitioners from Thoughtworks to Atlan name the same failure trap: boiling the ocean.
- Machine assistance drafts the model from existing governed artifacts (lineage, SQL, BI logic, glossaries) and has humans review and approve it.
So what #
So what happens when an agent would use the knowledge layer? Imagine a request to issue a €15,000 credit note to Acme Corp for a late delivery. Pointed at the raw data the agent will issue it: nothing there says that Acme’s MSA caps SLA penalties at 10% of the monthly invoice value. With that cap in the model, along with the process that says when and how to check it, the agent reads the contract term, retrieves the €100,000 monthly invoice value from the ERP, applies the 10%, finds that €15,000 exceeds the €10,000 cap, and routes the case to a human instead.
This becomes possible when business meaning is written down. How to do this and what makes the agent work is the subject of the next post.