The knowledge layer: keeping the model in sync with the business

The previous post described the knowledge layer as a library, whose catalog holds three things. The TBox is the ontology: the classes and relationships that say what could exist, that customers exist and that customers hold contracts. The asserted ABox holds the instances people put there deliberately and reviewed before they went in, including the processes the business runs. The observed ABox holds what machines put there: facts read from the source systems, and the traces of what agents did.

The example I gave is a credit note. An agent is asked to issue a €15,000 credit note to Acme Corp for a late delivery, and instead of working out what to do it retrieves the six approved steps of the credit note process. Those steps take it to Acme’s MSA, to the clause capping the SLA penalty at 10% of monthly invoice value, and to the €100,000 invoice value in the ERP, at which point the requested amount exceeds the €10,000 cap and the case goes to a human.

Described that way the layer sounds finished: the ontology defined, the process approved, the catalog filled. But it is never finished.

Change is a given #

A business changes continuously, so a digital twin of that business has to change with it. The two sides of the graph cope with this differently. The observed ABox (the facts loaded from systems of record) is automatically updated by the data pipelines that feed it. The TBox and asserted ABox (classes and curated instances) do not automatically refresh. Someone, for instance, has to notice that the company has started selling subscriptions alongside products. They have to see that the ontology has no way yet to express a subscription and that the credit note process does not know how to give credits on subscriptions. This is a manual process that requires human attention at the right time.

Next to a changing business, agents might also compose new processes. If agents repeatedly compose a similar process that has not been written down, then that becomes a candidate to be modeled.

A model that does not get updated to include “subscriptions” or document often-composed undocumented processes, will go stale. Building a knowledge layer is therefore not a one-off project to define an ontology, but requires continuous operations.

In theory a human could run all of these operations. In practice, that would mean a lot of time was spent at human speed. Instead this time needs to be shifted to machines. To maintain model-business alignment, the model has to evolve with the business. This is the self-evolving knowledge layer.

Self-evolving ontologies #

A self-evolving ontology is not one that rewrites itself. It is one that notices it does not accurately describe the business and proposes the correction. It watches how it is used, and where it sees divergence between use and model, it drafts the change: this class is missing, this relationship has not been traversed in six months, this process step keeps being taken but was never written down. A human still approves, as described in the evolution loop below.

This is not settled yet. There are two views: “ontology as a contract” and “ontology as a living system”.

In the contract view the value of the model comes from being fixed. It is an agreement between domains about what words mean, which needs to be held long enough to be relied on. Change is therefore an exception: deliberate, scheduled and negotiated between the people who own the definitions. Machine assistance is welcome for the bootstrap, but the model itself is a human artifact.

In the living system view the value comes from the model matching the business as it is today. Agents will make mistakes if they keep trusting a definition that was agreed two years ago, but no longer describes reality. Change is the normal operating mode, so the tooling and way of working to maintain the graph is built around a continuous stream of proposals and a review queue, rather than a quarterly review meeting. The model is co-authored: agents write, humans decide.

Both positions accept that the model has to change and that humans decide. They differ in what they consider the default state, which failure they want to avoid, and which failure they are willing to risk.

The evolution loop #

The loop to make the model self-evolving is: bootstrap from governed artifacts → observe use → agents propose → humans approve → repeat.

Bootstrap is based on existing, governed inputs. For the ontology, these are business glossaries, BI reports, existing conceptual enterprise data models, industry reference schemas (like FIBO or HL7), API contracts (OpenAPI or GraphQL), and master data management (MDM) configurations. For the processes, it is the places the how is already captured: application and ETL code, workflow configurations, runbooks and work instructions, and the event logs that record what actually ran.

Observe usage through interactions and new data arriving. Most important are queries that failed in production, process execution traces (the input for process mining), agent memory, new arriving unstructured data, human feedback, and queries spanning two domains that return contradictions or resolve one entity twice. The earliest signals usually come from people: in conversational analytics a user who says “that is not what we mean by active customer” is flagging a gap in meaning.

Agents interpret this data and propose adjustments. In the ontology those are semantic: a missing class, a relationship that does not apply anymore, an entity that exists twice because two domains identify it differently. For the processes they are structural: an undocumented step, a condition that is narrower than how the business actually applies it, or a composed path that has recurred often enough that it deserves approval.

Human experts review and provide feedback. In this feedback loop agents run simulations to show the impact of the changes, including the impact on earlier questions and answers, and the quality of information retrieval. A proposed process change will replay runs that already happened and show where it would have decided differently. Finally the expert approves.

Process drift is easier to catch than ontology drift. The process execution traces contain the evidence to evaluate how well the runs adhere to the approved processes. Process mining turns those traces into the processes as they actually run and compares them with the approved ones. It allows you to count how often a step is skipped or see how often an unapproved composed path is executed. When agents repeatedly compose a similar path that was never written down, mining surfaces it as a candidate.

When an LLM is involved in a process execution its decision intent needs to be analyzed as well. For instance to understand how it decided which of the two conflicting agreements applied, or why it bypassed a step. This reasoning is captured while the decision is being made. If you ask an LLM after the fact why it made a choice, it won’t share its actual internal logic; instead, it will simply generate a plausible-sounding post-hoc justification. By understanding the reasoning and conditions that triggered a process execution, you can understand when and how process drift occurs. Then you decide whether other adjustments are needed, or approve the agent-composed process as a standard process that the next agent simply follows instead of composing it again.

Follow that far enough and the process stops needing an agent at all. A high-volume process, where every step is specified and all branches covered, should run as code. That is cheaper, faster and more predictable than a model working out the same thing every time. What is left for the agent is where judgment is needed: requests that arrive as prose, a contract with a clause nobody modeled or a case where three agreements apply and one needs to be chosen.

Subject matter experts become citizen ontologists. The evolution loop supports domain experts who have learned enough modeling to judge a change, rather than modeling experts who have to learn the domain. Stardog’s CTO Evren Sirin makes the same claim: “we are all ontologists now”, with PhD ontologists being replaced by meaning stewards whose job is to maintain model coherence. The loop is what makes that possible: what arrives in front of them is a drafted change with its impact already simulated, rather than a blank model to author.

It does assume those experts are trained well enough to model and are given tools that fit the job, and that they have the hours to spend. That is what the loop costs.

Ontology operations #

Modeling a single scoped domain once is the easy part. The hard part starts afterwards: keeping the model matching a business that has changed, growing it into the next domain without breaking the first, and then verifying everything still works. Hardly anyone is addressing ontology operations, such as retrieval quality measurement, versioning, deprecation, cross-domain alignment and review-throughput monitoring. Little to no tooling exists, which means you will have to build them. The following functionality needs to be covered at a minimum.

Measure retrieval quality continuously. The evolution loop describes how the model changes. With every change, what agents retrieve will be altered. Evaluating retrieval quality is a broader topic in RAG systems in general, but a knowledge layer has more to work with than generic RAG: it has the graph and the execution traces. Create a golden set of questions with agreed answers and replay it after every approved change. Alongside it, track the metrics the traces already give you: how often a step is skipped, how often an unapproved composed path runs, how often an exception fires.

Version everything on the asserted side. Earlier I said that every decision an agent makes is reconstructable. That is only possible if you can retrieve the facts, the definition and the process as they were when the decision was made, rather than as they are now. The observed side covers this for the facts through bi-temporality. The asserted side has discrete, reviewed releases that need to be versioned explicitly.

Deprecate deliberately. Retiring a concept or a process is a different operation from changing one. It is easy to skip. A class that has not been traversed in a year, or a process that has since moved into code, should be marked as such. Left in place it could become a modeling liability.

Align domains on identity, not on definitions. Domain ontologies built separately produce a Tower of Babel when you try to join them. What makes a cross-domain join possible is a small set of core entities (customer, contract, product) carrying one governed identity that every domain resolves against. On top of shared identity the definitions are free to diverge, since plurality is modeled rather than resolved. What stays hard is structural mismatch: one domain’s contract is a single node, another’s is a bundle of line items. No amount of definition plurality reconciles that. Those need a deliberate mapping and someone who can decide across domains to approve it.

Watch the queue, not just the model. Proposal volume against review latency is the health metric of the whole loop. When proposals outpace review capacity, you risk going stale or get many rubber-stamped approvals.

What comes next #

Model coherence is possible. The main constraint is that all of it costs the time of the people who know how the business actually works, and who are already busy enough.

Whether their time is worth this is a different question than whether the architecture is sound. The last post in this series is about what a knowledge layer costs, what it returns, and how to start with one use case and repeat the process to add more.