Algorithmic Upkeep and Model Drift

Idea one

Uptime is not epistemic health

Conventional IT maintenance organised itself around availability. Is the server running? Is the application reachable? Did the integration complete? Are response times acceptable? Those questions are still necessary — a system that cannot be reached creates nothing — but they are no longer sufficient.

An agentic system can be online, responsive, securely connected and inside its cost envelope while its judgement has deteriorated. It still produces fluent answers, still invokes tools successfully, still completes workflows at speed. And it may be retrieving less relevant information, interpreting an instruction differently, relying on an outdated source hierarchy, failing to recognise a newly important exception, or expressing unjustified confidence in conclusions that no longer fit business reality.

The system has uptime. It does not necessarily have epistemic health.

An intelligent asset is not maintained because its dependencies were patched or its service-level agreement was met. It is maintained when its behaviour remains sufficiently grounded, calibrated, aligned and intelligible for the role it was authorised to perform. So the organisation has to move from availability monitoring to behavioural validation — and the difference between them is a single question. Availability monitoring asks whether the asset can operate. Behavioural validation asks whether it should continue operating at its present level of autonomy.

Idea two

The danger is that it remains convincing

Nothing dramatic has to happen for an authorisation to stop being legitimate. A model provider ships an update and a once-reliable prompt is followed less consistently. A policy repository is reorganised and document identifiers change, so retrieval still connects but some links now lead to archived guidance. An acquired business unit brings unfamiliar terminology and the agent starts treating a local classification as an enterprise risk category. A tool still returns data, but the field that indicated an active exception now carries a revised code.

No outage. Thousands of cases still processed. Response times excellent, dashboards green — and the conditions that made the original authorisation legitimate have changed.

So intelligent assets are perishable operating components, not static software products. A prompt library is not a set of durable instructions written once; it is a behavioural interface between organisational intent and a changing probabilistic model. A retrieval pipeline is not a search feature; its relevance depends on source authority, freshness, embedding quality, access rules and the relationships held in the ontology. An agent skill is not a fixed capability; it is a sequence of judgements, tool calls, policies, permissions and assumptions that can become unsafe when any one of them moves.

Even the meaning of a successful output drifts. A drafting agent still produces polished summaries while quietly dropping caveats its earlier version preserved. A coding agent still generates working code while choosing libraries no longer permitted under the security standard.

The danger is not that the system suddenly becomes unintelligent. The danger is that it remains convincing. Fluent deterioration is far harder to detect than visible breakdown — when an application crashes, people notice; when an agent escalates fewer cases because its confidence behaviour shifted, that reads as normal variation, and people may even welcome it.

Idea three

Maintenance debt, and the interval that must shorten

Organisations usually treat model evaluation as an event: select a model, run a benchmark, approve the use case, move on. But agentic performance is not fixed at the moment of approval. Foundational models change, APIs are deprecated, corpora grow, policies are revised, new prompt-injection methods appear, employees develop workarounds, the ontology evolves with the business language.

Maintenance debt accumulates whenever the organisation assumes a previously validated asset stays valid without recalibration. Like technical debt it can remain invisible while the system looks productive — but it concerns the continuing relationship between machine behaviour, organisational knowledge, human judgement and legitimate authority.

And it compounds, because agentic systems work through interdependence. A small drop in retrieval relevance alters a recommendation; that recommendation becomes another agent's context; the second agent uses it to choose a tool or prepare a decision; and if the person receives the outcome through an overloaded interface, the original weakness passes unnoticed. The failure need not originate in anything malicious or spectacular. It can emerge from ordinary neglect.

So inspection rhythms have to match consequence and volatility. A low-risk internal writing assistant needs light sampling and periodic prompt checks. A customer-facing or decision-support agent needs frequent regression testing, source-grounding audits and review of overrides. An agent that can execute transactions or reach sensitive information needs continuous monitoring of behaviour, permissions, tool usage and boundaries. The higher the autonomy and consequence, the shorter the maintenance interval must become.

That does not mean inspecting every output by hand — which would rebuild the verification tax this discipline exists to reduce. It means layered assurance: automated tests for known regressions, validator agents comparing outputs against authoritative sources, telemetry watching for unexpected shifts, and human experts spending their attention on high-risk deviations. The purpose is not universal suspicion. It is calibrated confidence.

Idea four

Prompts, retrieval and libraries are wear-parts

These get treated as minor implementation details. They are the operational parts through which organisational intent is translated into machine behaviour — they shape what the system notices, what it ignores, what it treats as authoritative, which actions it proposes, and how it responds to uncertainty.

A mechanical wear-part is not defective because it changes over time. It becomes risky when nobody inspects, calibrates, replaces or protects it as conditions change. Prompts behave the same way: the original design may be sound and carefully validated, and its effectiveness still depends on an environment that will not stand still.

In most organisations prompts emerge informally. Someone writes an instruction that works. Another team adapts it. A version is copied into a workflow tool. A third group changes a few words for a local need. What began as a helpful instruction ends up embedded in several processes with no clear owner, version history, performance record or retirement date. That is prompt sprawl — dangerous not because any one prompt is badly written, but because ungoverned variation makes behaviour impossible to understand or maintain. Two agents appear to do the same task while following subtly different instructions about sources, escalation, authority or tool choice. One is told to cite only approved internal sources; another merely to use available context. One is told to pause when policy evidence conflicts; another to produce the most helpful answer despite unresolved ambiguity. The difference stays invisible until an incident.

So: a prompt is not simply a request for output. It is a behavioural specification. It tells the model how to frame the task, which constraints to honour, what to prioritise, how to express uncertainty, when to use a tool and when to hand control back. If it governs a consequential workflow it deserves the discipline given to any operational specification — named ownership, so accountability is visible and a prompt cannot outlive the business rule that justified it, and version control recording what changed, why, who approved it, which workflows are affected and what tests were run, with the ability to roll back.

Idea five

The test set is itself an intelligent asset

A prompt is never interpreted in isolation. Its effect comes from the interaction between the prompt, the model, the retrieved context, the available tools and the actions the agent may take. A prompt that reliably produced cautious, source-grounded behaviour from one model version may produce shorter, more assertive, less consistent responses after a provider update. The wording has not changed. The behaviour has.

Which is why regression testing must be a routine maintenance activity rather than a pre-deployment event. The changes that warrant a run include a model update, an altered system instruction, a new tool, a revised ontology, a refreshed vector index, a modified permission, or a newly approved policy source.

And the library matters more than its size. Ordinary cases are often the easiest for a system to handle and the least likely to reveal deterioration. A maintained set includes routine examples but also ambiguous cases, exceptions, conflicting sources, adversarial inputs, obsolete-document traps, incomplete records, sensitive decisions, and cases where the correct answer is not an answer but an escalation.

For the service-resolution agent that means testing whether it recognises that a legacy classification needs specialised interpretation; whether it privileges current policy over an expired regional presentation; whether it spots a conflict between a contract amendment and standard guidance; whether it preserves the reason for uncertainty during a handoff; and whether it refuses to commit beyond its authority.

A test library that never changes may confirm that an agent still handles yesterday's world while failing to reveal that the world itself has moved on. So the test set is another intelligent asset requiring maintenance. And the honest summary of what all this buys: a system that performs well on these cases is not guaranteed to be safe — but a system that is not tested against them has not earned confidence.

When behaviour does change, autonomy may need to change with it: a workflow restricted to recommendation-only, an automatic action requiring authorisation until recalibration, a source withdrawn pending review, a prompt rolled back to a validated version. That is not failure. It is maintenance functioning as intended.