Nobody Uses the Old Model. Yet It Can't Be Turned Off

The article argues that the real problem with retiring old AI models is not the shutdown itself, but an architecture in which the model is called directly and an organization that does not know who depends on it — the "undeclared consumers" described by Sculley et al. (NIPS 2015). The author walks through tools for discovering dependencies (telemetry and distributed tracing before the scream test), model states in registries (MLflow aliases, Vertex AI), deprecation timelines at OpenAI, Azure, Amazon and Anthropic, and architectural prevention: an anti-corruption layer, an AI gateway, versioned prompts, two vector indexes, and blue/green with champion/challenger. It closes with a governance layer — a service catalog (Backstage), model cards, and consumer-driven contracts.

One operations team had roughly three thousand applications on its books and a hunch that a large share of them had long since stopped serving anyone. What was missing was a list of who called what. So they reached for a method that operations engineers call, in their jargon, a scream test: they simply switched off a thousand applications and waited to see who would shout. Nobody screamed. Nobody noticed anything and nobody cared.

It sounds like a feel-good story, and it is usually told as one. In fact it is a warning. The method only works when there is a safety net underneath it — the ability to bring a service back practically instantly, and an archive of data in case a straggler turns up, say a monthly or quarterly batch job. Without that net it is a gamble. And the fact that for years nobody in the company knows what depends on what is not an operational detail. It is a symptom of something older.

What is expensive is not the old model hogging a GPU that cannot be switched off. Expensive are the two things standing behind it. First, an architecture in which switching it off means rewriting applications. Second, an organization that cannot answer the question of what will break once you unplug it. The first is a problem of code, the second a problem of records and accountability — and both have to be solved, because solving only one is not enough.

A proper model lifecycle does not begin with deployment into production. It begins with the design of an interface and an entry in the catalogue. Anyone who deals with retirement only at the moment the model starts eating into the compute budget lost the game a few years earlier — at the first direct call to the vendor's library that nobody wrote down.

Naming that symptom is not new. As early as 2015, a group of Google engineers described, in Hidden Technical Debt in Machine Learning Systems (Sculley et al., NIPS 2015), a phenomenon that fits the problem described above like a key into a lock: undeclared consumers. A model's output tends to be freely available, and over time another system quietly hooks into it as its own input — without anyone keeping a record of it. The authors put it bluntly: such couplings are at best expensive and at worst dangerous, because they create hidden tight coupling. The model then cannot be safely removed, because nobody documented its dependencies.

With language models, a compute bill is added on top — one that, unlike a forgotten virtual machine, does not light up in the list of idle instances. The cost of inference hides inside API calls, and an underutilized GPU is hard to find. A model that almost nobody calls can therefore quietly nibble away memory and compute for months before anyone notices. Abandoned models like this are sometimes called zombie models — models that nobody governs and nobody retires.

Discovering the consumers is therefore the first step, and fortunately also the one best covered by tooling. Before you reach for a scream test, turn on request logging on the inference server and roll out distributed tracing — OpenTelemetry, Jaeger or Zipkin can assemble, from propagated headers, a graph of who calls whom and reveal links that grew organically and that nobody ever drew. Metrics from inference servers such as Triton, KServe, vLLM or Ray Serve will additionally tell you how many requests actually flow to a given model. A model with zero traffic is a candidate for retirement; only then does a scream test make sense, and even then with a prepared rollback and archived data. The order matters: telemetry first, screaming afterwards.

Once you know who calls the model, the formal side of things comes into play. A model is not either "alive" or "dead"; it passes through a state machine, and that state machine has to be machine-readable.

Model registries have been through this and learned their lesson. MLflow historically offered four fixed stages (None, Staging, Production, Archived), but since version 2.9 it labels them superseded and points users towards aliases and tags. The difference is not cosmetic: an alias is a movable named pointer, say @champion, and several of them can point at a single version, which a fixed stage could not do. Google's Vertex AI goes further still and prescribes no convention at all: the only alias the registry knows on its own is called default, and it determines which version is used when nobody says otherwise. Everything else — production, staging, challenger or anything else — you name yourself. It sounds like an absence of design; in fact it is the core of it. A registry should not impose a process, it should offer a pointer that can be moved.

Why do I insist on this? Because "it's deprecated" written in a wiki is read by nobody and no automation reacts to it. Whereas a state read out of a registry through an API can be plugged into CI, into an alert, into a gate, into a dashboard. The difference between an organization that knows and an organization that suspects is not whether it has it written down — it is whether it has it written down somewhere that can be read without a human.

The protocol layer has its own tools for this, they just go almost unused. The Sunset header, which announces the date on which a resource will stop responding, was defined by Erik Wilde in RFC 8594 back in May 2019; its younger sibling Deprecation only made it into a standard as RFC 9745 in March 2025. The accompanying Link points to a migration guide, and after the shutdown date the status code 410 Gone is typically returned. Frankly, most APIs do not send these headers and ordinary HTTP libraries silently discard them, so on their own they save nothing; the value comes from the discipline around them. More interesting is what the model vendors write about their own lifecycles — because those are the timelines against which you can set your own.

And those timelines can be read as a price list of patience. OpenAI promises at least six months' notice for generally available models, three for specialized variants, and for previews possibly as little as two weeks. Azure keeps generally available versions for at least twelve months and gives at least sixty days' warning before retirement. Amazon has three states for models in Bedrock (Active, Legacy and EOL), which can be read programmatically from the modelLifecycle field, and guarantees twelve months on the platform and six months in the Legacy state; as of February 2026 it added a phase worth the attention of anyone who thinks that staying on an old model is free: after three months in Legacy the model moves into a public extended-access mode, where it keeps running — but at a higher price set by its provider. Postponing migration has thus been issued an invoice.

Anthropic then does something without parallel in this industry. Beyond a sixty-day notice period for publicly released models, it has committed to preserving the weights of all such models for at least as long as the company exists, and before retiring a model it conducts a so-called retirement interview — a conversation in which it asks whether the model has any wishes. Claude Opus 3 was the first to go through this entire process, on 5 January 2026; formally it is retired, in practice it remains available to subscribers and via the API on request. The company adds a sentence that sums up this whole article better than I can: the cost of maintaining availability grows roughly linearly with each additional model served. That is precisely why models get retired at all — and precisely why one day it will apply to you too.

Everything described above is still firefighting, though. If retiring a model hurts, the problem is not that you cannot switch things off — it is how you switched them on. And the cure is several decades old.

It is called the anti-corruption layer, and it was first described by Eric Evans in his 2003 book Domain-Driven Design; today Microsoft has it in its pattern catalogue as well. Its purpose is a single one: not to let the model of an external, changeable subsystem "infect" the design of your application. Translated into the language of language models — your application never calls the vendor's library directly. It talks to your own interface, and the translation to a specific vendor is handled by an adapter hidden behind that interface. When you change models, you change or add an adapter; the domain stays untouched. It is the same idea as hexagonal architecture, ports and adapters, or the well-known adapter and strategy design patterns. Nothing revolutionary, just finally applied to the fastest-obsolescing component in the stack.

In today's practice this layer is most often embodied by the so-called AI gateway. A single internal endpoint behind which many models and vendors hide. The open-source LiteLLM claims a hundred-plus providers under one interface and lists Netflix and Lemonade among its production users; Cloudflare, Kong, Portkey, MLflow AI Gateway, or the Vercel AI SDK in TypeScript, promise essentially the same — switching a model becomes a change of configuration, not of code. That this is not merely marketing is shown by DoorDash, which builds its AI code reviewer deliberately model-agnostic so it can reach for the best model across vendors and open source and measure them with the same set of tests.

It is the OpenAI-compatible interface that has meanwhile become the de facto industry standard — it is spoken by vLLM, TGI, Ollama, llama.cpp and by compatible endpoints of other large vendors, so that in the ideal case you swap a model by rewriting the server address and the model name. Just do not expect more of that compatibility than it can bear. Mechanical swapping, yes; identical behaviour, never guaranteed. The FutureSearch team, which pushes tens of thousands of calls a day across three large vendors, sums it up by saying that the interfaces look interchangeable on paper, but the differences will break your code — a different tool-calling schema, different enforcement of structured output, different handling of the system prompt, different prompt-caching mechanics, differently (un)supported temperature. Anthropic itself warns that its compatibility layer is intended mainly for testing and comparison, not as a production solution, and openly admits what gets lost in it: system messages are merged into one, schema enforcement in tool calls is ignored, prompt caching does not work. A gateway therefore unifies the eighty percent that is common; the remaining twenty, where the vendors differ, is exactly where their competitive advantages live — and your migration risk with them.

Two more things belong to the interface and get forgotten. Prompts are not part of the code, they are versioned artifacts — they belong in a registry (Langfuse, Portkey and others), where a movable production tag points at a version and the code references only that tag, so that both promotion and rollback happen without a deployment. And embeddings are the hardest portability case of all, because the vector space is bound to a specific model: a query through model B simply will not find vectors stored by model A. The name, version and dimension of the embedding model therefore need to be pinned in the configuration, and every change to it treated as a schema migration.

This is where it becomes clear why a nice gateway alone is not enough. Switching an application from one model to another is something a gateway does trivially — it is a change of weight on a route, not a new deployment. That is the essence of blue/green deployment as Martin Fowler described it in 2010: two environments as similar as possible, instant switchover, instant rollback. Blue is the old model, green the new one, and the gateway is the switch.

But a paragraph was later added to that pattern which is absolutely fundamental for AI systems: the database has to be changed before the application, and in such a way that during the transition it serves both the old and the new version at once. Only then do you have somewhere to fall back to. And search built on a vector index cannot satisfy that condition in principle — old and new vectors lie in incompatible spaces, as I described above. A blue/green application without a blue/green index is an illusion of safety.

The solution is a two-index architecture, designed in from the start. The new index is planted alongside the old one: a background batch job recomputes the entire corpus with the new model while the old index keeps serving live traffic. Newly arriving documents are written to both. On a separate set of queries you verify that the new index has degraded neither the completeness nor the relevance of the results. And only then do you switch — atomically, by flipping a pointer: in Elasticsearch or OpenSearch with an alias, in Milvus by swapping the collection, over pgvector by swapping the table or the view. No query ever sees a half-recomputed index. The dominant cost here is not storage, but a single compute pass over the whole corpus. It sounds like extra work; it is the same work you would do for a database schema migration, just applied to the place where it is usually forgotten.

The switch itself should then not be a leap. The proven sequence runs from offline evaluation on a golden set, through shadow traffic, where the new model receives a copy of production requests and its outputs are logged but served to nobody, through a canary with a gradually growing share of traffic, all the way to a full switchover, with the old model kept warm in case of a rollback. SageMaker has shadow testing built in directly and handles the promotion of a shadow variant to production under the hood precisely as a blue/green, with no loss of availability. In the machine learning world this pattern even has its own name: champion/challenger. In production there is always just one winner, serving all the traffic, while the challengers run in the shadows — a continuous contest, not a one-off migration. Its by-product is exactly what you need at the moment of deprecation: a fresh comparison of the old and the new model on live data, not a supposition.

Even if you built a perfect gateway, kept prompts versioned and the index behind a movable pointer, you would still be left with a question no layer of code will answer: who breaks? Imagine forty applications, ten owners and one shared embedding model that is due to be mothballed. This is where large companies come apart, while small ones do not feel it — not because they do it better, but because their whole stack fits inside one person's head.

In its engineering blog on the Michelangelo platform, Uber published figures in 2024 that deserve a moment of silence: roughly four hundred active machine learning projects, over twenty thousand training runs a month, more than five thousand models in production, and ten million real-time predictions per second at peak. Five thousand models will not fit in anyone's head. Uber got them under control by having the platform — not the individual applications — know which model runs where, and by building it out of pluggable components that can be swapped.

A model registry tells you what exists. It does not tell you who calls it. That is what a service catalogue is for — the best known being Backstage, which originated at Spotify and has since been donated to the CNCF. Its software catalogue tracks ownership and metadata for all software, explicitly including services, websites, libraries, data pipelines and machine learning models, and assembles a dependency graph out of them. That is exactly the list the team from the opening story did not have.

The second layer is documentation of the model itself. Model cards were described by Margaret Mitchell and co-authors in 2019 at the FAT* conference (arXiv:1810.03993): a standardized sheet of nine sections stating the intended use, metrics, limitations, who created the model and in what version. Today it is the backbone of governance and, among other things, an input for the obligations arising from the European AI Act. A model without a model card is, in 2026, roughly what a library without a README was in 2010.

The third layer is the one most companies do not have, and yet it is the most effective. It is called consumer-driven contracts, and it was formulated by Ian Robinson in 2006 in the text Consumer-Driven Contracts: A Service Evolution Pattern — published on Fowler's website, which is incidentally why its authorship is so often misattributed. The principle runs against intuition: the contract is not defined by the provider; instead, each consuming application publishes the part it actually depends on. From those partial contracts the provider then sees what it may change and what it may not. Robinson also named the disease this cures: without contracts, the provider and all of its consumers have to jump at once. The tool that does this in practice is called Pact, and it has a command with a telling name, can-i-deploy — a gate in CI that answers the question "may I retire this model version without breaking anyone?" mechanically, not by email.

One last piece remains: how to move those forty applications from the old model to the new one when each is owned by someone else and nobody has the time. The answer has the same author as blue/green and is older still, from 2004 — the strangler fig. In nature that tree grows over its host, gradually replaces it, and in the end an empty space is all that is left of the original trunk. In software it means putting a façade in front of the old model (which is, incidentally, the same gateway discussed above), and that façade then routes each application either to the old model or to the new one. You migrate one at a time. When the last one moves across, the old model is unplugged and nobody notices, because nobody calls it any more. The pattern catalogues of both big clouds agree, moreover, that for as long as the two models coexist, the fig needs to be combined with an anti-corruption layer — in the interim the old and the new must live side by side without poisoning each other.

Where your company stands in all this can even be measured. In its text on MLOps, Google distinguishes three levels — from level zero, where everything is manual, through an automated ML pipeline to full CI/CD automation. Microsoft has a five-stage maturity model in its architecture centre, from "no MLOps" to "fully automated operations". There is no point pretending you can leap from zero to four. But there is a point in knowing which rung you are standing on, because that determines whether your next investment is a catalogue or contract tests.

Now, honestly, against everything above. There are two objections and both are strong.

The first concerns architecture. It does not say "that is too much work"; it says "that is the wrong work". Anyone who abstracts to a portable interface voluntarily gives up the vendor's superpowers — efficient prompt caching, native tool calling, reasoning modes, batch APIs, bespoke fine-tuning. The sharpest formulation came from one financial-AI vendor in a corporate blog post about vendor lock-in: "model-agnostic" effectively means "suboptimal on every model". The source has an obvious commercial interest in that thesis, but that does not make it false. Build on the lowest common denominator and you get a ceiling set by the weakest of your vendors. Added to that is the myth of portable prompts: a prompt tuned for one model often falls apart on another — not because it is bad, but because it counts on shared context that does not hold elsewhere. Migration is not free even with a perfect gateway. And the gateway itself is a new dependency: it adds a network hop and latency, it is another service to operate, and in the extreme it becomes a new lock-in, just one floor up. For a startup that needs to be first to market, a deliberate deep hook-up to a single vendor can be a rational bet. The advice "never use vendor-specific features" is simply wrong for such a company.

The second objection concerns governance and is just as unpleasant. Catalogues go stale. A year after it is introduced, the mandatory "owner" field holds the name of someone who no longer works at the company. A model card gets written once, at deployment, and then never again. Contract tests are another set of tests to maintain. And the whole of governance is a tax that yields nothing until things are on fire — and when they are on fire, people reach for switching things off and waiting to see who screams anyway. Manual metadata is theatre, because real accountability does not arise from a form.

Both objections are largely true and there is no point going around them. An answer to them exists, and in both cases it is an answer about degree, not about principle.

With architecture, the difference is between a thin layer and a maximalist one. The goal is not to build a universal adapter that forbids anything specific. The goal is a good-enough seam — one where you can change your mind later. That this is not academic was shown again in early July 2026 by DoorDash: because the architecture of its code reviewer is model-agnostic, both the scout model and the reviewer model are independently swappable variables. When they ran them against a hundred real pull requests from their own repository, it turned out that their production line-up was not the best one — the winning combination, with an open-source model in the scout role, found more genuine issues and came out somewhat cheaper. The company itself points out that the numbers hold for its code and its criteria. But the point stands: anyone locked in to a single vendor cannot even ask the question. The objection "suboptimal on every model" holds for an architecture without measurement. With measurement it turns into its opposite.

Even so, I do not want to brush it aside entirely. When some vendor feature is decisive for a given task, by all means use it — just record the resulting lock-in as a deliberate, bounded and priced cost, not as an accident you discover at shutdown time. It can be managed like technical debt: calculate the price of exit, keep it within limits, and pay it down on purpose.

With governance, the difference lies in whether the data is derived from reality or filled into a form. Who is the owner? Whoever commits to the repository — pull it from git, not from a spreadsheet. Who calls the model? Whoever shows up in the traces and in the gateway's access logs — generate the dependency graph from telemetry, not from a questionnaire. What do the consumers depend on? On whatever passes their contract test in CI, not on what they think. A catalogue that is a mirror of operations cannot go stale, because it has no way to. A catalogue filled in out of obligation always will. (Backstage itself advises the opposite in its documentation: automation should apparently be used only to generate the first file, and from then on humans should maintain the metadata. I hold that the difference lies in the nature of the data. What follows from fact — who commits, who calls, what passes a test — should be derived, and must not be overwritable by hand. What follows from intent, that is, what the model is actually for and where it must not be used, has to be written by a person, who then has to accept that it will age.)

The team that switched off a thousand applications and heard nobody scream got lucky. A company that unplugs its old model painlessly in a few years' time and frees up the compute will not need luck — it will have a dependency graph from telemetry, versioned prompts, an index behind a movable pointer, a contract in CI, and a gateway on which switching models is a matter of one line of configuration.

So one last question, and it is uncomfortably concrete: try, right now, to guess how many applications in your company call your oldest deployed model. Then go and find out. The difference between that guess and that number is exactly the size of the problem you will one day have to pay for.

The text was produced on the basis of in-depth research into primary sources (vendor documentation, engineering blogs, academic papers, canonical descriptions of design patterns) and was subsequently re-verified sentence by sentence.

Time-sensitive data. The deprecation periods of the individual providers, the retirement date of Claude Opus 3 (5 January 2026), the extended-access mode in Bedrock and the figures on the volume of models on Uber's Michelangelo platform (2024) come from those companies' public materials and change on a scale of months. Everything was verified as of 11 July 2026; if published later, it must be checked against current documentation again.

What is not documented. The story in the opening circulates in the operations community and I was unable to trace its primary source; I give it without identifying the company, as an illustration of the method, not as a documented case.

What is disputed. The question of portability versus deliberate lock-in is strategically contested and there are strong arguments on both sides. The formulation that "model-agnostic" means "suboptimal on every model" comes from the corporate blog of a financial-AI vendor, that is, from a source with an interest in the outcome; in this text it therefore appears as the strongest form of the objection, not as a documented finding. The numbers from DoorDash's benchmark hold for its own code and its own criteria; the company itself stresses this, and they cannot be transferred elsewhere.

Key primary sources. Sculley, D. et al.: Hidden Technical Debt in Machine Learning Systems, NIPS 2015 (papers.nips.cc) — the concept of undeclared consumers. Evans, E.: Domain-Driven Design (2003) — Anti-Corruption Layer; Microsoft Azure Architecture Center, Cloud Design Patterns — a modern description of the pattern. Fowler, M.: BlueGreenDeployment (2010) and StranglerFigApplication (2004), martinfowler.com. Robinson, I.: Consumer-Driven Contracts: A Service Evolution Pattern (2006), martinfowler.com; Pact — documentation of the can-i-deploy command. RFC 8594 (The Sunset HTTP Header Field, E. Wilde, May 2019) and RFC 9745 (The Deprecation HTTP Response Header Field, S. Dalal and E. Wilde, March 2025), datatracker.ietf.org. OpenAI — Deprecations (developers.openai.com); Microsoft Learn — Azure OpenAI model retirements; AWS — Bedrock model lifecycle (the modelLifecycle field, extended-access mode from 1 February 2026) and SageMaker shadow testing; Anthropic — Commitments on model deprecation and preservation and An update on our model deprecation commitments for Claude Opus 3, plus OpenAI SDK compatibility (platform.claude.com). MLflow — Model Registry documentation (stages superseded since version 2.9 in favour of aliases); Google Cloud — Vertex AI Model Registry, How to use model version aliases. LiteLLM (github.com/BerriAI/litellm), Cloudflare AI Gateway, Kong AI Gateway, Portkey, MLflow AI Gateway, Vercel AI SDK. FutureSearch: LLM API Differences That Break Your Code (futuresearch.ai). DoorDash Engineering: How DoorDash built an AI code reviewer engineers actually listen to and How we learned to trust our AI code reviewer at DoorDash (6 July 2026). Backstage — Software Catalog documentation (backstage.io), project donated to the CNCF; Mitchell, M. et al.: Model Cards for Model Reporting, FAT* 2019, arXiv:1810.03993. Uber Engineering: From Predictive to Generative — How Michelangelo Accelerates Uber's AI Journey (2024). Google Cloud: MLOps: Continuous delivery and automation pipelines in machine learning — levels 0 to 2; Microsoft: MLOps maturity model — five stages.

The concept, structure and editorial line of the article are the work of the author, who prepared the content outline, established the key theses and directed the entire creative process. Generative AI (Claude, Anthropic) was used as a tool for research, for locating primary sources and for developing the author's content outline into finished prose.

The author edited the outputs throughout, verified the key findings and approved the final wording. No part of the text was published without human review. All factual data were verified against the publicly available sources cited in the text.

The procedure complies with the transparency requirements for AI-generated content under Article 50 of Regulation (EU) 2024/1689 (the AI Act). #poweredByAI