Why A Working AI Pilot Proves Almost Nothing

A working pilot and the production system meant to replace it share a model and almost nothing else. That is not a complaint about scope creep. It is a description of what the two artifacts are actually for — and it is why hardening a pilot is structural work rather than a tuning pass.

The pilot exists to answer one question — can the model do the thing? — and it is allowed to answer under curated data, predictable load, and a builder who is always in the room. Production asks the opposite question. What happens when the model cannot do the thing, on a record nobody is watching, inside a system that has to keep its promises to a downstream consumer anyway? None of the pilot's architecture was built to answer that, because it was never asked. If the question is not how to cross this gap but which pilots deserve the crossing at all, the decision framework for pilot selection covers that first.

Research on hidden technical debt (NeurIPS, 2015) named the shape of this two decades into the practice: the machine-learning code is a small fraction of a real-world ML system, and the surrounding infrastructure is the rest. A survey of ML deployment challenges (ACM Computing Surveys, 2022) catalogs where practitioners actually stall, and it is at every stage except the one the pilot rehearsed. ML LABS engineered the backend of a medical-device cloud platform that reached clinical production in two countries — the cloud backend behind HeartSciences' ECG management platform, with AIM Consulting building the frontend. Almost none of the difficulty in that system was in the models. It was in everything that had to be true around them.

What Production Asks Of A Pilot

Production asks three questions the pilot never had to face, and every one of them is answered in infrastructure rather than in weights. What happens on failure? Who learns that a result was produced? And what changes when the second customer arrives?

Answering those questions changes which numbers you watch. On the ECG platform, the signals that mattered in production were never the aggregate accuracy figures on a model vendor's data sheet. They were per-provider tail latency at the 95th and 99th percentile, retry rates broken down by failure class — network, malformed response, model error — and the share of studies where two providers disagreed on a finding a clinician would act on. A pilot has no reason to instrument any of that, because a pilot has no failure classes worth separating and no second provider to disagree with the first.

There is a fourth question hiding behind the three, and it is the one that makes the others permanent: how do you ever change the model again? A pilot swaps models by editing a notebook. A live system needs a way to run the candidate beside the incumbent and compare outputs without serving them, which is why frameworks for shadow deployment (Microsoft Research, 2024) belong in the architecture rather than in a later cleanup. Build the swap path early and model improvements stay cheap; skip it and every future model change reopens the same negotiation with production.

This is the honest reason the transition is expensive. The work is not concentrated in one hard problem you can staff against; it is distributed across the ingestion path, the retry semantics, the delivery contract, and the per-customer configuration surface, none of which existed before. The three sections below take the questions in order, because they are the load-bearing ones — and the same three decide whether an AI feature is genuinely build-ready long before any of them get written.

Design For The Terminal State

The failure question has one good answer, and it is a design property rather than an operating metric: every record reaches a definite terminal state, and that state is queryable. Not "the pipeline is reliable" — that is a hope with a percentage attached. A record that entered the system either completed, failed permanently, or is sitting in a named state that a human can be asked to resolve, and you can list every record in each bucket without opening a log aggregator.

That property is not free, and it is not achievable retroactively. It forces decisions at design time about idempotency, about what a retry is allowed to re-do, and about which side effects are permitted to fire more than once. On the ECG platform, each inference request carries an idempotency key, so a retry after a provider timeout never produces a second billable result for the same study, and reconciliation jobs catch any provider invoice that disagrees with the platform's own ledger before it reaches the customer. Billing is the unforgiving case — a duplicate charge is a customer-visible error that no amount of model quality excuses — but the same discipline governs anything a retry can duplicate: a delivered message, an alert, a downstream write.

Terminal states also require an opinion about what should never have entered the system at all. A malformed record that the model cheerfully scores is not a success, and it does not announce itself: research on data validation for ML (MLSys, 2019) demonstrates that systematic input validation catches anomalies that would otherwise degrade model output without surfacing an error anywhere. Rejection at the door is itself a terminal state, and it is a far cheaper one than a plausible wrong answer travelling downstream.

Every record reaches a definite terminal state, and that state is queryable. That is a property you design in before the first retry is written — not a metric you measure afterward, and not a dashboard bolted on after the fact.

graph TD
    A["Record<br/>Submitted"] --> B["Inference<br/>Dispatched"]
    B --> C{"Provider<br/>Responds?"}
    C -->|"Yes"| D["Result<br/>Reconciled"]
    C -->|"Timeout"| E["Retry With<br/>Idempotency Key"]
    E --> C
    C -->|"Exhausted"| F["Failed<br/>Permanent"]
    D --> G["Complete"]
    F --> H["Queryable<br/>Terminal State"]
    G --> H

    style A fill:#1a1a2e,stroke:#0f3460,color:#fff
    style B fill:#1a1a2e,stroke:#0f3460,color:#fff
    style C fill:#1a1a2e,stroke:#ffd700,color:#fff
    style D fill:#1a1a2e,stroke:#16c79a,color:#fff
    style E fill:#1a1a2e,stroke:#ffd700,color:#fff
    style F fill:#1a1a2e,stroke:#e94560,color:#fff
    style G fill:#1a1a2e,stroke:#16c79a,color:#fff
    style H fill:#1a1a2e,stroke:#16c79a,color:#fff

The reliability engineering behind that inference layer is the fullest version of this argument, and the shape generalizes well past clinical systems. The same question governs a crowd-question summarizer that collapses hundreds of inputs into a few answers and a signal extractor pulling from noisy open-web sources, where every adapter owns its own backoff and its own definition of a permanent failure. Scale changes the cost of getting it wrong. It does not change the requirement.

The Integration Surface Is Per-Site

The delivery question is the one a pilot cannot even pose, because the pilot's consumer is a notebook and production's consumer is somebody else's system with its own opinions. In the HL7 work on the ECG platform, the wall was that every hospital's accepted message differs on details the specification calls optional — and the specification is not the authority on what the receiving system will take.

Rejections at that boundary are quiet. A major EHR vendor's interface engine returns an application acknowledgment carrying AE or AR status in MSA-1, the result lands in an error queue, and the ordering clinician simply sees nothing arrive. The causes are mundane, and each one is a single field: a patient identifier that includes or omits leading zeros, an ordering provider reference pointing at a retired NPI, an observation identifier using a LOINC code the receiving system has never mapped to a local result type, an order control code signalling a workflow it does not support. Each looks like a configuration footnote during setup. In aggregate they are what stands between a correct AI result and a clinician who can act on it, which is why the signals worth instrumenting are the ones the hospital's own interface team reads when triaging — NACK status codes in returned MSA segments, ERR segment field-level error locations, and counts of messages stuck past their expected processing time. The result-delivery system is built around exactly those returns.

The second-customer question has the same answer as the first, arrived at from a different direction. The ECG platform was designed from the start so that onboarding a new organization is configuration, never code — HL7 field mappings, enabled AI models, invoice pricing, and storage provisioning are all per-organization settings, and validation refuses to construct a message at all when a required per-org value is missing, so a misconfiguration fails loudly at the boundary instead of producing a plausible wrong message. That is a design-time judgment, made before the second customer existed, and it is the difference between a system that grows by configuration and one that grows by fork. It is also what makes a separate test, US production, and UK production deployment tractable rather than a maintenance tax.

The Missing Operational Owner

None of this survives launch without someone accountable for the system afterward. That is the real boundary condition on every architecture in this article: a production AI system with no owner degrades along axes nobody is watching — models drift, costs creep, and failures stop being reported because there is no one to report them to. Drift is a solved problem on paper, and statistical tests for detecting dataset shift (NeurIPS, 2019) are well characterized and available off the shelf. Someone still has to run them, read them, and act on what they say, and that is a standing job rather than a launch task. The engineering above buys you a system whose decay is visible.

It does not buy you someone looking.

What that costs is on the record. On a hedge-fund engagement, unnecessary and polluted data had accumulated unnoticed while the aggregation step was quietly discarding information the models needed; once someone actually looked, storage costs came down by more than 60% and the models performed 2% better. The full account of why systems compound only under ownership sits with the ownership argument, and it is the case to read before deciding that launch is the finish line. The same gap explains why security posture on live AI systems is an operating discipline rather than a pre-launch checklist.

First Steps

1. Enumerate your record states and mark the terminal ones. List every state a record in your pilot can occupy, then ask which ones a record can enter and never leave. Those are the states that will fill up in production.
2. Instrument the rejection path, not the happy path. Capture what the downstream system says when it refuses your output — its error codes, its field-level locations — not what your own service logged when it thought it succeeded.
3. Move one hardcoded per-customer value into configuration. Then make the system refuse to start without it. The refusal is the point: it converts a silent misconfiguration into a loud one.

Harden The Path To Production

The path across the gap is not a rewrite and it is not a tuning pass. It is a sequence: give every record a terminal state, instrument the boundary where your output meets someone else's system, and externalize the per-customer values before the second customer forces you to. Do those three and the model becomes the easy part of the system, which is what it was supposed to be. Skip them and no amount of accuracy will make the pilot deployable, because none of the three failures they prevent are model failures.

Where a team lands on that sequence is the honest signal for what to do next. A pilot with a known terminal-state gap and a downstream consumer already waiting is ready for a production workflow build, which takes one contained workflow the whole distance — serving, retry semantics, integration contract, and configuration surface — and carries its first 30 days of operation with it, because that is when the design decisions above meet reality. A pilot whose consumer is several systems rather than one is an integration problem before it is a build problem and should be scoped as one; the property valuation engine, the roaming optimization platform, and the network optimization work behind it each reached production through a different door, and deploying enterprise agents adds another.

ML LABS has taken more than ten heavy-workload systems to production over fifteen-plus years, and the argument of this article is what that work is made of. Terminal states, instrumented boundaries, and externalized configuration are not hardening chores to be survived once and forgotten. They are the operating surface of the system you are about to own, and the engineering that carries a pilot across the gap is the same engineering that keeps it standing on the other side.

References

1. Sculley, D., et al. Hidden Technical Debt in Machine Learning Systems. NeurIPS, 2015.
2. Paleyes, A., Urma, R.-G., & Lawrence, N. D. Challenges in Deploying Machine Learning: A Survey of Case Studies. ACM Computing Surveys, 2022.
3. Breck, E., et al. Data Validation for Machine Learning. MLSys, 2019.
4. Rabanser, S., Günnemann, S., & Lipton, Z. Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift. NeurIPS, 2019.
5. Microsoft Research. Experimentation Platform (ExP). Microsoft Research, 2024.