How To Move AI Forward In Regulated Work

An analysis of HL7 v2 optionality (JAMIA, 2009) counted 4,132 data elements in the standard clinical result message, with 85% of them designated optional. That single number says something uncomfortable about regulated software: the specification does not tell you what correct looks like. It tells you what is permitted, and the actual bar is set by the receiving system, the auditor, and the clinical consequence of getting it wrong.

Which is why AI in regulated systems is misdiagnosed as a contradiction. It is not. The contradiction is running AI with startup-style ambiguity inside an environment that depends on evidence, change control, and recoverable decisions. Regulation is not what kills the initiative — weak operating discipline is, and it does its damage by treating the regulated environment as a late-stage review problem rather than an early-stage design constraint.

ML LABS engineered the backend of a medical-device cloud platform that reached clinical production in two countries: the cloud ECG backend built for HeartSciences, with AIM Consulting building the frontend. That system runs across three environments — test, US production, UK production — multi-tenant, with per-organization configuration and data residency separation, and every design decision in it carried regulatory weight before it carried engineering weight. What follows is the execution model that made that possible, stated as three control layers, each of which closes a failure mode the other two cannot.

1. Evidence layer that records data assumptions, evaluation results, and decision rationale.
2. Change layer that governs how models, prompts, rules, or workflows are modified over time.
3. Operating layer that defines human review, escalation, monitoring, and rollback in production.

graph TD
    subgraph Evidence["Evidence Layer"]
        A1["Data provenance"]
        A2["Decision rationale"]
        A3["Evaluation results"]
    end

    subgraph Change["Change Layer"]
        B1["Change taxonomy"]
        B2["Approval gates"]
        B3["Re-validation triggers"]
    end

    subgraph Operating["Operating Layer"]
        C1["Human review points"]
        C2["Escalation paths"]
        C3["Rollback triggers"]
    end

    Evidence -->|"What can change"| Change
    Change -->|"How it runs"| Operating
    Operating -->|"New evidence"| Evidence

    style Evidence fill:#1a1a2e,stroke:#16c79a,color:#fff
    style Change fill:#1a1a2e,stroke:#ffd700,color:#fff
    style Operating fill:#1a1a2e,stroke:#0f3460,color:#fff

Evidence Before Scale

The first question in a regulated environment is not "can the model work?" It is "what evidence would make this system acceptable to operate?" That evidence includes data provenance, evaluation boundaries, performance on known edge cases, and the reasoning behind the acceptance threshold — and it is an engineering artifact, not a documentation exercise. Evidence reconstructed after a build has accelerated is thinner and harder to defend than evidence designed into the execution path, for a structural reason: the facts that make an audit trail useful (what the system state was before a change, and why the change was made) are only available at the moment of the change.

The AI Risk Management Framework (NIST, 2023) and the Generative AI Profile (NIST, 2024) both treat governance, traceability, and monitoring as operating requirements rather than as reports. Good Machine Learning Practice guidance (FDA, 2021) and predetermined change control plans for ML-enabled devices (FDA, 2025) point the same way: if the system can change, the change path must be characterized before scale. And the proposed 2025 healthcare security rule updates (Federal Register, 2025) eliminate the distinction between "required" and "addressable" controls, which makes comprehensive audit logging mandatory rather than aspirational.

What that looks like in a shipped system is specific. On the ECG platform, clinical reports move through a lifecycle that is signed, confirmed, and locked, and every post-signature modification is tracked with a full audit trail, because the report is a regulated artifact and its history is part of it. The property underneath is worth more than any metric: every record reaches a definite terminal state, and that state is queryable. A study is never "probably fine" or "still processing since Tuesday" — it is in a state a query can name, which is what turns reconciliation into a query instead of an act of heroism.

The evidence layer is not a report you write after the build. It is a system behavior you design before the first line of production code runs.

Change Control Before Drift

AI systems drift in more ways than ordinary software. Models, prompts, retrieval corpora, rules, and human review thresholds all change, and in a regulated setting each is a controlled change with downstream consequences. The change model has to exist before the system becomes operationally important: which changes are pre-approved, which require re-validation, which trigger rollback. Those are execution questions with engineering answers, not policy questions with committee answers.

Security hardening belongs inside this layer rather than beside it. On the ECG platform, security work ran inside the delivery cadence rather than being deferred to a hardening phase at the end: rotating secrets and least-privilege IAM, PHI kept out of logs and error responses, input validation across every externally reachable API surface, and per-organization session isolation with no fallback authentication path. Each finding is cataloged, severity-ranked, and remediated as a controlled change with its own evidence trail — the same discipline described in the security hardening of a live AI platform.

Fail-Safe By Default

The instinctive approach to sensitive data is to strip the fields known to be sensitive. That approach fails quietly when a new field appears that the filter does not cover, and in a regulated system a quiet failure is a compliance incident that nobody is paged for.

The stronger pattern inverts the default: define what data is permitted to pass, and block everything else. A new field is held until it is explicitly classified and approved. Guidance on de-identification (HHS, 2024) reinforces the shape — the safe harbor method works from a defined set of identifiers, not from open-ended exclusion logic that has to anticipate every future field.

Operating Controls In Production

A regulated AI system needs its production behavior specified as precisely as its model: who approves exceptions, who sees low-confidence output, how incidents escalate, and how the system reverts when behavior drifts. Without that, "human oversight" is a phrase in a slide rather than a control in a system — which is the distinction oversight of AI agents in production turns on.

• Authentication and session controls. Hardening, bounded session windows, and structured auth logging that supports reconstruction after the fact.
• Network and perimeter controls. Defense in depth, consistent access rules, and automated infrastructure validation.
• Multi-tenant isolation. Cross-organization access blocked at every layer as a system invariant, not an application convention.

Each integration point — authentication providers, clinical data systems, payment processors — needs its own audit trail, because each is a boundary where responsibility changes hands. SP 800-53 (NIST, 2024) requires organizations to generate, protect, and retain audit records for monitoring, analysis, and investigation. Every integration boundary is therefore a logging boundary, and the ones that are not are the ones that will be asked about first.

When Scope Is The Real Blocker

The layers do not help if the thing being controlled is undefined. If the workflow is still vague, if the risk owner is unnamed, or if the team cannot say what human review is required, then the problem is unresolved scope wearing regulation's clothes, and no control model resolves it. Scoping comes first, and it is a cheaper problem to solve than the one that follows it.

The other boundary is depth. If the system category implies a formal regulatory pathway the team has not navigated before, control design has to precede the build — not because regulation is exotic, but because the cost of discovering a required control after the architecture is set is measured in rework rather than in effort. The bottleneck in regulated AI execution is not headcount. It is having someone who understands both the control requirements and the system architecture well enough to decide without a round-trip, and that person is scarce in exactly the organizations that need the decision fastest.

First Steps

1. Name the first use case. Narrow enough that the evidence path can be designed concretely for every workflow transition it contains.
2. Define the change taxonomy. Which changes are pre-approved, which require re-validation, which trigger rollback — with fail-safe-by-default for anything touching sensitive data.
3. Write the control loop. Authentication, session management, logging, isolation, and rollback triggers, specified before the system becomes operationally important.

Design The Control Path First

The bar for a regulated AI system is checkable before anyone writes production code, and it is short. Every consequential action produces a durable audit record naming who acted, what changed, why, and what the state was before and after. Every category of change has a defined path: pre-approved, re-validated, or blocked. Every record has a terminal state a query can name. And sensitive data flows through an allowlist, so a field nobody has classified cannot leave the building. An organization that can point at those four properties in its architecture is ready to build; one that cannot is going to retrofit them under time pressure, which is the expensive version of the same work.

In a regulated environment the fastest way to settle this is to prove the control path on a real workflow before committing the budget: an AI system design produces the architecture, a written build plan, a fixed-price quote, and a working spike that runs inside your VPC against a de-identified or synthetic extract where production data cannot be granted — which is precisely the constraint regulated buyers assume makes a spike impossible. The full delivery story behind this execution model is the AI-ECG management platform, and the messaging layer that carries its results into hospital systems is the HL7 result delivery system.

References

1. National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST, 2023.
2. National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework: Generative AI Profile. NIST, 2024.
3. U.S. Food and Drug Administration, Health Canada, and MHRA. Good Machine Learning Practice for Medical Device Development: Guiding Principles. Regulatory Reference, 2021.
4. U.S. Food and Drug Administration. Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices. Regulatory Reference, 2025.
5. U.S. Department of Health and Human Services. Healthcare Security Rule — Strengthening Cybersecurity of Electronic Protected Health Information. Federal Register, 2025.
6. U.S. Department of Health and Human Services. Guidance Regarding Methods for De-identification of Protected Health Information. HHS, 2024.
7. National Institute of Standards and Technology. Security and Privacy Controls for Information Systems and Organizations (SP 800-53 Rev. 5). NIST, 2024.
8. Sujansky WV, Overhage JM, Chang S, Frohlich J, Faus SA. The Development of a Highly Constrained Health Level 7 Implementation Guide to Facilitate Electronic Laboratory Reporting to Ambulatory Electronic Health Record Systems. Journal of the American Medical Informatics Association, 2009.