Production begins after deployment: operating an enterprise AI product

A successful deployment proves that the deployment worked. It does not prove that the AI product is accurate, safe, affordable, supportable or valuable. Those are things you find out afterwards, continuously, and only if the product was built to tell you.

Production readiness needs evidence across four dimensions:

1. Service health: availability, latency, errors, capacity and the health of dependencies.
2. AI quality: task success, groundedness, correctness, safety and escalation.
3. Business outcome: adoption, time saved, completion, resolution or another agreed measure.
4. Control health: access reviews, policy compliance, audit completeness, data retention and change control.

Most AI dashboards cover the first dimension well and the other three barely at all.

Fynbos Health Administrators is fictional, and its systems and figures are illustrative. By now Fynbos has three products built on the boundaries, network paths, identities, retrieval controls and authority envelopes from the earlier parts. This part follows them into operation: a phased go-live, a model upgrade that went wrong in one language, and the operating model that caught it.

A phased go-live

Fynbos did not switch everything on at once:

• The Policy Assistant went to all staff, after the disciplinary-records fix from Part 4 passed its negative access tests.
• The Member Desk copilot went to one Durban team of 40 contact-centre agents first, with results compared against the same team's previous month, then to the whole contact centre.
• The Claims Query agent ran read-only for its first month, then drafting, and only then credits.

Each step had an exit criterion agreed in advance by the business owner. None of them was "the deployment succeeded".

Observability has layers, and each has an owner

On the Foundry side, at the time of writing (October 2026), OpenTelemetry-based tracing into Application Insights is generally available for prompt and hosted agents. The agent monitoring dashboard's metrics, continuous evaluation of production traffic, red-team scans and alerts are in preview. Microsoft's agent monitoring dashboard guidance gives the current state. Plan your production monitoring on the generally available parts and treat the rest as useful additions that may change.

Trace data is sensitive data

Traces can contain prompts, retrieved passages, member details, tool arguments and outputs. Logging everything makes debugging easier and creates a new store of sensitive data that nobody designed.

Foundry is cautious by default. Tracing is off until you enable it, and capturing message content is off even when tracing is on; Microsoft advises enabling content capture only in development unless your compliance position allows it. Fynbos traces every request in production with message content off, and enables content capture only in its test environment, against synthetic member data. Decide redaction, sampling, retention, access and export rules before you turn detailed tracing on, not after the first incident shows you what it captured.

Evaluation is part of release management

Evaluation is not a gate a product passes once.

Before deployment, run the complete golden dataset, safety tests, restricted-access cases, adversarial inputs and performance tests.

During controlled rollout, use internal users, shadow traffic, a limited cohort or a narrow task scope, and compare results with the existing process. Fynbos's Durban team was the cohort.

In production, sample real interactions for evaluation, respecting privacy and access controls, and watch for drift in topics, retrieval sources, languages and user behaviour.

After any material change, re-run the relevant set. Material changes include the model or its version, the system prompt or policy, a retrieval source, chunking or index, a tool description or API, a safety threshold, the orchestration framework, a gateway routing rule, and the user channel.

Here is how that worked for Fynbos. Model versions do not last forever: at the time of writing, Microsoft gives at least 60 days' notice before retiring a generally available model, and a typical lifecycle is about 18 months from general availability. So model upgrades are a routine event, not an exception, and need a routine process. A planned upgrade of the copilot's model passed the full English evaluation set. Two days into production, sampled evaluation showed answers about co-payments in Afrikaans getting worse: correct in substance often enough, but wrong on which plan option a co-payment applied to. The product owner rolled the deployment back within the day. The evaluation set gained more Afrikaans and code-switched co-payment cases, and the upgrade went out again two weeks later, after passing them.

"The new model is better" is not release evidence. The changed system has to pass the product's own agreed tests. More on building those in Evaluating AI agents.

Service levels the platform can actually influence

A single "AI accuracy SLA" hides several different things. Separate them:

• platform availability;
• model endpoint availability;
• end-to-end response time;
• retrieval success;
• task completion;
• correct-answer rate on an approved test set;
• time to human handover;
• incident response and recovery.

Do not promise a universal percentage, such as "98% accurate", without defining the dataset, scoring method, task class, confidence interval, exclusions and what happens when it is missed. A benefits FAQ, a clinical recommendation and a financial transaction need different thresholds, and Fynbos sets them per task: its copilot's threshold for co-payment answers is stricter than for general benefit descriptions, and its Claims Query agent's credit tool has no accuracy target at all, because it is governed by its envelope, not by a percentage.

AI incidents

AI products fail in familiar ways and new ones:

• model or dependency outage;
• throttling or quota exhaustion;
• retrieval of restricted information;
• a harmful or materially incorrect response;
• an unauthorised tool action;
• a successful prompt injection;
• unexpected behaviour from a new model version;
• a cost spike;
• sensitive information in logs;
• a corrupted knowledge source or stale index.

Runbooks should cover detection, containment, rollback, stakeholder notification, evidence preservation and recovery. AI products have some useful containment actions that ordinary applications do not: disable one tool; route to a safer model; switch to read-only; raise the approval threshold; remove a knowledge source; or suspend the product and fall back to the human process. Fynbos's runbooks name the person who may take each action, and the Claims Query agent's credit tool can be disabled independently of the rest of the agent.

The whole bill

Token cost is one line on the bill. Include all of them:

• model input, output and cached-token consumption;
• provisioned capacity, where used;
• API Management as the AI gateway;
• search indexing and query capacity;
• storage, Cosmos DB and any cache;
• application compute;
• private endpoints, firewall and network processing;
• logging, traces and retention;
• evaluation and red-team runs;
• content safety calls;
• build agents and deployment environments;
• human review and support effort.

Measure cost per useful outcome, not cost per token. Fynbos reports cost per resolved member query: everything above, divided by the queries that did not need a callback. A more expensive model that resolves a query correctly the first time can cost less than a cheaper one that causes rework or a second call. AI observability and cost and Governing the AI bill go further into allocation and ownership.

Infrastructure as code is necessary, and not governance

Terraform, Bicep or another controlled tool should create repeatable environments, and the code should cover more than whether services exist: the resource and project structure, networks and private endpoints, diagnostic settings, managed identities and role assignments, policy assignments, model deployment configuration where supported, tags and cost allocation, alerts and budgets, and environment-specific parameters.

But repeatable deployment does not decide which answers are acceptable, whether processing is lawful, or what an agent may do. Keep prompts, evaluation datasets and tool schemas versioned with their own review paths, owned by the people accountable for them. Infrastructure code cannot express every product policy, and presenting it as if it did is one of the most common gaps in AI proposals.

The minimum operating model

One person may hold several roles in a smaller organisation. The decisions still have to be named, and every alert and evaluation threshold needs an owner and a runbook. Central monitoring that nobody is accountable for interpreting is a log archive, not operations.

The reference model, in one picture

The six parts of this series fit together as seven layers.

Each view of the architecture should prove something about these layers to a particular audience, as Part 1 set out: the executive view proves value and accountability; the identity view proves authority transitions; the data and RAG view proves what the AI may know; the agent action view proves what it may do; the operations view proves who notices and responds.

Any proposal or design for an enterprise AI product should be able to answer five questions:

1. What business capability are we delivering?
2. Which components are mandatory, which optional, and which already supplied?
3. How will data, identity and actions be controlled?
4. Who owns the platform, the product, the knowledge and operations?
5. What evidence will prove production readiness?

The production acceptance checklist

Business and scope

• A named business owner accepts the defined use case and its exclusions.
• Success measures and prohibited outcomes are documented.
• Human escalation and an alternative service path exist.

Architecture

• Platform, product and landing-zone boundaries are explicit.
• Mandatory and optional components are separated.
• Data, identity, network and action flows are traceable.
• Resilience, recovery and dependency failure are tested.

Security and privacy

• Least-privilege identities and access reviews are in place.
• Public access, private endpoints, DNS and egress are verified.
• Data classification, retention and trace handling are approved.
• Processing locations and cross-border implications are recorded.
• Prompt injection and tool abuse have been tested.

Quality and responsible AI

• Golden datasets have domain-owner approval.
• Quality and safety thresholds are task-specific.
• Restricted-access and multilingual tests are included where relevant.
• Model, prompt, retrieval and tool changes trigger re-evaluation.

Operations and cost

• Dashboards cover service, quality, control and business signals.
• Runbooks and named support responsibilities exist.
• Cost allocation, budgets and anomaly alerts are configured.
• Model retirement and replacement have an owner.

Common mistakes

• One percentage for AI accuracy. Without a test set, scoring method and task context, the number means nothing.
• Central monitoring with no operational owner. Every alert and threshold needs a person and a runbook.
• Infrastructure as code presented as governance. It makes environments repeatable; it does not decide what is acceptable.
• Treating a model upgrade as a configuration change. It is a release, and it re-runs the evaluation set.
• Turning on full trace content in production by default. You have just built an unplanned store of personal information.
• Counting only tokens. The gateway, search, network, logging, evaluation and human review are on the bill too.

What to take from this part

Deployment is one event inside a continuing agreement. Production is the agreement.

Observe seven layers, each with an owner. Service health alone is not readiness.

Evaluation is release management. Any material change goes back through the gate, in every language the product serves.

Measure cost per useful outcome. Not cost per token.

Name the decisions. One person may hold several roles; no decision may be unowned.

That completes the series. The platform pattern it started from, Gajanan Hatkar's reference architecture discussed in Part 1, supplies the technical foundation. Microsoft Foundry supplies governed resources and projects, model deployments, agents, evaluation and tracing; the landing zone supplies connectivity, policy and operations; API Management applies shared runtime controls; AI Search can trim retrieval by permission. None of them can own the business outcome. The organisation still has to decide what the AI may know, what it may do, which failures are acceptable, when a human must step in, how quality is measured and who carries responsibility after go-live. That is the difference between deploying AI services and building an AI capability.

Start again from Part 1, or see every part on the series page. For a broader catalogue of production concepts, see the agentic AI production field guide.

How CloudNala can help

We run production readiness reviews for AI products built on Microsoft Foundry and similar platforms: checking the evidence behind each line of the acceptance checklist, designing observability and trace handling that respects the data, setting task-specific quality thresholds with your domain owners, writing AI incident runbooks, and building cost-per-outcome reporting. We use the same structure in proposals and architecture workshops, so the design is argued from evidence rather than from a service catalogue.

Work with CloudNala

CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.

Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.

Book an AI Production Readiness Review or write to us at consult@cloudnala.co.za