Running AI Systems With the Discipline of a Release
Treats an AI system as a production system rather than a demo: a hard spend ceiling, deterministic validation between the model and the decision, approval boundaries on anything irreversible, and a defined refusal behaviour. That is production operations applied to AI, not prompt work.
What this is
Most AI projects die between the demo and production, and they die of the same causes a release dies of: nobody bounded the cost, nobody defined what happens when the output is wrong, and nobody decided who is allowed to spend money or change state without asking.
I have designed and validated six AI systems since July 2026. Four are live at real traffic. None of them is interesting because it uses a model. They are interesting because each one forced a delivery decision that had nothing to do with the model.
What I actually run
- A live assistant that reads the accounts a merchant has already connected and turns them into decisions, with a deterministic validation layer between the model and anything that changes or spends.
- A live commerce storefront on a self-hosted platform, with the payment path built on a licensed provider rather than improvised.
- A payment aggregator, where the integration surface is a regulated one and the failure modes are somebody’s money.
- A catalog site with search analytics, where relevance is measured rather than assumed.
- An internal company operating system — accounting, budgets, approvals, and model routing under a hard spend ceiling.
- Advertising integrations with Meta, Google and a marketplace network, feeding the assistant’s decisions.
How I operated
- Set the spend ceiling before the first prompt. Model routing runs under a hard cap. A system whose cost is unbounded is not a product; it is a bill with a user interface. Deciding which model handles which request is a trade-off between quality and money, and it belongs to whoever owns the outcome, not to whoever wrote the call.
- Put something deterministic between the model and the decision. A generated claim survives only if it traces back to captured data. On the merchant assistant that check is explicit: no tracing, no answer. On the reporting work it means the numbers are recomputed rather than repeated.
- Named what the system must refuse. Every one of these has a case where the right behaviour is to stop and ask, or to say nothing. A model that never declines has not been designed; it has been released.
- Gave every irreversible action an owner. Reading is free. Changing, sending, and spending surface to a human first. That boundary is drawn per system, and it is written down.
- Validated against reality, not against vibes. Live traffic, real failures, real cost. A system that works in a demo and degrades under load is not finished, and the only way to know is to watch it with real users on it.
What it changed about how I think about delivery
I came to AI from release and launch governance, and the surprise was how little translated from the model side and how much translated from the delivery side.
Cost ceilings are capacity planning with a different unit. A validation layer is a change gate applied to a claim instead of a deployment. Approval boundaries are the same conversation as who signs off a production change. A refusal path is an error state, and every system needs one whether or not it has a model in it.
The part that genuinely is new: a model’s failure is quiet. A broken release throws. A confident wrong answer does not, so the monitoring has to be built around the output’s meaning rather than around its presence. That is the one place where I had to build something I had never built before.
What this proves
That I can take an unfamiliar technology and run it the way production software has to be run: bounded, observable, reversible where it matters, and honest about what it cannot do.
It is not a claim to be an ML engineer. I am not training models. I am the person who makes an AI system survive contact with users, a budget, and a bad day.