Building Deterministic Systems Around Probabilistic Models

The distance between an AI demonstration and an AI system in production is not model quality. It is the engineering around the model — and specifically, how much of the system's behaviour is determined by code rather than by inference.

This is the pattern we apply, and the reasoning behind each part of it.

Start by drawing the boundary

Before any implementation, write down three lists: what the system will do automatically, what it will do with human approval, and what it will never do.

The third list matters most and is usually the one nobody writes. It is where irreversibility, regulatory exposure and reputational risk are handled. A payment above a threshold, an adverse decision about a person, a public communication, a change to a safety-critical parameter — these belong on the third list, and putting them there at the start costs nothing. Discovering them later costs a great deal.

Give the model a bounded job

A model should have a job description you could hand to a competent contractor: extract these fields from this document; rank these items against these criteria; draft a response in this register; classify this into one of these categories.

What it should not have is discretion over what happens next. Sequencing, permissions, eligibility and escalation belong in code, because code can be reviewed, tested, versioned and reasoned about. When a prompt starts to contain business rules, that is a signal the boundary has slipped.

The practical test: could you replace the model with a different one and re-run your evaluation set without re-validating the workflow? If not, the model is carrying architecture.

Make orchestration explicit

Use a workflow engine, a state machine, or plain code with clear states — but make the process a first-class artefact rather than an emergent property of prompt chaining.

An explicit orchestration layer gives you three things that matter operationally: you can see where an instance of the process currently is; you can retry a failed step without re-running the whole thing; and you can change one step without disturbing the others. Chained prompts give you none of these.

Ground the model, and own the grounding

Retrieval is where most production quality is won. It is also where most teams under-invest, because curating a corpus is unglamorous work compared with model selection.

Someone must own what is in the corpus, what version of each document is authoritative, what has been retired, and who may see what. Access control in particular must be enforced at retrieval time by the system, not requested of the model in an instruction. A model asked politely not to reveal something is not an access-control mechanism.

Evaluate before, and keep evaluating after

Agree the evaluation criteria before the build, with the people who will rely on the output. Build a representative set of cases including the awkward ones — the ambiguous document, the out-of-scope question, the adversarial input.

Then keep running it. Model providers update models. Corpora drift. User behaviour changes. The most expensive AI failure is silent degradation, because nothing announces it. Track human override rates alongside accuracy: a rising override rate is often the earliest signal that a system has stopped being useful, and it appears well before anyone files a complaint.

Instrument everything

Log inputs, outputs, retrieved context, model version, prompt version, latency, cost and the human decision that followed. Not for compliance theatre — for the ordinary engineering reason that you cannot debug what you did not record, and for the specific reason that in twelve months somebody will ask you to explain a single decision.

Design the failure path

Every step should have a defined behaviour when the model is unavailable, slow, or returns something unusable. In most enterprise contexts the correct behaviour is to fall back to the existing human process rather than to fail the transaction.

Systems built this way are less impressive in a demonstration and considerably better in production. They are also far cheaper to change, which — given how quickly the underlying models move — is the property that determines whether what you build this year is still an asset next year.