How do you write an SOP that AI can actually follow?

Write down the outcome, the inputs, what the contested words mean, the ordered steps, the rules for the awkward cases, one worked example, one counter example, and a list of conditions that make the output acceptable. Then add a concrete trigger for when the machine should stop and ask a person. Ten sections, most of them short. The document that results is a specification rather than an instruction, and that difference is the whole reason it produces consistent results.

This is the artefact, not the system. Our guide on systems thinking for AI workflows covers what surrounds the model: the input quality, the boundary of the step, the verification and the feedback loop. This article is about the document you hand it.

A prompt is an instruction, a specification is a contract

Most people write prompts, get inconsistent results, and conclude they need better prompting. Occasionally that is true. More often the problem is that they wrote an instruction for one occasion and are now using it as a process.

An instruction says what to do. A specification says what done means, which is a different and more demanding thing to write. It forces you to answer questions you have been carrying implicitly, and answering them is most of the value, whether or not a machine is ever involved.

If you cannot describe how you would check the result, you do not have a process yet. You have an intention.

This is why the exercise pays off even when the eventual answer is that the task should stay human. Writing the specification is how you find out.

Why your existing SOP fails when a model reads it

Company procedures are written for colleagues. Colleagues arrive with shared context, so the document can leave a great deal unsaid and still work. A new starter reading it will hit an ambiguity, notice it, and ask.

A model does not notice. It resolves the ambiguity silently, in whichever direction the surrounding text makes most likely, and continues in a confident register. The same document that produces a useful question from a person produces an invented answer from a model.

Three habits from human documentation cause most of the damage.

Undefined nouns. "Summarise the relevant sections." Relevant to whom, judged how. A colleague knows because they attended the meeting. Nothing in the document says.

Implicit ordering. Steps listed as a set rather than a sequence, where one genuinely depends on another. People infer the dependency from experience.

Instructions to use judgement. "Use your discretion for edge cases" is a reasonable line in a document for humans and an empty one for a machine, because it names no rule at all. It is the single most common instruction that cannot survive being handed to a model.

The ten sections

Most of these are two or three lines. The document does not need to be long, it needs to be unambiguous.

Four of these deserve more than a table row, because they are the ones people either skip or write badly.

Definitions do more work than anything else

Here is a test that takes ten minutes and usually finds the real problem.

Print your procedure. Circle every noun that two competent colleagues could define differently. Words like relevant, complete, customer, active, urgent, recent, senior, issue, item, section. Now write one line for each explaining what it means here, in language somebody could check.

Not "recent means the last little while" but "recent means within the previous ninety days from the date of the run". Not "an active customer" but "a customer with at least one paid invoice in the last twelve months".

This single section removes more inconsistency than rewriting the instruction ever does, because ambiguity in a noun propagates through every step that touches it. Two runs of the same process that disagree almost always disagree about a word, not about a procedure.

Counter examples beat more examples

Positive examples are useful and have a limit: they show a point in the space of acceptable answers without showing its edges. Given three good examples that happen to share an accidental property, that property tends to get carried into everything that follows.

A counter example does something a positive example cannot. It marks a boundary. "This output is wrong because it summarised the appendix, which is not part of the document body" tells you where acceptable stops, which is exactly the information a near miss exploits.

The most valuable counter example is the plausible one. An obviously bad output teaches nothing. Take a result that looked fine and was quietly wrong, include it, and write one line explaining what disqualifies it. These are easy to collect, because every correction you make by hand is a candidate.

Acceptance criteria must be checkable by a stranger

Write conditions someone who was not involved could verify without exercising judgement. If verifying requires the same expertise as doing the task, you have not written a test, you have written a wish.

Forms that hold up:

• Counts. Every section of the source appears in the output. There are between four and eight items.
• Presence. Every claim carries the sentence it came from. Every row has a date.
• Verbatim match. Each quoted sentence appears word for word in the input. This one is powerful because it turns a fabrication into a failed text search rather than something a reader has to notice.
• Range. No output shorter than a floor you set. No figure outside a plausible band.
• Structure. Valid JSON, required keys present, dates in one format.

Forms that do not hold up: well written, appropriate, professional, comprehensive, high quality. Two reviewers will disagree about every one of them, which means they cannot fail anything.

Escalation needs a trigger, not a mood

"If you are unsure, ask" is in almost every specification and it does very little. It requires the model to have a calibrated sense of its own uncertainty and to act on it against the pull of producing a fluent answer. That is not a reliable mechanism.

Replace it with conditions that are observable in the input or the output.

• A required field is absent from the input.
• The source does not contain a sentence supporting a claim you are about to make.
• A value falls outside a stated range.
• Two decision rules both apply and give different answers.
• The input is longer than the amount you were told you would receive.
• More than a stated number of items fail the acceptance criteria.

Each of these can be checked mechanically. The instruction then becomes "stop and return the escalation notice" rather than "use your judgement about whether to worry".

Testing your specification honestly

You will read claims that a particular format improves accuracy by some percentage. Treat those with suspicion unless the test design is published, because the number depends on the model, the task, the input quality, the evaluation method and the sample size, and changing any of them moves it.

Rather than trust a figure, run a small comparison on your own task. This is genuinely cheap.

1. Prepare three versions. The vague prompt you use now, your existing human SOP as written, and the specification built from the ten sections.
2. Fix a sample. Ten to twenty real inputs, chosen before you look at any results, including at least two you know are awkward.
3. Run each version on each input, more than once. Repetition matters, because consistency across runs is a separate property from correctness and often the one you actually need.
4. Score against your own acceptance criteria. Not a general impression. The criteria you already wrote, applied mechanically.
5. Record four things. How many outputs passed, how many differed between runs of the same version, how many needed manual correction, and how long reviewing took.

That last number is the one that decides whether the process is worth running at all. A version that passes more often but takes longer to check has not necessarily helped you.

Write the results down with the date and the model you used, because both will change and you will want to know what the comparison was actually against.

A worked example

Start with a request of the kind that circulates in most teams: "Go through the support inbox and pull out the recurring complaints so we can prioritise fixes."

A person can act on that. A model given the same sentence will produce a confident list every time, and the lists will differ from each other in ways nobody notices until a decision gets made on one.

The same task as a specification, compressed:

Outcome. A list of complaint themes, each with a count and two verbatim example sentences.

Inputs. Export of support messages received in the stated month, one message per record, sender name removed.

Definitions. A complaint is a message reporting that something did not work as expected, which excludes feature requests and pricing questions. A theme is a group of at least three complaints about the same feature. Recurring means appearing in at least two separate weeks.

Steps. Classify each message as complaint or not. Group the complaints by feature. Discard groups below the threshold. Order by count.

Decision rules. A message raising two distinct problems counts in both groups. A message that is a complaint and a feature request counts as a complaint. Ambiguous cases go to the escalation list rather than being assigned.

Counter example. A theme called "user experience issues" is wrong because it is not a feature and cannot be acted on.

Acceptance criteria. Every example sentence appears verbatim in the input. Every theme names a specific feature. Counts sum to no more than the number of classified complaints.

Escalation. Stop if more than one in ten messages cannot be classified, or if the export contains fewer records than stated.

Notice what changed. The task did not become more complicated. The disagreements that were going to happen anyway got resolved in advance, in writing, by the person who actually knows the answers.

What this will not fix

A specification makes a process repeatable. It does not make it correct, and it does not make a model capable of something it cannot do.

If the task requires knowledge that is not in the input, no amount of structure supplies it. If it requires a judgement your organisation has never made explicit, writing the specification will surface that, and the honest outcome is often that a person has to decide the rule before anything can be automated.

There is also a maintenance cost. A specification describes a process at a point in time, and processes drift. A stale specification is worse than none, because people trust it. Date it, and re-read it whenever the output surprises you.

Finally, the more precisely you specify a task, the more obvious it becomes when the task did not need a model at all. Several times the honest conclusion has been that the rules were mechanical enough to be a short script. That is a good outcome, not a wasted afternoon.

Where to start this week

Take one process you already hand to a model and write only two sections: definitions and acceptance criteria. Skip the rest for now.

Those two are where most of the inconsistency lives, and writing them takes about twenty minutes. If the output stabilises, finish the document. If it does not, you have learned that the problem is upstream in the input, which is the subject of a different article.

How this was put together: the format here comes from writing process documentation for our own work and from watching where instructions to models come apart. It is not a controlled study. We have deliberately published no accuracy percentages or improvement figures, because any such number depends on the model, the task, the input and the scoring method, and quoting one without that context would be misleading. The three version comparison described above is offered so you can measure your own case rather than trust ours.