Systems thinking for AI workflows

Two people can send the same model the same instruction and get results that differ enormously in usefulness. Most of the time the difference has nothing to do with the wording. It is in what they fed it, what they had already decided a good answer looks like, and what they do with an answer that misses.

That is the whole argument for thinking in systems. A model is a component. Components behave according to what surrounds them, and the surroundings are the part you control.

Why a good prompt still produces work you cannot use

Language models are optimised to produce plausible continuations. Plausibility and accuracy correlate, which is why they are useful, but they are different measures, and nothing in the output signals which one you got. A confident paragraph about a document the model only partly received looks exactly like a confident paragraph about a document it received in full.

This has a practical consequence that gets missed. Reviewing AI output is harder than reviewing human output, because human errors usually come with tells. Someone unsure of a claim hedges, or the prose goes vague where the knowledge runs out. Model output stays uniformly smooth across the boundary between what it knows and what it is filling in.

So a process that relies on someone spotting a bad answer by reading it will leak. The check has to be structural: compare against the source, verify a figure, confirm a citation resolves. Something a person performs rather than senses.

The four parts a prompt does not describe

When an AI workflow disappoints, the instruction is usually not the weakest part. Four other things are, and each of them is a design decision somebody has to make explicitly.

The input. What material reaches the model, in what form, and how complete is it. A PDF converted badly, a page whose main content was buried in navigation markup, a deck where the speaker notes were dropped. The model cannot ask what it is missing, so it fills the gap.

The boundary. What this step is responsible for and what it is not. A step asked to summarise, judge relevance and suggest next actions all at once will do all three at a mediocre level, and you will not be able to tell which part failed.

The verification. What has to be true about the output before it counts as done, and who or what establishes that. This is the part most often left out entirely.

The loop. What happens to a failed output. If a bad answer is quietly fixed by hand and nothing is recorded, the process learns nothing and the same failure returns next week.

Notice that none of these are about the model. Swapping in a better model improves the first draft. It does not supply an input that was never collected or a definition of correct that nobody wrote.

Fix the input before you tune the instruction

Input quality is where the largest available improvement usually sits, and it is the least interesting thing to work on, which is why it gets skipped.

A model reasoning carefully over the wrong half of a document will produce a careful answer about the wrong half.

Four input problems account for most of it. Truncation, where a long source was silently cut. Contamination, where navigation, cookie banners and related-article blocks came along with the text and now compete for attention. Structure loss, where headings and tables that carried meaning arrived as undifferentiated prose. And staleness, where the source was correct last quarter.

All four are invisible in the output. The answer still arrives, formatted correctly, on the right topic. The only way to catch them is to look at what you actually sent, which almost nobody does after the first week.

This is the least glamorous discipline in the whole practice and the one with the best return. Getting clean text out of a page, a PDF or a deck before it reaches a model is exactly what our PDF Content Extractor and Website Content Extractor exist to do, and they run in the browser so the source material never leaves your machine.

Say what a correct answer looks like

Before asking a model to do something, write down how you would tell whether the result was right. If you cannot answer that, the step is not ready, and no instruction will make it ready.

The answer is often uncomfortably concrete. For a competitor brief, a correct output names the specific pages that changed and quotes the changed wording, so a reader can check it in one click. For a document summary, a correct output covers every section, which means you can count sections and confirm. For a classification, a correct output matches what a person would have chosen on twenty examples you labelled first.

Writing this down does two things. It usually simplifies the instruction, because you discover you were asking for several unrelated outcomes at once. And it gives you something to test against, which turns a vague sense that the output got worse into a number.

Build the check before you scale the volume

The most common failure pattern is not a bad output. It is a good output at low volume followed by unnoticed decay at high volume, because the review that existed at ten items per week was a person reading them, and at four hundred that person reads none.

So the verification has to scale with the work or the work should not scale. A few forms that hold up:

1. Source anchoring. Require every claim to carry the sentence it came from. Wrong claims become visible without reading the source, because the quote either supports the claim or does not.
2. Structural assertions. Check the shape rather than the meaning. Every section covered, every figure present, no output shorter than a floor you set. Cheap, mechanical, and it catches truncation and skipped input.
3. Sampling with a labelled set. Keep twenty or thirty examples you have judged yourself and re-run them whenever anything changes. This is the only reliable way to notice slow drift.
4. A named acceptance step. One person decides whether output is good enough to leave the building. Not a committee, and not implied by silence.

The acceptance step is the one people resist, because it feels like it undoes the automation. It does not. It moves the human from doing the work to deciding about the work, which is roughly a tenth of the time and the part where judgement was actually needed.

A worked example

A team wanted weekly summaries of regulatory updates relevant to their product. The first version was a prompt: paste the update, ask for a summary and an impact assessment. It worked for a month and then quietly stopped being trusted, because two summaries had described requirements that did not exist in the source.

Redesigned as a system, the same work looked like this.

Input became explicit. Full text pulled from the source page, checked for length against the original, with navigation stripped. The old version had been fed a truncated version of anything long, which is where both bad summaries came from.

The boundary got narrower. Two separate steps replaced one. First, extract the obligations as a list, each with the sentence it came from. Second, and only after a person had scanned that list, assess impact. The impact question was the one requiring product knowledge the model did not have.

Verification became structural. Each extracted obligation had to quote its source sentence, and the quote had to appear verbatim in the input. A wrong obligation now fails a text search instead of requiring someone to notice it.

The loop got recorded. Every correction went into a running file of examples, which became the labelled set used to check the process after any change.

The model and the prompt barely changed. What changed was everything around them, and the output went from occasionally fabricated to reliable enough to act on.

Why this is the skill that compounds

Model capability improves on someone else's schedule and arrives for everyone at once. Whatever advantage a clever prompt gives you today gets absorbed into the next release. The design of the process around the model does not work that way, because it encodes what your work actually requires, and nobody else can ship that for you.

It also transfers. Someone who has thought carefully about inputs, boundaries, verification and feedback for one workflow can set up the next one in a fraction of the time, with any model. That is a different kind of skill from knowing which phrasing currently performs best.

Where to start this week

Take one workflow where AI is already involved and answer four questions in writing: what reaches the model, what this step is responsible for, how you would know the output was right, and what happens when it is not. Whichever question you cannot answer is the one costing you the most.

In most cases it is the third. Nobody wrote down what correct looks like, so nothing in the process can tell good output from bad, and the whole thing rests on somebody remembering to read carefully.