Your model's output is a sample, not a return value
You ship a feature. It works. A week later a user sends a screenshot you can't reproduce, because the same input stopped producing the same output. That's the first thing you lose when a language model enters your code path: not correctness, but reproducibility.
Adapted from Chapter 1, “Why AI Development Feels Different”, of AI Engineering, Measured.
What I measured
Twenty calls, the same prompt, one open-ended question. On the default path I got 10 unique outputs out of 20. Passing temperature: 0 gave 4 unique out of 20.
I ran it twice. The default path returned 10 unique both times. Temperature 0 returned 3 the first time and 4 the second. Read that pair again: the measurement of the variance is itself variable. That isn't noise in the experiment. It's the thing the experiment is about.
Pinning the sampling helped a great deal. It did not get me to one.
Design for variance. Don't chase determinism.
The instinct on meeting a non-deterministic system is to find the setting that turns it off, then tune the prompt until the output is byte-identical and assertEqual means something again. Sampling controls buy a large reduction in variance and no guarantee. If your design needs byte-identical output to be correct, it has a defect a temperature setting will hide for a while and not fix.
What breaks first is the format, not the content
Across all 20 runs, in both conditions and both trials, every answer said the same thing. Ten distinct byte strings on the default path, all meaning the same. So:
- A cache keyed on the response string misses most of the time and still returns correct answers, so it looks healthy.
- A test asserting equality against a recorded response fails on a run where nothing is wrong.
- A regex against the wording matches sometimes, which reads like flakiness.
Code that compares strings when it should be checking meaning is the failure most likely to reach your users.
Four shifts to plan for
- Output is a sample, not a return value. Call twice with identical input and you can get two different strings. That's the designed behaviour.
- The contract is prose. Between your code and a model sits a paragraph of English with no type checker on it.
- Failure is silent and plausible. A wrong answer arrives in the same shape as a right one, with a 200 and a normal latency.
- The trust boundary moved. Model output is untrusted input that originates inside your own process, after your validation already ran.
About these numbers. Two trials of n=20 on one prompt, against claude-opus-4-6 on anthropic-sdk-python 1.2.0. Two observations are not a rate, and your numbers will differ. The chapter's exercise has you produce your own.
The full chapter, with the code, the diagram and a hands-on exercise, is in AI Engineering, Measured. Building something like this and want a second pair of eyes? Book a 1:1 session.