Deterministic AI: Why LLMs Aren't, and How to Build Products That Are Anyway

This guide explains that LLMs are not deterministic in practice—a Thinking Machines test of 1,000 runs at temperature 0 on one prompt produced 80 different completions—because of provider batching, best-effort seeds, model alias updates and drifting inputs. It argues the deterministic-vs-probabilistic choice should be made per decision, keeping LLMs for reading and writing while moving money, access and compliance decisions into code rules, calibrated classifiers or decision models. It also offers layers for constraining LLM output and a method for measuring per-input agreement rates on borderline cases.