Doc2Harness: Benchmarking how well AI builds agent harnesses

The authors propose Doc2Harness, a two-stage benchmark that evaluates long-horizon harness engineering: a build model constructs a candidate harness from an interface-only contract skeleton and a development document consolidating behavior across twelve frontier harnesses, and the frozen harness is then run per test case by a scripted mock model and scored via four observable channels. Across nineteen frontier models, each with three independent builds on 328 test cases, GPT-6 Astra achieves the highest pass rate at 53.05% while the weakest model reaches 27.44%, and failures cluster around how cross-cutting functionality is split across modules. The authors describe it as the first deterministic benchmark for measuring long-horizon harness engineering.