The decisive 3-arm run: the coach answered the same 119 questions three times each with normal thinking (default), a small bounded budget (low), and thinking turned off (disabled): 357 answers per arm, 1,071 total. "Failed" means the answer ended as a canned apology (a template) even after its one automatic retry.
⚠ Two limits on these numbers: the fact-shelf coverage flag is true for all 1,071 rows (that axis is degenerate here: question difficulty is the axis that discriminates), and nothing here is comparable to the July board-live baseline: different question population, so no before/after claim is made.
A small thinking budget wins the first attempt: near-top speed, and the fewest failures on the questions that matter most.
low beats disabled on quality in every difficulty bucket at essentially the same speed (6.2s vs 5.6s typical), and beats default on tactics-and-mate questions (13.3% vs 18.7% failed) while answering 2.3× faster on a fifth of the output tokens. Decision: attempt-0 = low, retry = default: live chat only; the postgame review keeps its own budget.
Scope correction · noted 2026-08-18
The two "live chat only" lines above describe the plan, not what shipped. The merged code applies attempt-0 = low to every intent, postgame review included, with no mode gate: thinkingForIntent returns low from a single default branch, and chat.ts uses the same pref for review as for live. The numbers on this page are unaffected.
CHART 1 · THE HEADLINE
Where failures cluster: difficulty × thinking budget
One dot per (difficulty bucket, arm): its position is that cell's failure rate. Dot area reflects how many questions the bucket holds. The pattern to see: every arm slides right on the bottom row, disabled is rightmost almost everywhere, and low is leftmost where it counts.
TAKEAWAY
Failures concentrate on tactics-and-mate in every arm (18.7% / 13.3% / 22.7%), well above each arm's own easier buckets. disabled fails most everywhere: it is the only arm that fails simple fact questions at a real rate (9.5%). And low is best exactly on the hard row: 13.3%, beating even default's unbounded thinking there.
Rates are final outcomes: every failed row got exactly one retry first. n per bucket: direct-fact 42, needs-line 240, tactical-or-mate 75 (per arm). Source: 2026-08-03-od3b-all-summary.json.
CHART 2
Which difficulty buckets drive the failures (Pareto)
All 90 failed answers across the three arms, sorted by which difficulty bucket produced them. Bars are share of all failures, split by arm; the line is the running total on the same 0-100% axis.
TAKEAWAY
Two buckets carry 94% of all failures. needs-line contributes the most by count (44) because it is two-thirds of the question set; tactical-or-mate contributes nearly as many (41) from a bucket a third that size: it fails at 3× the rate. Chart 1 carries the rate story; this chart says where the repair effort would land.
Counts are pooled across arms (23 + 28 + 39 = 90 failed answers of 1,071). Bucket sizes differ: the bar heights reflect volume × rate, not rate alone.
CHART 3
Speed vs. failures: the trade each arm offers
One point per arm. Left is faster, lower is fewer failed answers: the bottom-left corner wins. The thin tail on each point stretches to its 90th-percentile wait.
TAKEAWAY
low sits in the sweet spot: 0.6s slower than disabled at the median but 3.1 points fewer failures, and its tail stays under 12s. default buys nothing you'd wait for: 2.3× slower typical, a 47-second p90 tail, and it still isn't the best arm on the hardest questions. It keeps one job: the escalated retry, where its extra thinking room rescues about two-thirds of failed first drafts.
Latency is wall time to the final answer (nearest-rank percentiles). Failure rate is the arm's overall post-retry template rate, n=357 each.