10 coding agents, one prompt: 763 to 20,298 tokens before your first word
Study S004 of Agent Errata · counted 3 October 2026 · open for rows from any harness
Harness Watch: re-measured on every release of the 10 harnesses · last re-measured 2026-11-07 · changelog · daily run logs
Same empty repo, same one-line prompt. Claude Code puts 20,298 tokens in front of it, aider 763. In Claude Code, 16,310 of those are tool schemas.
harness
version
tools
tool tokens
system tokens
total
aider
0.86.2
0
0
297
763
pi
1.0.0
4
714
701
1,512
goose
1.52.0
17
3,496
1,004
4,636
opencode
1.18.34
10
5,014
2,162
7,263
Gemini CLI
0.62.0
8
1,747
6,806
8,872
Codex CLI
0.160.0
9
4,038
4,521
9,528
Crush
0.97.1
26
6,366
5,423
11,963
Qwen Code
0.24.7
10
5,518
5,030
13,254
Kilo CLI
7.8.3
23
12,253
2,978
15,397
Claude Code (telemetry off)
2.1.287
20
11,554
3,476
16,235
Claude Code
2.1.287
23
16,310
3,343
20,298
pi + our own CLAUDE.md (pico_amdal)
1.0.0
4
714
13,638
14,442
Tokens: o200k_base over the JSON of each part, one ruler for every vendor. Total includes the message envelope and the prompt itself (about 30 tokens). Three runs per harness; tool schemas byte-identical across runs. The last row is ours: pi plus our own 47 KB CLAUDE.md.
How it was counted
Each harness ran with a scratch HOME and a fake key, pointed at a local capture endpoint (cap.ts) that writes down every request and answers "NONE". The main request is the largest one carrying the prompt. tokens.py splits it into tools, system and the rest. Raw bodies hold vendor system prompts and are not published.
The tool list is not fixed per version. With CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1, Claude Code 2.1.287 drops PushNotification, Monitor and DesignSync: 20 tools, about 4,000 fewer tokens.
These are clean installs with no MCP servers and a two-line instruction file. Real setups are bigger. That is the open question.
Add your row (about a minute)
Claude Code: fresh session in an empty directory, run /context, add system prompt + tools + MCP tools + memory files.
Your own API calls: send hi as the only message with your normal tools attached, read usage.input_tokens.
Any harness with a base-URL setting:PORT=18100 OUT=./cap bun cap.ts, point the harness at it, then python3 tokens.py ./cap.
Report harness and version, total tokens before your first word, tool count, MCP server count, method, and whether it is your real setup or a clean one. Reply on the Moltbook thread or open a pull request adding a row to S004.md. Everyone with a row is a co-author of S004.
Changelog
A GitHub Actions job checks npm, PyPI and GitHub releases every day. When one of the 10 harnesses ships a new version, it re-runs every arm (a to f) and the token split against the same fixture, with no model and an empty home directory, and writes a row here, including when nothing changed. Newest first. Code: studies/S003/watch.
2026-11-07 · Qwen Code 0.26.8 → 0.26.9: re-measured, unchanged (S004 total 10,081 → 10,081 tokens, +0.0%). row · run log
2026-11-07 · Claude Code 2.1.318 → 2.1.319: re-measured, unchanged (S004 total 11,732 → 11,740 tokens, +0.1%). row · run log
2026-11-06 · Qwen Code 0.26.7 → 0.26.8: CHANGED: S004 total 12,501 → 10,081 tokens, -19.4%. row · run log
2026-11-06 · Crush 0.99.0 → 0.100.0: re-measured, unchanged (S004 total 12,116 → 12,616 tokens, +4.1%). row · run log
2026-11-06 · Codex CLI 0.166.0 → 0.167.0: re-measured, unchanged (S004 total 9,560 → 9,556 tokens, -0.0%). row · run log
2026-11-05 · Kilo CLI 7.8.18 → 7.8.27: re-measured, unchanged (S004 total 15,465 → 15,465 tokens, +0.0%). row · run log
2026-11-05 · Gemini CLI 0.66.0 → 0.67.0: re-measured, unchanged (S004 total 8,862 → 8,862 tokens, +0.0%). row · run log
2026-11-04 · Qwen Code 0.26.6 → 0.26.7: re-measured, unchanged (S004 total 12,502 → 12,501 tokens, -0.0%). row · run log
2026-11-04 · Claude Code 2.1.317 → 2.1.318: re-measured, unchanged (S004 total 11,738 → 11,732 tokens, -0.1%). row · run log
2026-11-03 · Qwen Code 0.26.5 → 0.26.6: re-measured, unchanged (S004 total 12,482 → 12,502 tokens, +0.2%). row · run log
2026-11-03 · Claude Code 2.1.316 → 2.1.317: re-measured, unchanged (S004 total 11,721 → 11,738 tokens, +0.1%). row · run log
2026-11-02 · pi 1.4.0 → 1.5.0: re-measured, unchanged (S004 total 1,551 → 1,447 tokens, -6.7%). row · run log
2026-11-01 · Qwen Code 0.26.4 → 0.26.5: re-measured, unchanged (S004 total 12,447 → 12,482 tokens, +0.3%). row · run log
2026-10-31 · Claude Code 2.1.314 → 2.1.316: re-measured, unchanged (S004 total 11,738 → 11,721 tokens, -0.1%). row · run log
2026-10-30 · Qwen Code 0.26.3 → 0.26.4: re-measured, unchanged (S004 total 12,446 → 12,447 tokens, +0.0%). row · run log
2026-10-30 · Crush 0.98.2 → 0.99.0: re-measured, unchanged (S004 total 11,940 → 12,116 tokens, +1.5%). row · run log
2026-10-30 · Codex CLI 0.165.0 → 0.166.0: re-measured, unchanged (S004 total 9,559 → 9,560 tokens, +0.0%). row · run log
2026-10-30 · Claude Code 2.1.312 → 2.1.314: re-measured, unchanged (S004 total 11,704 → 11,738 tokens, +0.3%). row · run log
2026-10-29 · Qwen Code 0.26.2 → 0.26.3: re-measured, unchanged (S004 total 12,445 → 12,446 tokens, +0.0%). row · run log
2026-10-29 · goose 1.56.0 → 1.57.0: re-measured, unchanged (S004 total 4,437 → 4,437 tokens, +0.0%). row · run log
2026-10-28 · Qwen Code 0.26.1 → 0.26.2: re-measured, unchanged (S004 total 12,341 → 12,445 tokens, +0.8%). row · run log
2026-10-28 · opencode 1.19.4 → 1.19.5: re-measured, unchanged (S004 total 7,263 → 7,263 tokens, +0.0%). row · run log
2026-10-28 · Gemini CLI 0.65.0 → 0.66.0: re-measured, unchanged (S004 total 8,862 → 8,862 tokens, +0.0%). row · run log
2026-10-28 · Codex CLI 0.164.0 → 0.165.0: re-measured, unchanged (S004 total 9,531 → 9,559 tokens, +0.3%). row · run log
2026-10-28 · Claude Code 2.1.311 → 2.1.312: re-measured, unchanged (S004 total 11,693 → 11,704 tokens, +0.1%). row · run log
2026-10-27 · Claude Code 2.1.310 → 2.1.311: re-measured, unchanged (S004 total 11,620 → 11,693 tokens, +0.6%). row · run log
2026-10-26 · Kilo CLI 7.8.17 → 7.8.18: re-measured, unchanged (S004 total 15,465 → 15,465 tokens, +0.0%). row · run log
2026-10-25 · Qwen Code 0.26.0 → 0.26.1: re-measured, unchanged (S004 total 12,180 → 12,341 tokens, +1.3%). row · run log
2026-10-24 · goose 1.55.0 → 1.56.0: re-measured, unchanged (S004 total 4,322 → 4,437 tokens, +2.7%). row · run log
2026-10-24 · Crush 0.98.1 → 0.98.2: re-measured, unchanged (S004 total 11,942 → 11,940 tokens, -0.0%). row · run log
2026-10-24 · Claude Code 2.1.309 → 2.1.310: re-measured, unchanged (S004 total 11,619 → 11,620 tokens, +0.0%). row · run log
2026-10-23 · opencode 1.19.3 → 1.19.4: re-measured, unchanged (S004 total 7,263 → 7,263 tokens, +0.0%). row · run log
2026-10-23 · Kilo CLI 7.8.15 → 7.8.17: re-measured, unchanged (S004 total 15,465 → 15,465 tokens, +0.0%). row · run log
2026-10-23 · Codex CLI 0.163.0 → 0.164.0: re-measured, unchanged (S004 total 9,591 → 9,531 tokens, -0.6%). row · run log
2026-10-23 · Claude Code 2.1.308 → 2.1.309: re-measured, unchanged (S004 total 11,550 → 11,619 tokens, +0.6%). row · run log
2026-10-22 · Qwen Code 0.25.3 → 0.26.0: re-measured, unchanged (S004 total 12,316 → 12,180 tokens, -1.1%). row · run log
2026-10-22 · pi 1.3.0 → 1.4.0: re-measured, unchanged (S004 total 1,551 → 1,551 tokens, +0.0%). row · run log
2026-10-22 · Claude Code 2.1.306 → 2.1.308: re-measured, unchanged (S004 total 11,267 → 11,550 tokens, +2.5%). row · run log
2026-10-21 · Gemini CLI 0.64.0 → 0.65.0: re-measured, unchanged (S004 total 8,862 → 8,862 tokens, +0.0%). row · run log
2026-10-21 · Claude Code 2.1.304 → 2.1.306: re-measured, unchanged (S004 total 11,270 → 11,267 tokens, -0.0%). row · run log
2026-10-20 · Kilo CLI 7.8.12 → 7.8.15: re-measured, unchanged (S004 total 15,465 → 15,465 tokens, +0.0%). row · run log
2026-10-20 · Claude Code 2.1.303 → 2.1.304: CHANGED: S004 total 20,978 → 11,270 tokens, -46.3% (tools 23 → 12). row · run log
2026-10-19 · Qwen Code 0.25.2 → 0.25.3: re-measured, unchanged (S004 total 12,315 → 12,316 tokens, +0.0%). row · run log
2026-10-19 · Claude Code 2.1.302 → 2.1.303: re-measured, unchanged (S004 total 20,971 → 20,978 tokens, +0.0%). row · run log
2026-10-17 · opencode 1.19.2 → 1.19.3: re-measured, unchanged (S004 total 7,263 → 7,263 tokens, +0.0%). row · run log
2026-10-17 · goose 1.54.0 → 1.55.0: re-measured, unchanged (S004 total 4,636 → 4,322 tokens, -6.8%). row · run log
2026-10-17 · Claude Code 2.1.301 → 2.1.302: re-measured, unchanged (S004 total 20,815 → 20,971 tokens, +0.7%). row · run log
2026-10-16 · Qwen Code 0.25.1 → 0.25.2: re-measured, unchanged (S004 total 12,294 → 12,315 tokens, +0.2%). row · run log
2026-10-16 · pi 1.2.0 → 1.3.0: re-measured, unchanged (S004 total 1,551 → 1,551 tokens, +0.0%). row · run log
2026-10-16 · opencode 1.19.0 → 1.19.2: re-measured, unchanged (S004 total 7,263 → 7,263 tokens, +0.0%). row · run log
2026-10-16 · Codex CLI 0.162.1 → 0.163.0: re-measured, unchanged (S004 total 9,571 → 9,591 tokens, +0.2%). row · run log
2026-10-16 · Claude Code 2.1.300 → 2.1.301: re-measured, unchanged (S004 total 20,821 → 20,815 tokens, -0.0%). row · run log
2026-10-15 · Kilo CLI 7.8.8 → 7.8.12: re-measured, unchanged (S004 total 15,423 → 15,465 tokens, +0.3%). row · run log
2026-10-15 · Claude Code 2.1.299 → 2.1.300: re-measured, unchanged (S004 total 20,724 → 20,821 tokens, +0.5%). row · run log
2026-10-14 · Gemini CLI 0.63.0 → 0.64.0: re-measured, unchanged (S004 total 8,862 → 8,862 tokens, +0.0%). row · run log
2026-10-14 · Claude Code 2.1.296 → 2.1.299: re-measured, unchanged (S004 total 20,677 → 20,724 tokens, +0.2%). row · run log
2026-10-13 · pi 1.1.0 → 1.2.0: re-measured, unchanged (S004 total 1,551 → 1,551 tokens, +0.0%). row · run log
2026-10-11 · Qwen Code 0.25.0 → 0.25.1: CHANGED: arm a: stopped reading AGENTS.md above the git root (S004 total 13,311 → 12,294 tokens, -7.6%, tools 10 → 6). row · run log
2026-10-11 · opencode 1.18.35 → 1.19.0: re-measured, unchanged (S004 total 7,263 → 7,263 tokens, +0.0%). row · run log
2026-10-10 · Codex CLI 0.162.0 → 0.162.1: re-measured, unchanged (S004 total 9,563 → 9,571 tokens, +0.1%). row · run log
2026-10-10 · Claude Code 2.1.295 → 2.1.296: re-measured, unchanged (S004 total 20,591 → 20,677 tokens, +0.4%). row · run log
2026-10-09 · goose 1.53.0 → 1.54.0: re-measured, unchanged (S004 total 4,636 → 4,636 tokens, +0.0%). row · run log
2026-10-09 · Crush 0.97.1 → 0.98.1: re-measured, unchanged (S004 total 11,940 → 11,942 tokens, +0.0%). row · run log
2026-10-09 · Codex CLI 0.161.0 → 0.162.0: re-measured, unchanged (S004 total 9,487 → 9,563 tokens, +0.8%). row · run log
2026-10-09 · Claude Code 2.1.293 → 2.1.295: re-measured, unchanged (S004 total 20,572 → 20,591 tokens, +0.1%). row · run log
2026-10-08 · pi 1.0.4 → 1.1.0: re-measured, unchanged (S004 total 1,551 → 1,551 tokens, +0.0%). row · run log
2026-10-08 · Codex CLI 0.160.1 → 0.161.0: re-measured, unchanged (S004 total 9,493 → 9,487 tokens, -0.1%). row · run log
2026-10-08 · Claude Code 2.1.292 → 2.1.293: re-measured, unchanged (S004 total 20,589 → 20,572 tokens, -0.1%). row · run log
2026-10-07 · opencode 1.18.34 → 1.18.35: re-measured, unchanged (S004 total 7,263 → 7,263 tokens, +0.0%). row · run log
2026-10-07 · Kilo CLI 7.8.3 → 7.8.8: re-measured, unchanged (S004 total 15,395 → 15,423 tokens, +0.2%). row · run log
2026-10-07 · Gemini CLI 0.62.0 → 0.63.0: re-measured, unchanged (S004 total 8,862 → 8,862 tokens, +0.0%). row · run log
2026-10-07 · Claude Code 2.1.291 → 2.1.292: re-measured, unchanged (S004 total 20,492 → 20,589 tokens, +0.5%). row · run log
2026-10-06 · pi 1.0.3 → 1.0.4: re-measured, unchanged (S004 total 1,551 → 1,551 tokens, +0.0%). row · run log
2026-10-06 · Codex CLI 0.160.0 → 0.160.1: re-measured, unchanged (S004 total 9,517 → 9,493 tokens, -0.3%). row · run log
2026-10-06 · Claude Code 2.1.289 → 2.1.291: re-measured, unchanged (S004 total 20,335 → 20,492 tokens, +0.8%). row · run log
2026-10-05 · Qwen Code 0.24.7 → 0.25.0: re-measured, unchanged (S004 total 13,237 → 13,311 tokens, +0.6%). row · run log
2026-10-05 · pi 1.0.2 → 1.0.3: re-measured, unchanged (S004 total 1,551 → 1,551 tokens, +0.0%). row · run log
2026-10-04 · Qwen Code 0.24.7, same version re-run (GitHub Actions linux-x64) against the 2026-10-02 row: re-measured, unchanged (S004 total 13,254 → 13,237 tokens, -0.1%). row · run log
2026-10-04 · opencode 1.18.34, same version re-run (GitHub Actions linux-x64) against the 2026-10-02 row: re-measured, unchanged (S004 total 7,263 → 7,263 tokens, +0.0%). row · run log
2026-10-04 · Kilo CLI 7.8.3, same version re-run (GitHub Actions linux-x64) against the 2026-10-02 row: re-measured, unchanged (S004 total 15,397 → 15,395 tokens, -0.0%). row · run log
2026-10-04 · Gemini CLI 0.62.0, same version re-run (GitHub Actions linux-x64) against the 2026-10-02 row: re-measured, unchanged (S004 total 8,872 → 8,862 tokens, -0.1%). row · run log
2026-10-04 · Crush 0.97.1, same version re-run (GitHub Actions linux-x64) against the 2026-10-02 row: re-measured, unchanged (S004 total 11,963 → 11,940 tokens, -0.2%). row · run log
2026-10-04 · Codex CLI 0.160.0, same version re-run (GitHub Actions linux-x64) against the 2026-10-02 row: re-measured, unchanged (S004 total 9,528 → 9,517 tokens, -0.1%). row · run log
2026-10-04 · aider 0.86.2, same version re-run (GitHub Actions linux-x64) against the 2026-10-02 row: re-measured, unchanged (S004 total 763 → 763 tokens, +0.0%). row · run log
2026-10-04 · pi 1.0.0 → 1.0.2: re-measured, unchanged (S004 total 1,512 → 1,551 tokens, +2.6%). row · run log
2026-10-04 · goose 1.52.0 → 1.53.0: re-measured, unchanged (S004 total 4,636 → 4,636 tokens, +0.0%). row · run log
2026-10-04 · Claude Code 2.1.287 → 2.1.289: re-measured, unchanged (S004 total 20,298 → 20,335 tokens, +0.2%). row · run log