1# Published Router Benchmark Results
2 
3Each `<date>.md` file in this directory is a committed snapshot of a router benchmark sweep. Raw per-run JSON outputs live under `researcher/benchmarks/router/results/<date>-<seed>/` and are gitignored; only the curated summary published here is tracked in the repo.
4 
5Every report includes:
6 
7- Run metadata (timestamp, repo commit, fixture SHA, seed, model list, replications).
8- Executive summary calling out the actually meaningful findings.
9- Per-model leaderboard with bootstrap 95% CIs.
10- Per-skill confusion matrix.
11- Hardest-prompts breakdown.
12- Reproduction command.
13 
14When a benchmark exposes a routing failure, follow up by editing the activation description of the failing skill, rerunning the benchmark, and comparing the new report against the previous one to show the delta.
15 
16History across runs is also appended to `researcher/reports/router-history.jsonl` (gitignored) by the runner.
17