Instruction-following code editing across 225 Exercism exercises in six languages, scored on pass rate after one retry and on whether the edit was well formed.
May republish0 incidents recorded
Text equivalentAider Polyglot: best reproduced result 76.9 points. Remaining headroom has not been measured, so how much room is left is unknown — not zero.
Remaining headroom has not been measured. Unknown, not zero.
Contamination controls
static public Exercism-derived tasks
no private split or rolling refresh described
Licence
Apache-2.0 Recorded as republishable, so its figures appear on this site.
Update cadence
Maintainer-run; new models do not appear automatically
Access path
Raw YAML in the Aider repository
Incident log
Each entry states what was found, by whom, and when. The kind describes the finding, not anybody’s conduct. An entry without a primary source and a date is not published here.
No incidents recorded in this register.That is a statement about what we have recorded, not a finding that none exist. We have not audited every benchmark, and an absence here is an absence of evidence.