| Category | Cases | Last run |
|---|---|---|
| stringsString expansion, pattern handling, and text manipulation. | 8 | 0.018 msbash median: 7.36 ms |
| arithmeticInteger math, substitutions, and expression-heavy shell snippets. | 6 | 0.023 msbash median: 5.213 ms |
| startupSmall commands where interpreter startup dominates runtime. | 4 | 0.027 msbash median: 4.411 ms |
| pipesPipeline construction, streaming, and command chaining. | 6 | 0.03 msbash median: 9.414 ms |
| variablesVariable assignment, lookup, expansion, and environment handling. | 8 | 0.032 msbash median: 6.57 ms |
| controlConditionals, loops, case statements, and branching scripts. | 9 | 0.042 msbash median: 5.255 ms |
| toolsBuiltin and external-tool style command workloads. | 21 | 0.042 msbash median: 7.603 ms |
| ioFile reads, writes, redirects, and filesystem-facing commands. | 6 | 0.042 msbash median: 4.396 ms |
Benches
Latest benchmark snapshot
Static aggregate generated from repository result artifacts. Use the linked files for raw measurements and full eval traces.
| Category | Passed | Pass rate |
|---|---|---|
| system_info | 1/2 | 50%tasks passed |
| file_operations | 3/4 | 66.7%tasks passed |
| scripting | 5/7 | 68.6%tasks passed |
| json_processing | 8/8 | 100%tasks passed |
| data_transformation | 6/6 | 100%tasks passed |
| complex_tasks | 6/6 | 100%tasks passed |
| text_processing | 6/6 | 100%tasks passed |
| pipelines | 5/5 | 100%tasks passed |
| Run | Score | Tools |
|---|---|---|
| gpt-5.3-codex2026-05-26 | 93% | 86.8% |
| gpt-5.52026-05-26 | 92.7% | 91.5% |
| claude-opus-4-72026-05-26 | 97.8% | 90.3% |
| claude-sonnet-4-62026-05-26 | 94% | 91% |
| claude-haiku-4-5-202510012026-05-26 | 98.4% | 92.3% |
| claude-sonnet-4-62026-02-28 | 92.5% | 85.1% |
Indexes