Benchmarks
Cloud operations benchmarks
We're building a collection of benchmarks based on real cloud operations tasks. We use them to measure how effectively models perform and to learn how we can work with them more reliably.
AWS Operations
Updated Jul 29, 2026suite aws-ops
Ten realistic AWS operations tasks — incident fixes, security remediations, cost cleanups, audits, and forensics — run by three coding agents in skill and no-skill arms, three repetitions each.
10
Tasks
3
Agents
180
Runs
Agents
claudeclaude-opus-4-8[1m]
codexgpt-5.6-sol
cursorComposer 2.5 Fast
View benchmark
AWS Incident Response
Coming soonsuite aws-incident-response
Seven realistic AWS incident response scenarios — detection, containment, investigation, and recovery — evaluated across three coding agents.
7
Scenarios
3
Agents
Agents
claude
codex
cursor
Not released yet