Enterprise IT Agent Benchmark

IBM and Artificial Analysis built the first benchmark for enterprise IT agents. Every frontier model scored below 50%.

The best performer, Claude Opus 4.7, topped out at 47% across 59 real site-reliability tasks — the Kubernetes incident response and on-call work that keeps production systems alive. GPT-5.5 landed right behind at 46%.

Sit with that for a second. The models we're being told will replace teams can't yet clear a coin flip on the actual job.

This isn't an argument against AI agents. It's a map of where the value really is. The gap between a demo and a dependable system isn't the model — it's everything wrapped around it: the context, the guardrails, the escalation paths, the human judgment at the seams.

Companies treating agents as drop-in replacements are measuring the wrong thing. The ones pulling ahead are redesigning the workflow so a 47% model still produces a reliable outcome, because a human owns the other 53%.

The winners in this cycle won't have the best model. They'll have the best system built around an imperfect one.

What's the highest-value task you'd trust an AI agent with today — and what would have to be true before you'd trust it unsupervised?