Top LLM Evaluation Benchmarks: Measuring Reasoning, Code, and Agents

Evaluating a large language model isn't one test — it's five or six different tests wearing the same trench coat. A model that reasons beautifully through a logic puzzle can still hallucinate a function signature, miscount a sum, or forget which tool it already called. "LLM evaluation" as a single phrase hides a fact every production team eventually learns the hard way: reasoning, coding, math, tool use, and agentic workflows each fail in their own distinctive ways, so each needs its own test rig, its own metrics, and its own definition of "passing." 🧪

This matters because the cost of getting it wrong isn't abstract. A coding agent that passes a synthetic benchmark but breaks on a real, messy 40,000-line repository ships bugs into production. A tool-using agent that never gets tested on saying "I shouldn't call anything here" will confidently invoke a refund API with the wrong amount. A math-capable model that's graded only on final-answer accuracy can get the right number through completely broken reasoning — and nobody notices until it's wrong on a problem that looks similar but isn't. Evaluation is the seatbelt, not the speedometer. ⚠️

🔄 A note before you read on: the model names below will age fast

Every model named in this post reflects publicly reported behavior as of late September 2026. New frontier releases from OpenAI, Anthropic, Google DeepMind, xAI, and the open-weight labs (DeepSeek, Qwen, Meta, Mistral) now ship roughly every few weeks, so any "current leader" claim below has a short shelf life. Instead of chasing a moving target with a static list, bookmark one of these continuously-updated, independently-run trackers and check them whenever you're actually choosing a model:

• Artificial Analysis — cross-vendor benchmark, price, and speed comparison updated as new models ship.
• LMArena — community-run, crowdsourced head-to-head model rankings (formerly known as Chatbot Arena), open methodology.
• Hugging Face Open LLM Leaderboard — open-weight model rankings, reproducible and community-auditable.
• SWE-bench.com and the Berkeley Function Calling Leaderboard — the live, category-specific leaderboards referenced throughout this post.

Treat every model name below as "an example of the type," not a permanent ranking.

📑 In This Post

1. Quick Comparison: How the Five Categories Are Actually Tested
2. Multi-Step Reasoning: Grading the Path, Not Just the Answer
3. Coding Models: Evaluation Through Execution, Not Vibes
4. Mathematical Problem Solving: When the Right Answer Is the Wrong Signal
5. Tool Use: The Four-Step Contract Most Teams Only Test Once
6. Agentic Workflows: Evaluating a Journey, Not a Turn
7. Rolling This Out at Enterprise Scale
8. Common Mistakes (and Why They Happen)
9. FAQ
10. References & Further Reading
11. Summary

🔀 Quick Comparison: How the Five Categories Are Actually Tested

Before diving in, here's the shape of the problem. Each category needs a different "grader," and mixing them up is where most eval programs quietly go wrong.

Category	Primary Grader	What Breaks Without It
Multi-step reasoning	Human/LLM-judge on the chain, not just the final token	Models get lucky answers via broken logic; failures look identical to successes until the next slightly-different question.
Coding	Execution: unit tests, sandboxes, repo-level checks	Plausible-looking code that doesn't run, or "passes" by editing the test instead of fixing the bug.
Mathematical reasoning	Symbolic/step verification, not just final-answer match	Right answer, wrong method — a model that "gets lucky" fails silently on the next variant of the problem.
Tool use	Deterministic schema checks + irrelevance detection	Over-calling (invoking tools it shouldn't) or wrong-argument calls that pass a demo but corrupt real transactions.
Agentic workflows	Task-completion + trajectory review across many turns	An agent that "looks busy" for 40 steps but never actually completes the user's goal, burning cost and time.

1. Multi-Step Reasoning: Grading the Path, Not Just the Answer

🧒 Kid Analogy

Imagine a kid solving a maze by guessing every turn at random and occasionally stumbling onto the exit. If you only check "did they get out?", that kid looks exactly as good as one who carefully traced each path. You have to watch how they moved through the maze, not just whether they reached the end. 🧩

Real-world example: the field's answer to "how do we tell genuine reasoning from a lucky guess or memorized pattern?" has been a wave of benchmarks explicitly designed to resist shortcuts. The ARC Prize Foundation's ARC-AGI-2 presents novel abstract-reasoning puzzles that can't be solved by recalling a memorized answer, because each puzzle is generated to be unlike anything in typical training data. Separately, a large group of domain experts built Humanity's Last Exam (HLE) — thousands of graduate-and-above-level questions across dozens of fields — specifically because older benchmarks like MMLU had become "saturated," meaning frontier models were scoring so highly that the test could no longer tell strong models apart. Google DeepMind, OpenAI, and Anthropic all reference GPQA Diamond and HLE-style evaluations in their own model system cards as reasoning-ability signals, precisely because final-answer-only benchmarks stopped being discriminating.

Mechanically, reasoning evaluation happens in two layers:

1. Outcome scoring — did the final answer match the ground truth? Cheap, fast, but blind to how the model got there.
2. Process scoring — was each intermediate step logically valid? This usually requires either a human rater trained on a rubric, or an LLM-as-judge prompted with a step-by-step grading rubric and the reference reasoning trace, since manually reading every chain-of-thought at scale isn't feasible.

Well-designed reasoning evals also test long-context dependency tracking — whether a model correctly uses a fact mentioned early in a long prompt, or silently drops it (the well-documented "lost in the middle" failure mode, where information placed in the middle of a long context window is retrieved far less reliably than information at the start or end).

🏆 Who Shows Up on These Evals

• Gemini 3 (Google DeepMind)Regularly cited for strong GPQA Diamond and general scientific-reasoning scores.
• Claude Opus / Fable (Anthropic)Frequently cited on Humanity's Last Exam and long-document reasoning, aided by very large context windows.
• GPT-5 series (OpenAI)Tends to post balanced scores across most reasoning suites rather than dominating any single one.
• Grok (xAI)Often highlighted on AIME-style math-adjacent reasoning.
• DeepSeek R1 / Qwen (open-weight)Have closed much of the gap on process-scored reasoning, at a fraction of the API cost.

Exact rank order shuffles with every release — check the live trackers above for who's ahead this week.

✅ Worked example: a customer-support triage model is asked to route a ticket based on three separate constraints buried across a long chat transcript (account tier, prior refund history, current sentiment). A process-scored eval checks whether the model actually referenced all three constraints in its stated reasoning — not just whether it happened to route the ticket "correctly," since a model that ignores two of three constraints and gets lucky on the third will fail as soon as the constraints stop aligning.

💡 Harder case: the same triage model, tested on a transcript where two of the three constraints now conflict (a high-tier customer with a history of refund abuse). A shallow eval that only checks final routing will miss that the model never surfaced the conflict at all — it just picked one constraint and ignored the tension, which is exactly the kind of silent failure that surfaces in production as an angry escalation.

🎯 Use this when: you're shipping any feature where the model's conclusion depends on combining several pieces of information — triage, planning, multi-constraint recommendations, or anything with an "if this, then that, unless this other thing" shape.

2. Coding Models: Evaluation Through Execution, Not Vibes

🧒 Kid Analogy

If a kid tells you their book report is great, you don't just trust them — you read it, or better, you check it against the rubric. Code is even less forgiving: you don't read it and nod, you run it. If it compiles and passes the tests, it's real. If it "looks right" but throws an error the moment you execute it, it was never right at all. 💻

Real-world example: the industry's dominant coding benchmark today is SWE-bench, built from real, closed GitHub issues in popular Python repositories like Django and scikit-learn. A model is given the issue description and the repository state, and its patch is judged solely by whether it makes the previously-failing tests pass without breaking the ones that were passing. Anthropic, OpenAI, and independent evaluators (including Epoch AI, which tracks a continuously-updated version of the benchmark) all reference SWE-bench Verified — a smaller, human-reviewed subset created specifically to remove mislabeled or unsolvable tasks from the original set. This is a case study in itself: the original SWE-bench had known noise (some "correct" reference solutions didn't actually reflect a fixable, well-specified issue), so the benchmark's own maintainers shipped a verification pass before anyone could trust the leaderboard.

A production-grade coding eval usually stacks four layers, each catching a different failure class:

1. Unit execution — does the generated code run at all, and pass the given tests? (Function-level benchmarks like HumanEval pioneered this, though frontier models now score high enough on it that it has lost much of its power to distinguish top models.)
2. Repository-level consistency — does the patch respect the existing codebase's conventions, imports, and dependent files, or does it work in isolation but break something two files away?
3. Repair loops — when the first attempt fails a test, can the model read the error and iterate, the way a human developer does? This is closer to how coding agents (e.g., terminal-based coding assistants) actually operate day to day than a single-shot generation test.
4. Contamination-resistant, continuously refreshed problem sets — benchmarks like LiveCodeBench pull newly published competitive-programming problems on a rolling basis specifically so a model can't have memorized the answer during training.

🏆 Who Shows Up on These Evals

• Claude Opus / Sonnet (Anthropic)Paired with Claude Code as the agentic harness; consistently cited for strong SWE-bench Verified and repository-level performance.
• GPT-5 series + Codex (OpenAI)Widely used for agentic terminal and CI-style coding workflows.
• Gemini (Google)Frequently the price-performance pick for high-volume coding pipelines.
• DeepSeek / Qwen-Coder / Llama 4 / Codestral (open-weight)The names most often benchmarked as self-hostable alternatives that have narrowed the gap with closed frontier models.

Score gaps between these move often enough that this post won't rank them — the SWE-bench.com and Artificial Analysis links above show this week's order.

# Illustrative — NOT a real vendor snippet, just the shape of a repo-level eval loop
def run_coding_eval(patch, repo_snapshot, failing_tests, passing_tests):
    sandbox = spin_up_sandbox(repo_snapshot)
    sandbox.apply(patch)
    resolved = all(sandbox.run(t).passed for t in failing_tests)
    no_regressions = all(sandbox.run(t).passed for t in passing_tests)
    return {
        "resolved": resolved,
        "regression_free": no_regressions,
        "score": 1.0 if resolved and no_regressions else 0.0,
    }

✅ Worked example: a repository maintenance agent is given a real, previously-filed bug report about a date-parsing edge case. The eval doesn't just check "did the reported test pass" — it re-runs the entire existing test suite, because a fix that solves the reported bug by special-casing the input can quietly break three unrelated tests elsewhere in the file.

💡 Harder case: the same agent is given a vague issue ("dates sometimes look wrong") with no reproduction steps. This tests something SWE-bench-style benchmarks are explicitly weaker at: whether the model can ask a clarifying question or investigate the codebase first, rather than guessing at a fix — a very common real-world scenario that clean, well-specified benchmark issues don't capture.

🎯 Use this when: any coding assistant or agent is being evaluated for a codebase your team will actually maintain — not a one-off snippet generator.

3. Mathematical Problem Solving: When the Right Answer Is the Wrong Signal

🧒 Kid Analogy

A kid can get "42" as the answer to a word problem by adding two numbers that happen to sum to 42 — not because they understood the problem, but because they got lucky with which numbers they grabbed. A good math teacher checks the work, not just the answer at the bottom of the page. ➗

Real-world example: older math benchmarks like GSM8K and the original MATH dataset have become so thoroughly solved by frontier models — often approaching or exceeding 95%+ accuracy — that evaluation teams across the industry now treat them mainly as regression sanity checks rather than differentiators. The benchmark that took over the discriminating role is the AIME (American Invitational Mathematics Examination) problem set, refreshed with each year's new competition problems specifically to reduce the odds that a model has memorized the answer key from training data. Model vendors including OpenAI, Anthropic, and Google DeepMind now routinely report AIME-style scores in model system cards precisely because it forces genuine multi-step derivation rather than pattern lookup.

The core lesson for teams building their own math evals: final-answer matching alone is a weak signal. A robust math eval typically layers:

1. Answer-only match — fast, cheap, but blind to method. Useful only as a first filter.
2. Step verification — either symbolic (checking each algebraic transformation is valid using a computer-algebra system) or LLM-judge-based (comparing the reasoning trace to a reference derivation).
3. Perturbation testing — re-running the same problem with the numbers changed. A model that "understood" the method should transfer; a model that memorized a specific instance of the problem will fail the moment the surface details change, even though the underlying math is identical.

🏆 Who Shows Up on These Evals

• DeepSeek R1 (open-weight)Widely cited on MATH-500 and competition-style problem sets, often at a fraction of closed-model API cost.
• Grok (xAI)Frequently highlighted specifically on AIME-style contest math.
• Gemini 3 (Google)Claude Opus / Fable (Anthropic)Both post strong GPQA- and AIME-adjacent scores in their own system cards.
• GPT-5 series (OpenAI)Commonly used as the balanced, general-purpose baseline math evaluators compare newer entrants against.

As with coding, treat any specific score as a snapshot — check the live trackers above before a procurement decision.

✅ Worked example: a finance-adjacent assistant is asked to compute compound interest across several years with a mid-period rate change. Perturbing the interest rate and time horizon between eval runs (same structure, different numbers) reliably separates models that learned the underlying compounding logic from ones that pattern-matched a similar-looking training example.

💡 Key warning: a model scoring high on GSM8K or MATH today tells you almost nothing about frontier capability — those sets are largely saturated. If your eval suite still leans on them as headline numbers, you're measuring last generation's ceiling, not this generation's floor.

🎯 Use this when: your product makes numeric claims a human will act on — pricing, scheduling, unit conversions, financial estimates — where a confidently wrong number is worse than a visible "I'm not sure."

4. Tool Use: The Four-Step Contract Most Teams Only Test Once

🧒 Kid Analogy

Think of a kid using a vending machine. They have to decide whether they need to use it at all, pick the right machine, put in the right coins, and then actually use what comes out instead of staring at it confused. Miss any one of those four steps and the snack never gets eaten — even if the other three went perfectly. 🎰

Real-world example: UC Berkeley's Berkeley Function Calling Leaderboard (BFCL), first introduced through the Gorilla project and now widely cited as the de facto standard for this category, evaluates exactly this multi-step contract. It scores whether a model correctly decides to call a function versus staying silent, whether it selects the right function from a large candidate set, and whether its generated arguments match the expected schema — using an abstract-syntax-tree comparison method so it can scale to thousands of possible functions without needing a human to check each one by hand. Crucially, BFCL includes a dedicated "irrelevance" category: prompts where the correct behavior is to call nothing at all. This exists because early tool-using models were shown to over-call — invoking a tool confidently even when none of the available tools actually applied to the request.

A production tool-use eval generally scores four separate steps rather than collapsing them into one number:

1. Call-or-no-call — precision and recall on a deliberately-included slice of prompts where the right move is to say nothing.
2. Tool selection — did it pick the correct tool out of the available catalog, especially when several tools look superficially similar?
3. Argument construction — do the generated arguments match both the schema and the user's actual intent (a syntactically valid but semantically wrong argument, like a right-shaped but wrong-value amount, still fails)?
4. Result integration — after the tool returns data, does the model faithfully use it in its final response, rather than paraphrasing it into something subtly incorrect?

Multi-turn tool use adds a further wrinkle that single-call benchmarks miss entirely: benchmarks such as τ-bench specifically test tool-agent-user interactions that span several conversational turns, because a model that nails an isolated function call often loses track of state (what it already booked, refunded, or confirmed) once a real conversation with back-and-forth clarification is layered on top.

🏆 Who Shows Up on These Evals

• Claude (Anthropic)Frequently referenced for tool-use and computer-use reliability, including native structured function-calling and multi-step tool chains.
• GPT-5 series (OpenAI)Commonly benchmarked on BFCL and τ-bench-style multi-turn tool tasks given its function-calling API's wide production adoption.
• Gemini (Google)Regularly evaluated on tool use tied to native multimodal and search-grounding capabilities.
• Qwen / Llama 4 (open-weight)Now ship dedicated function-calling fine-tunes specifically to compete on BFCL-style scores.

BFCL and τ-bench are both live leaderboards — checking gorilla.cs.berkeley.edu directly always beats a static ranking here.

✅ Worked example: a support agent is given a refund request with an ambiguous amount ("refund my last order"). Scoring only "did a tool get called with a valid schema" would pass a call that refunds the wrong order — the eval has to separately check that the arguments, not just the call shape, matched the user's actual intent.

💡 Key warning: teams that only test the "happy path" — a clean prompt where a tool obviously should be called — never discover their over-calling problem until a real user asks something adjacent-but-irrelevant and the model fires a tool anyway. Irrelevance-detection prompts need to be a deliberate, sized slice of every tool-use eval set (BFCL-style guidance suggests roughly 10–15% of the set), not an afterthought.

🎯 Use this when: your model has access to anything that changes real-world state — payments, bookings, database writes, sending messages — where a wrong or unnecessary call has a cost outside the chat window.

5. Agentic Workflows: Evaluating a Journey, Not a Turn

🧒 Kid Analogy

Grading a single answer is like grading one photo. Grading an agent is like reviewing an entire school field trip — did the group actually get on the right bus, visit the museum, stay together, and come home, or did they wander off, backtrack twice, and technically end up near the school by accident? The destination matters, but so does whether the route made sense. 🗺️

Real-world example: agentic evaluation has moved well past single-turn benchmarks toward environments that simulate real, multi-step work. Benchmarks like OSWorld test whether an agent can operate a real computer desktop across dozens of actions to complete a task; BrowseComp-style evaluations test agentic web research where the agent must search, click, and synthesize information across many pages rather than answering from a single retrieved snippet; and long-horizon coding-agent benchmarks explicitly measure whether an agent can sustain a coherent plan across software changes that span multiple files and multiple work sessions, rather than a single isolated patch. Model vendors including OpenAI and Anthropic reference this class of long-horizon, tool-rich environment in their own system cards specifically because single-turn benchmarks systematically overstate how reliable a model will be once it's running unsupervised for an extended agentic session.

🏆 Who Shows Up on These Evals

• Claude Opus / Sonnet + Claude Code (Anthropic)Frequently cited on long-horizon coding-agent and computer-use benchmarks.
• GPT-5 series + Codex (OpenAI)Commonly evaluated on terminal-style and desktop-operation benchmarks.
• Gemini (Google)Regularly highlighted on agentic web-research benchmarks like BrowseComp, given native search and multimodal grounding.
• Grok (xAI)Evaluated on multi-agent debate and self-verification setups aimed at reducing hallucination during long sessions.

Agentic-benchmark rankings shift release to release — this is exactly the category where checking a live tracker matters most.

Grading an agent well requires looking at more than the final outcome:

1. Task completion rate — did the agent actually achieve the user's underlying goal, end to end (the "did the field trip get home" check)?
2. Trajectory quality — how many wasted or redundant steps did it take along the way? Two agents can both "succeed," but one burns three times the cost and latency doing it.
3. Recoverability — when a step fails (a tool errors out, a page doesn't load, a file doesn't exist), does the agent notice and adapt, or does it plow ahead as if nothing went wrong? This "self-correction under failure" behavior is increasingly treated as its own scored dimension, separate from raw task success.
4. Stopping behavior — does the agent know when it's done, or does it keep looping past task completion, burning budget on an already-finished job?

✅ Worked example: a research agent is asked to compile a competitor pricing comparison from five different websites. A trajectory-aware eval flags the run even if the final table is correct, because the agent visited the same page four times and never noticed one of the five competitor sites had already changed its pricing tier structure mid-session — a redundancy and staleness issue invisible to an outcome-only score.

💡 Harder case: the same agent hits a page that returns a login wall partway through. A shallow eval that only checks final output completeness rewards an agent that silently fabricates a plausible-looking number for the blocked competitor rather than flagging the gap — this is exactly the failure mode that long-horizon, recoverability-scored evals are designed to catch and single-turn benchmarks cannot see at all.

🎯 Use this when: you're shipping anything that runs for more than one turn without a human checking each step — research agents, multi-file coding agents, or any workflow automation running unattended.

6. Rolling This Out at Enterprise Scale

Building one good eval is a weekend project. Keeping five categories of evals honest across dozens of model and prompt changes, month after month, is an organizational problem. A handful of practices consistently separate teams that trust their eval numbers from teams that quietly stop trusting them:

1. Ownership and governance — someone specific owns each eval suite's definition of "passing," the same way someone owns a production alert's threshold. Without a named owner, thresholds drift silently as different engineers tweak the harness for their own convenience.
2. Test-set versioning and drift tracking — golden datasets need version numbers, exactly like code. When user behavior shifts (new product features, new slang, new edge cases showing up in real traffic), the golden set has to be revisited on a cadence, or it starts measuring a world that no longer exists.
3. CI-gated evaluation — every prompt or model change runs the relevant category's eval suite before it merges, the same way a code change can't merge with failing unit tests. This is what turns "we ran an eval once before launch" into an ongoing safety net.
4. Access control for evaluation data — golden sets increasingly include real (de-identified) user traffic, especially for agentic and tool-use evals where synthetic data can't capture real messiness. That data needs the same access controls, retention limits, and review as production user data, because it often is production user data.
5. Cost governance for LLM-as-judge calls — using a large model to grade another model's output scales cost linearly with eval volume. Enterprise teams typically cap judge-model spend with sampling strategies (grade every run for critical categories like tool use, sample a percentage for lower-risk categories) rather than grading every single output with the most expensive available judge.
6. Separate dashboards for training-time vs. inference-time metrics — a model's benchmark score at release time answers "was this a good model to ship." A production monitoring dashboard answers a completely different question: "is this same model still behaving the same way on today's real traffic." Conflating the two into one dashboard hides regressions that only show up after deployment (a subtle prompt template change upstream, a shift in the mix of user requests, a silent provider-side model update).
7. Alerting on quality regressions, not just latency or error-rate regressions — a coding agent that starts silently editing tests instead of fixing bugs, or a tool-use agent whose irrelevance-detection rate quietly drops, won't trip a traditional uptime alert. Quality metrics need their own alerting thresholds tied directly to the category-specific evals above.

Common Mistakes (and Why They Happen)

• Relying on one aggregate score. A single composite number averages away exactly the information you need — a model can be excellent at math and mediocre at tool use, and an averaged score hides that trade-off from anyone deciding whether to ship it for an agentic product.
• Using the same model as both generator and judge without a bias check. Models tend to rate their own output style, phrasing, and reasoning approach more favorably than an equally correct answer written differently — this is a well-documented self-preference bias in LLM-as-judge setups, and it's why serious eval programs cross-check judge scores against a different model family or human raters on a sample.
• No held-out test set — eval-on-training-data leakage. If any part of your golden set could plausibly have been in a model's training data (or in the training data of the judge model), a high score tells you about memorization, not capability. This is precisely why the field keeps rotating to newer benchmarks (this year's AIME set, freshly-pulled LiveCodeBench problems) as older ones age into the training window of newer models.
• Ignoring latency and cost as first-class eval dimensions. A model that scores two points higher on accuracy but costs five times more per call, or takes three times as long, may be the objectively worse choice for a given product — accuracy-only leaderboards routinely miss this trade-off entirely.
• Treating an offline eval pass as sufficient without live monitoring. Offline evals test a fixed, known distribution of inputs. Real users ask things your golden set never anticipated. A model that passed every offline check can still degrade in production the moment traffic patterns shift.
• Letting golden datasets go stale. Products change, user vocabulary changes, and edge cases evolve. A golden set frozen from eighteen months ago is measuring a product that no longer exists — it needs the same maintenance discipline as any other piece of infrastructure.

❓ FAQ

Why can't I just use one benchmark for "overall intelligence"?

Because no single benchmark tests every failure mode. A model can top a reasoning benchmark and still make basic tool-calling mistakes, or excel at coding while performing poorly on long-horizon agentic tasks. Category-specific evaluation exists precisely because these are different skills that don't transfer perfectly between each other.

Are benchmarks like GSM8K and HumanEval still useful?

Mostly as regression sanity checks now, not differentiators. Frontier models score high enough on both that they no longer separate top-tier models from each other — teams typically keep them as a "did we badly break something basic" tripwire, while relying on harder, fresher benchmarks (AIME, LiveCodeBench, SWE-bench Verified, GPQA Diamond, HLE) for actual model comparison.

What's the biggest blind spot in tool-use evaluation?

Irrelevance detection — testing whether a model correctly does nothing when no available tool actually fits the request. Teams that only test scenarios where a tool obviously should be called never discover their model's over-calling tendency until it fires an unnecessary or wrong action against a real system.

How is evaluating an agentic workflow different from evaluating a single response?

A single response is graded on one output. An agentic workflow has to be graded on the entire trajectory — how many steps it took, whether it recovered from failures along the way, and whether it recognized when the task was actually done — because two agents can reach the same final answer through very different (and very differently costly) paths.

Do I need a human in the loop if I already have LLM-as-judge scoring?

Yes, at least on a sample. LLM judges are efficient at scale but carry their own biases (favoring their own generation style, being fooled by confident-sounding but wrong reasoning). Spot-checking judge decisions against human review is how teams catch a judge that's silently drifted or developed a systematic blind spot.

🔗 References & Further Reading

• SWE-bench (official benchmark site): https://www.swebench.com/
• Berkeley Function Calling Leaderboard (Gorilla project, UC Berkeley): https://gorilla.cs.berkeley.edu/leaderboard.html
• ARC Prize / ARC-AGI benchmark: https://arcprize.org/
• OpenAI model and system documentation: https://openai.com/research/
• Anthropic model system cards and research: https://www.anthropic.com/research
• Google DeepMind model cards and publications: https://deepmind.google/research/publications/
• Live, continuously updated model trackers (use these for current model names/rankings, not this post): Artificial Analysis, LMArena, Hugging Face Open LLM Leaderboard

Product and organization names above (OpenAI, Anthropic, Google DeepMind, Berkeley/Gorilla, ARC Prize) are trademarks of their respective owners, referenced here for identification only. 

📝 Summary

• Multi-step reasoning needs process scoring (grading the chain, not just the answer) plus long-context dependency checks.
• Coding needs execution — unit tests, repo-level regression checks, and repair-loop testing — never just "does it look right."
• Math needs step verification and perturbation testing, because final-answer matching alone rewards lucky guesses.
• Tool use needs its four-step contract scored separately: call-or-no-call, tool selection, argument accuracy, and result integration — with a deliberate irrelevance-detection slice.
• Agentic workflows need trajectory-level review — completion, efficiency, recoverability, and stopping behavior — not just a final output check.
• At enterprise scale, evaluation needs the same rigor as production code: owned, versioned, CI-gated, access-controlled, cost-governed, and separately monitored between training-time and inference-time.
• The most common failure across all five categories is trusting one number instead of a category-appropriate metric suite — that's the habit worth breaking first.