Demystifying Evals for AI Agents
by Mikaela Grace, Jeremy Hadfield, Rodrigo Olivares, Jiri De Jonghe · Anthropic
Anthropic engineering guide to evaluating multi-step, tool-using agents. Defines tasks, trials, transcripts, and outcomes. Compares code-based, model-based, and human graders, explains pass@k versus pass^k, and separates capability evals from regression evals. Anthropic's newest and most substantive public writing on evals, and the clearest vocabulary for agent evaluation, where most new eval work now happens. It complements Hamel's product-centric material with grader taxonomy and trial statistics.
More resources on LLM Evaluation (Evals)
Improving Accuracy of LLM Applications
Walks through a systematic loop for making LLM applications reliable: build evaluation metrics, apply prompting and self-reflection, then fine-tune with LoRA and memory tuning. The running example is a text-to-SQL agent that hallucinates less each iteration.
Inspect: Open-Source Framework for LLM Evaluations
Documentation and tutorial for Inspect, the UK AI Security Institute's open-source Python evaluation framework. Covers datasets, solvers, custom scorers, model grading, sandboxed agent evals, log analysis, and running evaluation sets across models, with over 200 prebuilt evals.
AI Engineering: Building Applications with Foundation Models
Chip Huyen's 2025 O'Reilly book on building foundation-model applications. Chapters 3 and 4 cover evaluation: language-modeling metrics, exact evaluation, AI as a judge, comparative ranking, evaluation criteria, model selection, and designing an evaluation pipeline. The most rigorous textbook treatment of LLM evaluation methodology. Its two evaluation chapters (about 95 pages) give the conceptual grounding that blog posts skip.
Holistic Evaluation of Language Models (HELM)
Stanford CRFM benchmark paper that evaluates 30 language models across 42 scenarios and seven metrics: accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency. Shows how benchmark coverage and standardized conditions change what model comparisons mean. HELM is the canonical argument that single-number leaderboards mislead and that evaluation needs defined scenarios and multiple metrics.
Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
UIST 2024 paper introducing EvalGen, an interface that helps people align LLM-generated graders with their own judgments. Its user study identifies criteria drift: people refine their grading criteria as they grade outputs, so criteria cannot be fixed in advance.
Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge)
Literature review of about two dozen papers on using LLMs to grade LLM output. Covers use cases, prompting techniques, aligning judges to human labels, fine-tuned evaluator models, and documented weaknesses such as position, verbosity, and self-preference bias. The standard deep reference on model-graded checks, the most error-prone part of any eval stack.