LLM EvaluationAgentic AIFailure ModesAI Safety

How Most LLM Evaluation Frameworks Miss Agentic Failure Modes

This article delves into the limitations of current LLM evaluation frameworks, highlighting how they often overlook critical agentic failure modes. It explores the nuances of evaluating AI systems that exhibit goal-oriented behavior and interact with dynamic environments, proposing new perspectives

·25 min read
blog cover image
Table of Contents

Final-answer accuracy stays flat while the agent gets worse everywhere that matters.

01 THE PROBLEM

Agentic failure is the class of failure where an LLM system produces an acceptable final answer while the underlying decision process degrades.

That distinction matters because most evaluation frameworks still score the visible artifact: the final text, a pass/fail label, maybe a reference answer match, maybe an LLM-judge score. That works for single-shot tasks. It breaks for agents.

An agent can return the right answer after making the wrong retrieval call, choosing the slower tool, retrying six times, hitting a rate limit, ignoring a warning, or looping through irrelevant subgoals. The user sees “correct.” Your system pays in latency, cost, fragility, and eventually trust.

This is the evaluation gap: output quality remains stable while trajectory quality deteriorates.

The production consequence is not abstract. It shows up within weeks of shipping an agent into a live environment with real tool latency, partial outages, stale knowledge, changing APIs, and shifting user intent. You do not first see it as a dramatic collapse. You see it as a slow rise in p95 latency, token spend per successful task, timeout rate, and human escalations on a subset of edge cases.

The top-line benchmark does not move enough to trigger alarm.

That is why standard LLM evaluation is structurally blind to agentic failure modes.

The recent paper Evaluating Agentic AI in the Wild: Failure Modes, Drift Patterns, and a Production Evaluation Framework makes the core point cleanly: standard output metrics can remain stable while only cross-signal analysis between tool health and decision quality exposes degradation. That is the right diagnosis. The missing signal is not “did the model answer correctly?” It is “did the system make the right sequence of decisions under real operating conditions?”

Most teams are still evaluating agents like chatbots.

That is the original sin.

A chatbot can be judged at the response boundary. An agent cannot, because the agent’s job is not just to generate language. Its job is to choose actions under uncertainty.

If the system can search, retrieve, call internal APIs, generate SQL, manipulate files, trigger workflows, or ask clarifying questions, then the unit of correctness is not just the final output. It is the trajectory.

Trajectory failures are what kill production agents:

  • selecting the wrong tool
  • selecting the right tool with the wrong arguments
  • selecting the right tool too late
  • failing to recover after a tool error
  • repeatedly re-planning after receiving sufficient evidence
  • over-executing beyond the minimum steps required
  • accepting weak evidence and stopping too early
  • producing the correct answer for the wrong reason

These are not academic edge cases. They are the normal failure surface of any agent with permissions.

And they are exactly what most eval setups do not capture.

A final-answer metric misses whether the agent:

  • made 2 tool calls or 12
  • read the authoritative record or a stale cache
  • used a fallback pathway because the primary dependency was down
  • violated a latency or cost budget to get the answer
  • got “lucky” because the benchmark environment was unusually clean
  • failed on recovery behavior that only appears under load or partial failure

If you are a CTO or Staff+ engineer evaluating vendors or building internal agent infrastructure this quarter, this distinction should drive procurement, architecture, and staffing decisions.

Because the wrong evaluation model creates a dangerous illusion: the agent appears production-ready right until it starts making expensive, low-observability mistakes in workflows that touch customers or money.

02 WHY IT HAPPENS

The root cause is simple: most LLM evaluation frameworks were built for model outputs, not for decision-making systems.

That legacy shows up everywhere.

OpenAI-style evals, benchmark suites, and many internal test harnesses grew out of prompt evaluation. Their basic shape is familiar: input, expected output, judge. Even when teams add an LLM-as-judge or semantic grading, they are still collapsing the entire execution path into one verdict at the end.

That is convenient. It is also the wrong abstraction for agents.

An agent is a distributed system wearing a language interface.

It inherits failure modes from both worlds:

  • model failure modes: hallucination, brittle reasoning, context-window errors
  • distributed systems failure modes: timeouts, stale reads, partial outages, retries, eventual consistency, race conditions, bad retries, unbounded fan-out

If your evaluation layer only sees the language boundary, it misses the distributed-systems half of the problem.

This is not new in engineering generally. Google’s Site Reliability Engineering book made the case years ago that user-visible availability is not enough to run production systems; you need internal signals, error budgets, and service-level indicators tied to actual system behavior. Agent evaluation has not caught up to that lesson.

The structural reason most teams miss this is incentive design.

Teams are rewarded to show benchmark wins, faster demos, and visible answer quality improvements. They are rarely rewarded early for reducing unnecessary tool hops, building trajectory taxonomies, or creating replayable failure corpora. Those are slower, less demo-friendly investments.

So teams optimize what is legible:

  • benchmark pass rate
  • average judge score
  • overall task completion
  • side-by-side preference wins

Those metrics are not useless. They are incomplete.

Agents also create a second structural problem: non-determinism compounds across steps.

A single completion can vary. A multi-step plan with retrieval, tool calls, retries, conditional logic, and self-reflection varies far more. A benchmark with one “golden path” overstates reliability because real agent trajectories branch based on tool health, content freshness, and ambiguity in the input.

This is why replayability matters so much. Without deterministic enough traces, you cannot tell whether a changed result came from the model, the tool, the environment, or simple randomness.

Stripe’s engineering culture is useful here, even though the company’s public writing is not specifically about LLM agents. Stripe has consistently emphasized instrumentation, high-fidelity test environments, and operational rigor because money-moving systems punish ambiguity. That same mindset applies to agents with business-critical permissions. If the system can take action, observability cannot stop at the UI.

A third reason this happens: most eval frameworks treat tools as inert helpers.

In production, tools are not helpers. They are volatile dependencies.

Search quality drifts. Permissions change. Internal APIs add fields, deprecate fields, or silently alter behavior. A knowledge base gets partially reindexed. A vendor rate-limits at a lower threshold than last month. A CRM record is present but stale. A SQL warehouse returns data late because a downstream pipeline missed its SLA.

The agent has to make decisions inside that moving environment.

That means your evaluation target is not “Can the model answer this question?” It is “Can the system still make good decisions when one or more assumptions about the environment stop holding?”

Most teams do not test that.

They test the happy path, often against a frozen dataset, then call the agent “92% accurate.”

That number has very little operational meaning.

The Evaluating Agentic AI in the Wild paper points to continuous temporal drift as a defining challenge. That matches what experienced operators already know. A benchmark is a snapshot. Production is a stream.

There is another uncomfortable truth: agents can overfit to the evaluation harness.

The moment you define a narrow benchmark set with obvious expected outputs, the model and the team begin to optimize for those cases. You get cleaner traces, stronger final-answer metrics, and fewer visible regressions. But you also risk shrinking the behavioral distribution.

The system becomes better at looking correct in the eval environment and worse at exploring recovery strategies, asking clarifying questions, or escalating uncertainty in the wild.

This is not unique to LLMs. It is the same anti-pattern DORA research warns against in another form: optimizing the metric instead of the capability. Nicole Forsgren, Jez Humble, and Gene Kim’s work in Accelerate and the DORA reports repeatedly show that strong operational performance comes from improving system capabilities, not gaming lagging indicators.

For agents, the capability is robust task execution under uncertainty.

If your framework cannot see uncertainty-handling behavior, it cannot evaluate the thing you actually care about.

03 WHAT MOST GET WRONG

The most common mistake is thinking a better judge fixes a bad unit of evaluation.

It does not.

Teams notice string-match grading is brittle. They move to semantic similarity or LLM-as-judge. That is a genuine improvement for answer equivalence. It solves obvious false negatives like paraphrases and reordered reasoning.

It does not solve agent evaluation.

If the final answer is right, an LLM judge will often give the agent full credit even when the path was terrible.

That means the team upgrades the grader while keeping the wrong target.

This failure pattern shows up in comparative evaluations too. The paper Comparative Analysis of Agent Evaluation Frameworks: Stability, Detection, and Discovery in Enterprise Analytics Agents found that structurally detectable failures are easy to catch with simple checks, but semantically wrong behavior in agent trajectories is much harder. On one agent, regex checks performed well because the errors were surface-level. On another, regex-based evaluation dropped to 0% recall because the bug lived in semantic logic, not output structure.

That is the exact trap.

Teams overinvest in judging the artifact and underinvest in instrumenting the process.

The second mistake is treating all successful tasks as equal.

They are not.

If one trajectory completes in 3 tool calls and 9 seconds, and another completes in 11 tool calls and 41 seconds, those are not equivalent production outcomes. One is likely robust. The other is likely one API hiccup away from failure.

This matters because user-facing reliability degrades before answer quality collapses.

A support agent that still resolves the ticket but now needs two retries on every CRM lookup is already failing operationally. A coding agent that eventually generates the right SQL after scanning irrelevant schemas is already accumulating cost and fragility. A workflow agent that succeeds only by brute-force retries is hiding an incident waiting to happen.

Third mistake: teams do not define a failure taxonomy before they start evaluating.

Without a taxonomy, every bad run looks like “the model was wrong.”

That is too coarse to debug and too vague to improve.

In practice, agent failures usually fall into a smaller number of buckets:

  1. Planning failures
Wrong decomposition, skipped prerequisite, invalid assumption.
  1. Tool selection failures
Chose search instead of authoritative API, used public web when internal source existed.
  1. Tool execution failures
Bad parameters, malformed schema, pagination mishandling, auth errors.
  1. State management failures
Forgot previous result, contradicted prior evidence, lost user constraints.
  1. Recovery failures
Did not retry when it should, retried when it should not, failed to escalate.
  1. Termination failures
Stopped too early, looped too long, failed to ask for clarification.
  1. Policy failures
Took action without required confidence, used disallowed data, violated approval boundary.

Most frameworks never force you to label runs this way. So teams cannot see which failure mode is rising over time.

Fourth mistake: evaluating agents in benchmark conditions that remove the hard parts of production.

This one is subtle and expensive.

A pristine eval harness often has:

  • healthy tools
  • low latency
  • deterministic tool outputs
  • complete data
  • no permission edge cases
  • no interrupted workflows
  • no concurrent updates
  • no stale indexes
  • no user ambiguity

That is not agent evaluation. That is benchmark theater.

Netflix’s engineering culture is a useful parallel here. Netflix became well known for failure injection and chaos engineering because distributed systems behave differently under stress than under ideal conditions. Agents are no different. If you only test them when every dependency is healthy, you do not know how they behave in production.

Fifth mistake: no cost or latency budgets inside the eval itself.

This is where technically solid teams still make bad product decisions.

Suppose Agent A and Agent B both hit 88% task success on your benchmark. If Agent A averages 18 seconds and 14 tool calls while Agent B averages 6 seconds and 4 tool calls, they are not tied. If you ship Agent A at scale, your unit economics and user tolerance will punish you.

The Google SRE model is relevant again: a metric without an objective is not operationally useful.

For agents, “task success” without:

  • max latency target
  • tool-call budget
  • cost per completed task
  • escalation rate
  • retry ceiling

is not a decision metric.

Sixth mistake: no distinction between recoverable and non-recoverable failure.

An agent that catches a transient API error, switches to a fallback, and completes successfully should score differently from one that only succeeds because the benchmark never included failures in the first place.

Recovery is a first-class capability.

Cloudflare has written extensively about designing for reliability at the edge under volatile network conditions. The lesson transfers cleanly: resilience is not the absence of failure. It is graceful behavior during failure.

Agents need the same treatment.

Seventh mistake: teams believe vendor dashboards equal evaluation strategy.

They do not.

LangSmith, Braintrust, OpenAI Evals, and similar tools can be useful. They make traces visible, simplify dataset management, and support custom scoring. But no vendor can define your business-critical failure modes for you.

If your agent touches refunds, compliance approvals, CI/CD changes, customer support actions, or internal analytics queries, the highest-value eval cases will come from your own workflow boundaries and incidents, not a generic benchmark.

The cost of getting this wrong is not just model underperformance.

It is organizational delay.

A team that cannot localize failure ends up debating prompts, models, and frameworks in circles. They rerun benchmarks, switch judges, and compare leaderboard deltas while the production problem is hidden in tool orchestration or recovery logic. Weeks disappear.

For a Series B company trying to prove AI product retention this quarter, that delay is expensive.

04 THE FRAMEWORK

The evaluation framework that works for agents starts from one premise: the trajectory is the product.

Everything else follows from that.

Below is the practical structure I would use for any company moving from prototype to production agent behavior.

1. Define the unit of evaluation as task + trajectory + budget

Do not score just “Was the final answer correct?”

Score each run across four dimensions:

  1. Task outcome
Did the agent complete the task correctly?
  1. Trajectory quality
Did it use the right sequence of decisions?
  1. Operational efficiency
Did it stay within latency, token, and tool-call budgets?
  1. Safety and control compliance
Did it respect approval, data, and action boundaries?

This immediately changes what gets logged.

For every run, capture at minimum:

  • user intent / task ID
  • plan steps generated
  • each tool selected
  • tool arguments
  • tool outputs or hashes
  • retries and backoffs
  • branch points
  • final answer
  • completion status
  • human intervention required
  • wall-clock latency
  • token usage
  • cost estimate

If you cannot reconstruct the run, you cannot evaluate the run.

This is where a trace-based architecture matters. You want something closer to application telemetry than prompt logging.

Datadog’s engineering approach to observability is instructive here: correlated traces across services are more valuable than isolated metrics. Agents need the same correlation across model steps and tool interactions.

2. Build a failure taxonomy before expanding the dataset

Do not start by collecting 10,000 eval examples.

Start by defining 20–40 failure labels that matter to your system.

A lightweight taxonomy could include:

  • wrong tool selected
  • wrong tool parameters
  • unnecessary tool call
  • redundant replan
  • failed to ask clarification
  • relied on non-authoritative source
  • ignored tool error
  • exceeded retry threshold
  • exceeded latency budget
  • exceeded token budget
  • unsafe action attempted
  • failed to escalate uncertainty
  • looped without progress
  • stale context used
  • terminated prematurely

Then annotate a small but high-quality corpus manually.

For a first production pass, 200–500 carefully labeled traces are worth more than 10,000 weakly defined benchmark rows.

Why? Because you need diagnostic resolution.

This is the same principle high-performing engineering teams use in incident review. Stripe, GitHub, and Cloudflare all treat incident classification as a learning mechanism, not just a reporting exercise. If every incident is labeled “outage,” you learn nothing. If every bad agent run is labeled “hallucination,” you learn nothing.

3. Split evals into four layers

Most teams collapse evaluation into one suite. That is too blunt.

You want four distinct layers:

Layer A: Step-level correctness

Does each individual action make sense? Examples:
  • Was the right tool chosen?
  • Were the parameters valid?
  • Did the agent parse the tool result correctly?

Layer B: Trajectory-level quality

Was the full path efficient and logically sound? Examples:
  • Were steps redundant?
  • Did the sequence preserve evidence correctly?
  • Did it recover appropriately from tool issues?

Layer C: Outcome-level success

Did the task complete correctly from the user’s perspective?

Layer D: Stress and resilience behavior

What happens when tools slow down, fail, return stale data, or produce partial results?

Most eval frameworks stop at Layer C.

Production agents fail in Layers B and D first.

4. Add budgets, not just scores

Every task class should have explicit budgets.

For example:

  • customer support lookup: p95 under 8 seconds
  • analytics query generation: first valid draft under 20 seconds
  • internal workflow automation: max 5 tool calls before requiring replan
  • refund or account action: 0 autonomous execution without approval if confidence below threshold

You should also define budget breach severity.

Example:

  • latency > 2x target = degraded
  • tool calls > 150% of median successful path = inefficient
  • more than 2 retries on same dependency = likely stuck
  • token spend above 90th percentile successful baseline = investigate

Use DORA-style discipline here. The DORA four key metrics are not directly for AI agents, but the operational philosophy holds: define measurable service performance characteristics and watch trends over time, not just point estimates.

A benchmark pass without a budget is not deployable evidence.

5. Evaluate on live traces, not only synthetic tasks

Synthetic evals are useful. They are not sufficient.

Pull from:

  • recent failed production traces
  • near-misses caught by humans
  • long-latency outliers
  • high-cost successful runs
  • escalations from support or ops teams
  • incidents caused by stale or partial tool data

Then convert those into replayable evals.

This is where many teams underinvest because production data is messy. That mess is the point.

Linear’s product and engineering reputation comes partly from obsessing over real workflow friction rather than abstract feature completeness. Agent evaluation should take the same posture. Real traces reveal the awkward edge conditions synthetic tasks smooth over.

6. Create adversarial environment scenarios

A serious agent eval suite includes environmental perturbation.

Inject scenarios like:

  • primary retrieval tool returns empty set
  • CRM lookup adds 3-second delay
  • warehouse schema changed field name
  • one dependency rate-limits after first call
  • search index is 24 hours stale
  • permission denied on preferred tool
  • user prompt omits a critical constraint
  • two tools return conflicting information

Then score not only final success, but recovery strategy.

Did the agent:

  • ask for clarification?
  • switch to fallback?
  • cite uncertainty?
  • stop before unsafe action?
  • continue pointlessly?

Netflix popularized the value of testing under failure, not just under ideal conditions. For agents, this is not a nice-to-have. It is the only way to know whether your orchestration logic is robust.

7. Measure cross-signals, not isolated metrics

This is the most underused practice.

Do not track answer quality, latency, tool health, and escalation rate separately. Track their interaction.

Examples:

  • task success stable, but tool calls per task up 40%
  • judge score stable, but p95 latency doubled after retrieval index update
  • final-answer pass rate stable, but authoritative-source usage dropped from 92% to 67%
  • customer-visible completion stable, but human override rate rose in one workflow
  • output quality stable, but same-tool retry loops increased 3x

This is exactly the kind of cross-signal analysis called out in Evaluating Agentic AI in the Wild. Standard metrics miss degradation because no single metric breaches a threshold. The pattern only appears in combination.

You need dashboards and alerts for those combinations.

8. Separate exploration from execution modes

One of the deepest agent design mistakes is using the same behavior policy for every task.

Some tasks benefit from broad exploration. Others demand minimal, bounded execution.

Examples:

  • research assistant: tolerate longer trajectories, broader search
  • billing operations agent: tight action set, low retry ceiling, high approval sensitivity
  • developer agent in CI: explicit tool permission boundaries and deterministic checks before merge-impacting actions

Your eval framework should score these modes differently.

The same “curious” behavior that improves a research agent will often harm an ops agent.

GitHub’s work around Copilot and developer workflows consistently highlights the difference between assistive generation and trusted automation. That distinction should exist in your eval design too. Suggesting code and merging code are not the same risk category.

9. Use human review on disagreement slices, not everywhere

Human review is expensive. Use it surgically.

The highest-value slices for review are:

  • evaluator disagreement
  • high-cost successful runs
  • high-latency successful runs
  • first occurrence of a new failure label
  • safety-boundary edge cases
  • benchmark pass / production fail mismatches

This gives you a compact, information-rich feedback loop.

Do not waste expert reviewers on easy benchmark passes.

This mirrors how strong SRE and platform teams review operational anomalies: not every request, just the slices with unusual risk or novel failure characteristics.

10. Set release gates based on failure mode regression, not just aggregate score

Before shipping a model, prompt, tool, or planner change, require:

  • no regression in critical failure classes
  • no material rise in tool-call count for key tasks
  • no breach of latency/cost budgets on top workflows
  • no increase in unsafe-action attempts
  • stable or improved recovery under tool degradation tests

This is where leadership discipline matters.

A model upgrade that improves final-answer score by 2 points but increases unnecessary tool calls by 35% is often a net production loss.

Do not ship that change because the dashboard “looks better.”

11. Maintain a gold set of “expensive truths”

Every mature team eventually needs a manually curated set of difficult, high-value traces.

These are cases that:

  • caused incidents
  • almost caused incidents
  • exposed planner weaknesses
  • required expert adjudication
  • involve policy boundaries
  • exercise recovery behavior

Keep this set small, reviewed, and stable.

Think 50–200 cases, not thousands.

This is your agent equivalent of a regression suite for the workflows that can hurt the business. Figma, Shopify, and Vercel all have engineering cultures that emphasize feedback loops around real product experience, not just abstract correctness. Your expensive-truth set serves the same role for agent quality.

12. Tie eval ownership to the team that owns the workflow

Do not centralize all agent evaluation in a platform team.

Platform can provide infrastructure:

  • tracing
  • dataset versioning
  • run comparison
  • evaluator orchestration
  • dashboards
  • replay systems

But the workflow-owning team must define what “good” means.

A support automation team knows what a harmful escalation looks like. A revenue ops team knows which CRM fields are authoritative. A developer tools team knows which CI actions are too risky to automate. A finance team knows when a “mostly right” answer is unacceptable.

Platform ownership without domain ownership produces elegant but irrelevant evals.

That split also makes build-vs-buy decisions clearer. Use vendor tooling for generic eval plumbing if it accelerates you. Keep your domain-specific failure definitions in-house.

A concrete scorecard

For a production agent, I would score each task run with something like:

  • Outcome correctness: 0/1
  • Authoritative source used: 0/1
  • Tool choice quality: 0–2
  • Parameter correctness: 0–2
  • Recovery quality: 0–2
  • Termination quality: 0–2
  • Budget compliance:
- latency pass/fail - tool-call pass/fail - token-cost pass/fail
  • Safety compliance: pass/fail
  • Human escalation needed: yes/no

Then aggregate by workflow, not just globally.

Global scores hide too much. A 90% average can contain a catastrophic 60% on one workflow that touches refunds or deployments.

Tradeoffs this framework introduces

This approach is better. It is not free.

You will pay in:

  • instrumentation time
  • annotation effort
  • taxonomy maintenance
  • replay infrastructure
  • slower model release cycles
  • more cross-functional coordination with domain teams

For a simple internal research assistant, that may be overkill.

For any agent that can take action, touch customer data, or become part of the core product experience, it is the minimum serious posture.

The tradeoff is speed versus false confidence.

Most teams choose speed first because the benchmark is easier.

Then they spend the next quarter debugging ghosts.

05 STRATEGIC TAKEAWAY

You should treat agent evaluation as an operational reliability problem, not a model quality problem. If you apply that shift, your roadmap changes immediately: you invest less in benchmark cosmetics and more in traces, failure taxonomies, replay, and budgeted release gates. If you do not, you will likely ship an agent that looks acceptable in Q1 demos and spends Q2 accumulating latency, cost, and silent decision debt until a customer-facing workflow forces a rollback. For a CTO making platform decisions this quarter, that difference is the line between a reusable agent capability and another AI initiative that “worked in testing” but never earned production trust.

06 IMPLEMENTATION ANGLE

The most practical first move is not buying another eval product. It is instrumenting your agent runtime so every decision becomes inspectable and replayable. If you already have OpenTelemetry, use it. Represent tool calls, planner steps, retries, and approval boundaries as trace spans with consistent metadata. Once you can reconstruct runs, you can layer in vendor tooling like LangSmith, Braintrust, or custom dashboards without losing control of the underlying data model. related topic

Next, pick one workflow that matters and constrain scope aggressively. Not “customer support” broadly. Pick “order status and refund eligibility” or “internal analytics query drafting for weekly business review.” Build a 100-case eval set from real traces, label failures manually, and add two budgets: p95 latency and max tool calls. You will learn more from one narrow, operationally honest workflow in two weeks than from a broad benchmark program in two months.

If your engineering org is growing fast, this is also where team design starts to matter. A small platform layer should own trace collection, replay infra, and evaluator execution. Workflow teams should own failure definitions and release gates. Amplify can help engineering teams scale that split when headcount and ownership boundaries start getting blurry, but the operating model matters more than the logo on the tooling. The core requirement is clear accountability for what the agent is allowed to do, how it fails, and who decides that failure is acceptable.

07 FAQ

Q: Why are final-answer metrics not enough for evaluating AI agents? A: Final-answer metrics ignore the path the agent took to get there. An agent can return the correct answer after choosing the wrong tools, retrying excessively, or relying on stale data, which increases latency, cost, and fragility. The paper Evaluating Agentic AI in the Wild: Failure Modes, Drift Patterns, and a Production Evaluation Framework specifically argues that output-level metrics can remain stable while cross-signal analysis reveals degradation. Q: What is the biggest failure mode most LLM evaluation frameworks miss? A: The biggest missed failure mode is trajectory degradation without visible output failure. That means the agent still “passes” by final-answer standards while tool choice quality, recovery behavior, or budget compliance gets worse over time. This is especially dangerous in production because the first signals are often rising p95 latency, retry loops, and human escalations rather than obvious answer errors. Q: How should a company evaluate an AI agent in production? A: A company should evaluate the agent at four layers: step-level correctness, trajectory quality, final outcome, and resilience under degraded conditions. It should also score against explicit budgets for latency, tool calls, and token cost, similar to how SRE teams use service-level objectives in the Google SRE book. The minimum serious setup includes trace logging, a failure taxonomy, replayable production cases, and release gates tied to critical failure classes. Q: What metrics matter most for agent evaluation beyond accuracy? A: The most useful metrics are task success, authoritative-source usage, tool-call count, retry rate, p95 latency, token or dollar cost per completed task, human escalation rate, and unsafe-action attempts. DORA’s broader operational lesson applies here: capability and trend matter more than one benchmark snapshot. A stable accuracy score with a rising retry rate or doubled latency is already evidence of production degradation. Q: Should engineering teams build their own agent eval framework or buy one? A: Most teams should buy generic eval plumbing and build their domain-specific evaluation logic. Tools like LangSmith or Braintrust can help with traces, datasets, and experiment comparison, but they cannot define what counts as a harmful refund action, a bad CRM lookup, or an unacceptable deployment suggestion in your business. Keep workflow-specific failure taxonomies, replay cases, and release gates in-house because that is where the real risk lives.

Enjoyed this article?

Share it with your network

LatAm Engineering Insights

Stay ahead of the curve

Weekly insights on hiring LatAm developers, salary trends, tech stack analysis, and exclusive job opportunities.

No spam, unsubscribe anytime. We respect your privacy.

Salary Insights

Real market data on LatAm developer salaries

Hiring Tips

Best practices for remote LatAm teams

Exclusive Roles

Early access to new job opportunities

Join 2,500+ CTOs, Engineering Managers, and Developers