LLM Evaluation: How to Measure Real-World Performance

A strong benchmark score doesn’t necessarily carry over to tasks in a real business environment. In a 2025 study, DAIL-SQL (with GPT-4o) achieved 86.6% execution accuracy on Spider 1.0, a basic academic text-to-SQL test. However, it only achieved 5.68% accuracy on Spider 2.0-lite, a text-to-SQL benchmark built around substantially more complex enterprise data environments.

The study’s results illustrate a key challenge in LLM evaluation: measuring performance when the task, data or operating conditions change. Teams need evaluation cases that represent the work the application will perform, scoring methods that fit the output and, when an LLM judge does the scoring, evidence that its judgments line up with human reviewers. After deployment, the same evaluation process can be used to detect regressions as the system changes.

LLM evaluation is the systematic measurement of a large language model or application output against a defined quality standard. Depending on the task, that standard may include factual accuracy, coherence, instruction following, safety and groundedness.

Unlike a conventional software test or classical model evaluation that asks how well a trained model generalizes to unseen examples, an LLM evaluation must address the potential of multiple valid outputs. Two answers might use different language but be equally correct and useful, for example, while a fluent answer might score well on surface similarity but contain an unsupported claim that makes it inaccurate.

For this reason, LLM evaluation uses a combination of metrics, executable checks, human judgments and model-based judges to determine how well an output satisfies the criteria for the task.

The discipline spans more than one stage of an AI system’s lifecycle. During model selection, teams often compare several models on the same fixed set of tasks. Later, they typically evaluate a deployed application that combines retrieval, prompts, tools and post-processing with the underlying model. Those evaluations answer different engineering questions and generally require different test data.

National Institute of Standards and Technology (NIST) treats generative AI evaluation as a test-and-measurement discipline: its GenAI program runs structured evaluations designed to measure the capabilities and limitations of generative systems across multiple modalities.

Learn best practices for enabling observability in your AI workflows and common feedback functions used for generative AI application evaluation:

Benchmark scores are useful for comparing different models, but most public benchmarks aren’t designed to predict production performance. A benchmark score reflects performance on the benchmark’s task, data set and scoring method. If those conditions differ substantially from production, the score communicates very little about how the system will perform in a real business environment.

In Snowflake’s internal evaluation of text-to-SQL systems developed to mirror real-world tasks, the difference was substantial. A model scoring above 90% on Spider reached only 51% on a business intelligence evaluation set designed around real user questions and schemas. Public text-to-SQL benchmarks tend to contain relatively well-formed questions and clean schemas, but production environments contain business terminology, ambiguous requests, complex schemas and logic that may not appear explicitly in the question.

Benchmarks also have a shelf life. The Stanford Institute for Human-Centered AI’s 2026 AI Index documents rapid gains on several frontier evaluations, including cases in which performance rose sharply over a single year. As top models begin to saturate a benchmark, small score differences become less informative about which model will perform better on real application tasks.

In practice, teams face two separate measurement challenges: comparing models under controlled conditions, and testing a system against the conditions it will encounter in practice.

A model evaluation usually asks which model performs a defined task best. The test stays fixed while the candidate model changes, which makes the comparison useful for model selection, routing decisions or periodic re-evaluation as new models become available.

A system evaluation has a different objective. Here, the test covers the application pipeline: retrieval, prompt construction, model behavior, tool selection, tool calls and any post-processing that occurs before the user sees an answer.

The distinction shows up most clearly when something goes wrong. A weak model might consistently misinterpret a class of questions, for example, while a capable model might produce the wrong result because retrieval returned an irrelevant document, a tool received the wrong argument or the prompt omitted necessary context.

System evaluation is closely tied to the application’s architecture. In Snowflake’s text-to-SQL research, Cortex Analyst reached more than 90% accuracy on real-world use cases, roughly twice the accuracy of a single-prompt frontier-model approach and about 14% higher than other evaluated market solutions. The result illustrates why model choice alone doesn’t describe system performance: the surrounding retrieval, semantic context and orchestration all affect what the model receives and how its output is produced.

No single evaluation method covers every kind of output. The best choice depends largely on whether the task has an objectively checkable result and how much human judgment the quality standard requires.

• Benchmarks and leaderboards provide standardized tasks and known scoring procedures, which makes them useful for broad model comparisons. Their value falls as the target task diverges from the benchmark data.
• Rule-based and executable verifiers check outputs programmatically. SQL can be run and compared with an expected result, for example, and structured output can be validated against a schema. Where a deterministic check is available, it usually provides a cheaper and more reproducible signal than model-based scoring.
• Human review supplies the reference judgments used to establish whether other evaluation methods are behaving as intended. Humans also remain important for probing novel or ambiguous failures that an existing test suite may not contain.
• LLM-as-a-judge uses one model to score another model’s output against defined criteria or a rubric. It provides a practical way to evaluate open-ended qualities such as groundedness or relevance at greater volume, provided the judge itself has been validated.

In practice, evaluation suites often combine these methods. Verifiers handle outputs with deterministic checks, humans establish trusted labels, and LLM judges extend those judgments across larger test sets. Public benchmarks are useful upstream, where the goal is comparing candidate models rather than certifying a production application.

An LLM judge needs to be tested against the same reference used to evaluate any other scoring method: human judgments. Start with a held-out set of outputs that trained annotators have already scored, run the judge against those cases and compare the results.

The comparison should show more than a single accuracy score. Precision and recall reveal whether the judge tends to miss problems or flag acceptable answers, while agreement measures such as Cohen’s kappa show how closely its judgments track human labels after accounting for chance.

Snowflake researchers observed the significance of those distinctions when they evaluated LLM judges for groundedness, answer relevance and context relevance. The groundedness judge reached 81% F1— a measure that balances precision and recall into a single score — but its Cohen’s kappa was 0.54, indicating only moderate chance-adjusted agreement with human labels. The other judges showed very different precision and recall profiles as well. A seemingly strong aggregate score, in other words, can conceal the specific ways a judge disagrees with people.

How much disagreement is acceptable depends on what the score is used for. A judge used to surface questionable responses for human review can tolerate more noise than one used to automatically block a release, for example.

Test the judge for systematic bias

Agreement on a validation set doesn’t guarantee neutral scoring. LLM judges can show repeatable preferences that have little to do with the quality criterion being measured.

Position bias is one example. If reversing the order of two candidate answers changes which one the judge prefers, the evaluation is partly measuring presentation order. A simple consistency check — score the same pair in both positions — can expose that behavior.

Self-enhancement bias creates a different problem when the same model generates and evaluates an answer. Research has found that models may favor their own outputs, which is a good reason to separate generation from judging when practical.

Treat judge validation as a repeatable test. Record the rubric, judge model and calibration set used for each run, along with the consistency-check results. If the rubric, model or calibration data changes, validate the judge again.

Write the rubric as part of the evaluation

An LLM judge needs an explicit scoring instruction. The rubric defines the criteria being judged, the scale used to express the result and examples that anchor the meaning of each score.

For groundedness, for instance, a rubric might ask whether every substantive claim is supported by the supplied context. An answer relevance rubric would instead ask whether the response addresses the user’s question. Combining those criteria into a vague instruction to rate the answer’s “quality” makes the score harder to interpret and harder to debug.

Coarser scales can also reduce ambiguity. A four-point rubric with clearly differentiated anchors gives both human annotators and model judges fewer adjacent categories to distinguish than a 10-point scale, while still separating clearly unacceptable, mixed, acceptable and strong responses.

Once teams begin using judge scores to gate releases or monitor production behavior, the rubric effectively becomes part of the system’s test specification. Changing it changes what the evaluation measures, so rubric revisions should be versioned and revalidated rather than treated as prompt cleanup.

Many familiar language-model metrics are useful when quality can be measured against a known target. BLEU, ROUGE, METEOR and BERTScore compare generated text with one or more reference answers, using different measures of lexical or semantic similarity. Perplexity instead measures how well a model predicts a sequence of text.

These metrics are relatively inexpensive to compute and easy to reproduce, but they’re limited for open-ended generation. They also don’t directly measure whether an answer is grounded in the application’s source material or useful in a real business context.

Retrieval-grounded applications usually need additional diagnostic measurements. Three are particularly good:

• Context relevance: Did retrieval return information that’s useful for answering the question?
• Groundedness: Are the claims in the answer supported by the retrieved context?
• Answer relevance: Does the response actually address what the user asked?

These measurements help separate failures that can look identical at the final-answer level. They also explain why a single F1 score often hides important behavior. In Snowflake’s RAG-triad evaluation, answer relevance reached 99% precision and 66% recall, while context relevance reached 51% precision and 87% recall. Yet their F1 scores were much closer than those underlying numbers suggest.

For this reason, precision and recall should stay visible whenever an aggregate score is used. Composite reliability scores can still help teams track a system over time, particularly in dashboards and release gates, but the component metrics need to remain available underneath them so engineers can tell what changed.

For retrieval-grounded systems, traces add another layer of diagnosis. Recording which documents were retrieved, which context was passed to the model and how intermediate steps were handled lets the team connect a poor groundedness or relevance score to the part of the pipeline that produced it.

The evaluation data set defines the work the system is actually being tested on, so even precise scoring is misleading if the cases don’t resemble production use. The value comes from curation: each case represents a behavior the team wants to measure and includes enough context and labeling to judge it consistently.

Build cases from real usage

Production queries are usually the best starting point because they capture the terminology, ambiguity and edge cases that users actually introduce. Teams can sample representative traffic, then deliberately increase the weight of high-value workflows and failure patterns that have already surfaced.

Over time, the set can also serve as a regression record. When a production bug is fixed, adding a corresponding case to the evaluation suite helps ensure that a later change doesn't quietly reintroduce it.

That process produces what teams often call a golden set: a curated collection of examples whose expected outcomes have been reviewed closely enough to support repeatable evaluation. The goal isn’t to make the collection enormous, but rather to make the cases representative enough that changes in the score correspond to changes practitioners care about.

Keep development and held-out data separate

Once engineers begin tuning prompts, retrieval settings or judge behavior against an evaluation set, that set starts influencing the system being measured. Repeated optimization against the same cases can gradually turn the evaluation into another development target.

A held-out portion protects against that effect. Teams can use one split while iterating and reserve another for less frequent evaluation, keeping the second set out of the normal tuning loop.

Versioning is equally important. If cases, labels or rubrics change casually between runs, scores from different dates stop describing the same test. Recording the data-set version alongside evaluation results preserves comparability and gives teams a way to distinguish an application regression from a change in the test itself.

Extend governance standards to evaluation data sets

When test cases come from real customer questions, documents or traces, they inherit the sensitivity of their source data: access controls, retention rules and handling requirements apply to the evaluation set as well.

The governance consideration reaches beyond storage. Sending production-derived questions and context to an external evaluation service may create a new copy of governed data, with its own access and retention path. For teams working under security, privacy or compliance requirements, evaluation architecture has to account for those data movements explicitly.

Human labeling introduces its own operational requirements. Annotators need written guidance, examples and adjudication rules so that two people applying the same rubric reach reasonably consistent judgments.

A pre-release evaluation answers a bounded question: did the system meet its quality bar on the test set we ran before shipping? Once the application is exposed to live traffic, the inputs begin to drift away from that controlled sample. Users phrase requests differently, source data changes, retrieval indexes refresh and tool behavior evolves around the model.

Production evaluation extends the measurement discipline into that environment. Instead of treating online monitoring as a separate field, teams can apply the same judges, rubrics and diagnostic metrics to sampled live traffic, then compare those results with the baselines established before release.

Continuity is useful because it connects a regression to the same quality dimensions engineers are already using. If groundedness drops after a retrieval change, for example, the team can investigate the retrieved context rather than waiting for a generic error rate or support ticket to reveal the problem.

The evaluation suite can also sit directly in the change pipeline. A fast subset might run on every pull request, while a larger suite runs before release. When a score crosses a predefined threshold, the change can be blocked for review before it reaches production.

When production monitoring surfaces a new failure pattern, teams can add a labeled example to the offline evaluation set so future releases are tested against it.

Ready to apply these evaluation principles to AI agents? Learn how to measure not just the final answer, but also tool use, execution paths, reliability and governance across the full agent workflow. Explore AI agent evaluation >

The visible cost of model-based evaluation is inference. Every scored case requires one or more judge calls, so the basic arithmetic is straightforward:

evaluation cost = number of cases × evaluators per case × run frequency

A judge call typically costs a cent or even less, depending on the model, token usage and evaluation design, which sounds inexpensive. But a 500-case suite scored across four dimensions and run on every pull request adds up.

The largest cost differences usually come from evaluation design rather than the per-call price alone. Teams can reduce unnecessary judge inference by matching the method to the output:

• Use programmatic verifiers where a result can be checked deterministically.
• Use smaller judge models for high-volume routine scoring.
• Reserve stronger judges or human review for ambiguous cases and adjudication.
• Sample production traffic instead of evaluating every interaction.
• Run a small regression suite frequently and the full suite less often.

Evaluation design affects release velocity as well as spend. A large judge suite on every change can turn evaluation into a pipeline bottleneck, so teams often keep frequent checks narrow and run broader evaluation at release boundaries or when an earlier check surfaces a problem.

Evaluation data often includes production questions, retrieved documents and traces, and it often carries the same governance requirements as the source data. Moving those artifacts into a separate evaluation service creates another place to manage access, retention and sensitive information.

With Snowflake, teams can keep evaluation data and traces alongside the data their applications already use, run evaluations against that governed context and analyze the resulting scores with the rest of their operational data. Snowflake’s AI evaluation and observability capabilities build on TruLens, the open source framework for evaluating qualities such as groundedness and relevance.

For applications using Cortex AI, the same environment can support the data, evaluation artifacts and operational analysis around the system. This reduces unnecessary data movement and gives teams a consistent place to examine scores, traces and application behavior as the evaluation moves from pre-release testing into production monitoring.

Reliable LLM evaluation starts with a deceptively simple question: what are you actually trying to measure? The answer determines the rest of the evaluation design.

A benchmark is useful when you need a standardized comparison, while production readiness depends on test cases that reflect real users, real data and the full application pipeline. From there, teams can choose the right scoring method, validate any LLM judge against human labels and carry the same evaluation criteria into production monitoring.

A well-maintained evaluation data set gives teams a consistent reference point as models, prompts and application components change. Representative cases and stable labels make it possible to compare versions against the same definition of acceptable behavior.