LLM outputs are probabilistic, open-ended, and often subjective. A language model can return something that's almost right, plausibly wrong, or confidently incorrect — and it's not always obvious which. Unlike traditional software where a function either returns the right value or it doesn't, evaluating LLMs requires deliberate systems. If Cartara flagged this in your diff, you likely added an AI feature without a way to know if it's actually working well.

Why Evaluation Is Non-Negotiable

Skipping evaluation means:

• You don't know if a prompt change made things better or worse
• Model updates silently degrade your product
• You ship regressions you could have caught
• You can't make evidence-based decisions about quality

Evaluation is the discipline that turns "I think this is working" into "I know this is working."

A key mindset shift: evaluation is ongoing, not a one-time gate before launch. Quality drifts as prompts change, models update, and user behavior shifts.

Types of Evaluation

1. Human Evaluation

A person reads the output and judges it. The gold standard — but slow, expensive, and hard to scale.

When to use it:

• Building your initial eval dataset (you need human judgments as ground truth)
• Final review before major launches
• Auditing a sample of production outputs periodically
• Calibrating automated evaluators

Formats:

• Binary: correct / incorrect
• Rating scale: 1–5 quality score
• Comparative: output A vs. output B, which is better?
• Rubric-based: score on multiple dimensions (accuracy, tone, completeness)

Comparative ratings ("which is better?") are more reliable than absolute ratings ("how good is this?") — humans are better at relative judgments.

2. Automated Evaluation

Code-based checks that run without human intervention. Fast, cheap, scalable — but limited to what you can express programmatically.

• Exact match — does the output equal the expected answer? Works for classification, entity extraction, structured outputs.
• Regex / string matching — does the output contain specific patterns? Good for format compliance.
• JSON schema validation — does the output conform to the expected structure? Essential for any structured output use case.
• Semantic similarity — how similar is the output to a reference answer in meaning? Uses embeddings to compare.

3. LLM-as-Judge

Use a (usually stronger) LLM to evaluate the output of another LLM. Scales like automated evaluation but can assess nuanced quality that exact-match metrics cannot.

```python

Example LLM-as-judge prompt

evaluation_prompt = f"""
You are an expert evaluator. Rate the following response on a scale of 1-5
for accuracy, helpfulness, and tone. Return a JSON object with scores and
brief reasoning.

Response to evaluate:
{model_output}

Reference answer:
{expected_output}

Return: {{"accuracy": 1-5, "helpfulness": 1-5, "tone": 1-5, "reasoning": "..."}}
"""

judge_response = client.messages.create(
model="claude-opus-4-5", # Use a strong model as judge
messages=[{"role": "user", "content": evaluation_prompt}]
)
```

Best practices for LLM-as-judge:

• Use the strongest available model as judge
• Provide a detailed rubric — don't just ask "is this good?"
• Ask the judge to reason before scoring
• Validate judge reliability against human ratings on a sample
• Watch for positional bias (judges tend to favor the first option in A/B comparisons)

Key Quality Dimensions

Define 2–4 dimensions most critical for your use case. Trying to measure everything dilutes focus.

Building an Eval Dataset

An eval dataset is a collection of inputs with known-good expected outputs. It's the foundation of reproducible evaluation.

What makes a good eval dataset:

• Representative — covers the real distribution of inputs, including common cases, edge cases, and adversarial inputs
• Challenging — easy cases don't differentiate models or prompts; include hard cases that expose weaknesses
• Diverse — varies across topics, phrasings, user types, and content lengths
• Accurately labeled — human-reviewed expected outputs; garbage labels produce garbage eval results
• Appropriately sized — start with 50–100 examples; grow to 500+ as you mature

How to build it:

1. Mine real inputs from production traffic — real user inputs are more valuable than synthetic ones
2. Write edge cases manually — things the system struggles with, adversarial phrasings
3. Generate synthetically using an LLM, then review and filter
4. Label carefully — have multiple reviewers label ambiguous cases

Treat your eval dataset like code — store it in version control, review changes.

Running Evals

```python
results = []
for example in eval_dataset:
output = call_llm(example.input, prompt=current_prompt)
score = evaluate(output, example.expected)
results.append(score)

Aggregate: pass rate, mean score, failure mode analysis

summary = {
"pass_rate": len([r for r in results if r >= threshold]) / len(results),
"mean_score": sum(results) / len(results),
}
```

When to run evals:

• Before and after every prompt change
• Before and after every model version change
• On a scheduled basis in production (sample and evaluate)
• After any significant change to retrieval or tool configuration

Automate this in your CI/CD pipeline where possible — an eval run that gates deployment is the LLM equivalent of a test suite.

Preventing Regressions

A regression is when a change makes quality worse — a prompt tweak that improves one case breaks another, or a model update subtly degrades performance on your use case.

Never change a prompt without running evals before and after. This is the single most important habit. Even "obviously better" prompt changes can introduce regressions.

Maintain a regression test suite. A subset of your eval dataset focused on cases that have failed in the past. These are your canaries — every known failure mode should have at least one test case.

Pin model versions in production. Model providers update models without notice. Pin to a specific version (e.g., claude-sonnet-4-5 not just claude-sonnet) and evaluate before upgrading.

Log and sample production outputs. Production drift — gradual quality degradation on real-world inputs — is invisible without this.

The regression workflow:
```
Proposed change (prompt / model / config)
↓
Run eval dataset → Baseline scores
↓
Run eval dataset against new version → New scores
↓
Compare: improvements vs. regressions
↓
Net positive → deploy with monitoring
Regressions found → investigate, refine, or reject
```

Evaluation for Agentic Systems

Evaluating agents is harder than evaluating single LLM calls — the output is a sequence of actions, not a single response, and intermediate steps can be wrong even if the final answer is right.

Add these dimensions for agentic systems:

• Trajectory quality — did the agent take sensible steps, or waste steps and loop?
• Tool use accuracy — did it call the right tools with the right arguments?
• Recovery ability — when a tool errors, does the agent adapt appropriately?
• Task completion rate — what % of tasks does the agent complete without human intervention?
• Efficiency — how many steps, tokens, and API calls did it take?

Recommended Tools

For most small teams getting started: Langfuse or PromptFoo are the lowest-friction options.

Related Concepts

• Prompting Best Practices
• Structured Outputs
• RAG — Retrieval-Augmented Generation
• The MCP Ecosystem