LLM vs RAG vs Agent Evaluation: Why One Framework Doesn't Fit All

The Evaluation Gap

Most teams start with an LLM, add retrieval, then bolt on tool use — but keep using the same "vibe check" evaluation throughout. This is a category error. Each paradigm shift introduces fundamentally new failure modes that the previous evaluation layer cannot catch.

%%{init: {'theme': 'dark', 'themeVariables': { 'darkMode': true }}}%% graph TD classDef rootNode fill:#2d3748,stroke:#4a5568,stroke-width:2px,color:#e2e8f0; classDef llmNode fill:#2b6cb0,stroke:#2c5282,stroke-width:2px,color:#ebf8ff; classDef ragNode fill:#2f855a,stroke:#276749,stroke-width:2px,color:#f0fff4; classDef agentNode fill:#c05621,stroke:#9c4221,stroke-width:2px,color:#fffff0; classDef leafNode fill:#1a202c,stroke:#4a5568,stroke-width:1px,color:#a0aec0; Eval[AI System Evaluation]:::rootNode Eval --> LLM[LLM Evaluation]:::llmNode Eval --> RAG[RAG Evaluation]:::ragNode Eval --> Agent[Agent Evaluation]:::agentNode %% LLM Dimensions LLM --> L1(Correctness):::leafNode LLM --> L2(Coherence):::leafNode LLM --> L3(Safety):::leafNode LLM --> L4(Hallucination Rate):::leafNode %% RAG Dimensions RAG --> R1(Retrieval Precision/Recall):::leafNode RAG --> R2(Answer Faithfulness):::leafNode RAG --> R3(Citation Accuracy):::leafNode RAG --> R4(Latency Budget):::leafNode %% Agent Dimensions Agent --> A1(Tool Use Accuracy):::leafNode Agent --> A2(Planning Quality):::leafNode Agent --> A3(Task Completion):::leafNode Agent --> A4(Multi-step Reasoning):::leafNode Agent --> A5(Error Recovery):::leafNode

Level 1: LLM Evaluation — The Foundation

Raw LLM evaluation is about generation quality in isolation. No tools, no retrieval, no multi-step plans. The input is a prompt; the output is text.

Core Dimensions

Correctness: Factual accuracy against ground truth. Use exact match, F1, or semantic similarity for open-ended tasks.
Coherence: Logical flow, consistency within the response, adherence to style/tone constraints.
Safety: Refusal rates for harmful requests, PII leakage, bias detection.
Hallucination Rate: Fabricated facts, invented citations, confident falsehoods. Measure with attribution verification.

Practical Approach

Build a golden dataset of 200-500 prompt-response pairs covering your task distribution. Run on every model swap, prompt change, or temperature adjustment. Use LLM-as-judge (calibrated against human labels) for semantic dimensions; exact match for structured outputs.

Level 2: RAG Evaluation — Retrieval Meets Generation

RAG adds a retrieval subsystem whose quality directly bounds generation quality. You now have two coupled systems to evaluate.

New Failure Modes

Retrieval failure: Relevant docs not in top-k (recall) or irrelevant docs polluting context (precision)
Faithfulness violation: Model ignores retrieved context and hallucinates, or contradicts retrieved facts
Citation mismatch: Claims don't align with cited sources, or citations point to wrong passages
Latency regression: Retrieval + reranking + generation exceeds budget

Evaluation Stack

Retrieval metrics: Precision@k, Recall@k, MRR, nDCG — against labeled query-doc pairs
Faithfulness: LLM-as-judge: "Does the answer contradict any retrieved document?" (binary or Likert)
Answer relevance: "Does the answer address the user's question given the retrieved context?"
Citation quality: Automated check: do cited doc IDs actually contain the claimed information?

Golden Dataset Design

Your RAG eval set needs: (query, relevant_doc_ids, expected_answer, expected_citations). Minimum 100 examples spanning query types: fact lookup, comparison, aggregation, multi-hop reasoning.

Level 3: Agent Evaluation — Autonomy Introduces State

Agents add planning, tool use, and multi-step execution. The output is no longer just text — it's a trajectory of observations, actions, and intermediate states.

New Failure Modes (That LLM/RAG Eval Misses)

Tool selection errors: Calling the wrong function, wrong parameters, or hallucinating functions
Planning failures: Infinite loops, missing steps, wrong order, unable to decompose complex goals
State corruption: Losing context across steps, contradictory actions, variable scope bugs
Error recovery: Getting stuck on tool failures instead of retrying, falling back, or asking for help
Task completion: Reaching a terminal state that satisfies the user goal (not just "stopped")

Evaluation Methodology

Trajectory comparison: Compare agent's action sequence against a reference trajectory (exact match, edit distance, semantic equivalence)
Tool use accuracy: Per-step: correct tool? correct args? valid output parsing?
Goal achievement: Binary or graded: did the final state satisfy the user intent? Use an LLM judge with access to the full trajectory.
Efficiency metrics: Steps to completion, token cost, wall-clock time, human interventions required
Robustness testing: Inject tool failures, network timeouts, ambiguous instructions — measure recovery rate

Unified Evaluation Architecture

Don't build three separate eval systems. Build one framework with pluggable evaluators:

class EvaluationSuite:
    def __init__(self):
        self.evaluators = {
            'llm': [CorrectnessEvaluator(), CoherenceEvaluator(), SafetyEvaluator()],
            'rag': [RetrievalEvaluator(), FaithfulnessEvaluator(), CitationEvaluator()],
            'agent': [TrajectoryEvaluator(), ToolUseEvaluator(), GoalAchievementEvaluator()]
        }

    def evaluate(self, system_type: str, inputs, outputs, metadata):
        results = {}
        for evaluator in self.evaluators[system_type]:
            results[evaluator.name] = evaluator(inputs, outputs, metadata)
        return aggregate(results)

Tooling Landscape

Category	Tools	Best For
LLM Eval	Promptfoo, LangSmith, DeepEval, Ragas	Prompt regression, model comparison
RAG Eval	Ragas, LlamaIndex Eval, TruLens	Retrieval + generation quality
Agent Eval	AgentBench, ToolBench, LangSmith, custom trajectory diff	Multi-step reasoning, tool use
Unified	LangSmith, Weights & Biases, MLflow	Experiment tracking across all three

Key Takeaways

Match eval to system complexity. Don't use agent eval for a RAG pipeline; don't use LLM eval for an agent.
Invest in golden datasets early. They're the highest-leverage asset you'll build.
Calibrate LLM judges. Uncalibrated judges are just expensive vibe checks.
Run eval in CI. If it's not blocking merges, it's documentation.
Monitor production drift. Eval sets rot; production data tells you when.

The teams shipping reliable AI systems aren't the ones with the best models — they're the ones with the best evaluation discipline.