The Concept: AI Evaluating AI

Why use a model to grade a model

For nuanced quality questions — "is this summary faithful?", "is this answer helpful?", "does this response match brand tone?" — there is no regex you can write. You need a grader that understands meaning, context, and pragmatics. That grader is another LLM, prompted to judge.

The pattern is straightforward:

System under test produces an output.
A judge prompt presents the input, the output, and the criteria.
The judge model returns a score (or pass/fail) plus a written rationale.
The score feeds into the same dashboard as deterministic metrics.

Why it works (mostly)

Strong judge models — GPT-5, Claude Opus 4.x, Gemini Pro — are remarkably aligned with human ratings on many evaluation tasks. Multiple academic studies show LLM judges agreeing with humans at roughly the same rate as humans agree with each other (typically 70-85%). For most product evals that is good enough, especially when you cannot afford continuous human annotation.

Principle: An LLM judge is a sampling-aware, prompt-engineered measurement tool. Treat it like an instrument: calibrate it, validate it, and re-check its readings.

Where it fails

Position bias — in pairwise comparison the judge tends to pick whichever output is shown first or last more often than chance.
Self-preference — judges trained on a base model favor outputs from that same base model.
Length bias — judges often reward longer outputs even when shorter would be better.
Cost — every judged sample is an extra API call. At scale this matters.
Brittle prompts — small changes to the rubric flip judgments.

Use a strong judge for a weaker system

Best practice: judge model should be at least as capable as the model under test, ideally stronger. Judging a Claude Opus output with Haiku is asking for trouble — Haiku will miss subtleties that Opus introduced.

Code

Minimum-viable LLM judge·python

JUDGE_PROMPT = """You are an expert evaluator.

User asked: {input}
Assistant answered: {output}

Is the answer correct, complete, and helpful for what the user asked?
Reply ONLY in JSON:
{{\"verdict\": \"pass\" or \"fail\", \"reason\": \"one short sentence\"}}"""

import json

def llm_judge(case, output, judge_model):
    prompt = JUDGE_PROMPT.format(input=case["input"], output=output)
    raw = judge_model.complete(prompt, temperature=0)
    parsed = json.loads(raw)
    return parsed["verdict"] == "pass", parsed["reason"]

When to reach for a judge — a decision tree·python

def pick_grader(eval_case):
    """Cheaper graders first; judge as last resort."""
    if eval_case.has_exact_reference and eval_case.is_short:
        return "exact_match"
    if eval_case.has_reference_phrase:
        return "contains"
    if eval_case.is_format_check:
        return "schema_validate"
    if eval_case.is_classification:
        return "f1_score"
    if eval_case.has_reference_text and eval_case.compare_meaning:
        return "bertscore"
    return "llm_judge"  # everything else

The Concept: AI Evaluating AI

Why use a model to grade a model

Why it works (mostly)

Where it fails

Use a strong judge for a weaker system

Code

External links

Exercise

Progress

Comments 0