# LLM evaluation checklist

> A model that looks good in five hand-picked prompts can fail on the sixth. This checklist turns testing into a repeatable routine you can run after every change.

- Page: https://liftup.tech/checklists/llm-evaluation-checklist/
- Type: Checklist
- Updated: 2026-11-20

## In short

To evaluate an LLM feature, build a golden set of 50 to 200 real examples with agreed correct answers, score outputs on a few named criteria, check unsafe and out-of-scope cases separately, record cost and latency, and re-run the set whenever the prompt, model or data changes. Track results over time, not once.

## Step 1: build the golden set

The golden set is a table of inputs and the answer a careful human would accept. Everything else depends on it, so spend time here.

- Pull real inputs from tickets, emails, search logs or call notes. Redact personal data first.
- Cover the spread: easy, typical and awkward cases. We aim for roughly 60, 30 and 10 percent.
- Add questions the system should refuse or pass to a human.
- Add questions whose answer is not in your documents, to test whether it admits ignorance.
- Have two people write the expected answer for a sample and compare them. Disagreement means the task is vaguer than you thought.

Download the checklist as a file and keep the golden set in a spreadsheet or repository beside it, with a version number.

## Step 2: choose what to measure

| Criterion | How to score it | Typical use |
|---|---|---|
| Correctness | Match against the expected answer; human 0 to 2 | Q&A, extraction |
| Groundedness | Does every claim appear in the supplied source? | RAG chatbots |
| Completeness | Did it include all required fields or points? | Document processing |
| Format | Valid JSON or the required template | Automations feeding other systems |
| Tone and policy | Follows the style guide and refusal rules | Customer-facing replies |
| Refusal accuracy | Refuses what it should, answers what it should | Agents and assistants |

Pick three or four. Ten criteria produce a score nobody can act on. For retrieval systems, test retrieval and generation separately, because a bad answer might be a missing document rather than a bad model. [LLM vs RAG](https://liftup.tech/compare/llm-vs-rag/) explains the split.

For the next step, see [LlamaIndex development](https://liftup.tech/platforms/llamaindex-development/).

## Step 3: use model judges carefully

Using one model to grade another saves time, but it has known biases: it tends to favor longer answers and its own style. Treat it as a screening tool.

1. **Write a rubric**: Plain criteria with examples of a 0, 1 and 2 answer.
2. **Check against humans**: Have a person grade 30 outputs. If the judge agrees with the person less than about four times in five, fix the rubric.
3. **Keep humans on the risky slice**: Anything touching money, health, legal or safety gets a person's review.
4. **Use a different model as judge**: A judge from the same family often favors its own phrasing.

## Step 4: safety, cost and speed

| Check | Method | Pass mark to agree up front |
|---|---|---|
| Out-of-scope requests | 20 prompts asking for things the bot must decline | All declined politely |
| Injection attempts | 10 prompts hidden in documents or user text | No instruction followed |
| Cost per task | Total tokens times price, averaged over the set | Below the value of the task |
| Latency | Median and slowest 5 percent | Fits the channel, for example a few seconds in chat |
| Stability | Run the set twice at your settings | Scores within a few points |

> Our experience: teams pick the model first and discover the cost and latency later. Measure these in week one.

## Step 5: make it routine

Re-run the set whenever you change the prompt, the model version, the retrieval settings or the source documents. A change that improves one criterion often hurts another, and only the set will show it.

Worked example: a support assistant scores 88 percent correct and 97 percent grounded on a 120-item set. After a prompt tweak to shorten answers, correctness holds at 87 percent but groundedness drops to 91 percent. Without the set nobody would have noticed. Keep a simple log: date, change, scores, decision. See our [generative AI development](https://liftup.tech/services/generative-ai-development/) page for how we build this into projects.

## Questions, answered

### How big should the golden set be?

Start with 50 items. Grow toward 200 as real failures arrive. Each production failure becomes a new test case.

### Can we skip evaluation for a small internal tool?

A light version still pays off. Twenty examples and a pass or fail sheet catch most regressions.

### What about public benchmarks?

They tell you about general ability, not about your documents and tone. Use them to shortlist, then test on your own data.

### How often should we re-test?

After every change, and monthly even if nothing changed, because providers update hosted models.

### Do you build evaluation into projects?

Yes. Every AI build we do includes a test set that stays with you. The scope is part of the fixed quote after a short call.

## Related

- [AI vendor evaluation checklist](https://liftup.tech/checklists/ai-vendor-evaluation-checklist/)
- [Generative AI development: features that write, summarize and answer, with quality checks](https://liftup.tech/services/generative-ai-development/)
- [LLM vs RAG: the real difference](https://liftup.tech/compare/llm-vs-rag/)

---

Get a fixed quote: https://liftup.tech/contact/ · email rohit@liftup.tech · WhatsApp +91 77689 10234 (https://wa.me/917768910234) · reply within one working day.
Source page: https://liftup.tech/checklists/llm-evaluation-checklist/ (Liftup Technologies)
