Human Evaluation of AI Models

You changed the system prompt, fine-tuned the model or switched to a different provider. The automated tests passed and the metrics went up by a couple of percent. But are the answers actually better for your users? Metrics are bad at answering that question.

A text task rarely has a single correct answer. Classic metrics like BLEU and ROUGE measure word overlap with a reference answer. They can't tell when a response became clearer or more polite — or when it confidently made something up. Public benchmarks test a model's general abilities, not how it handles your product, your customers and your language.

That's why teams building LLM products sooner or later turn to human evaluation (human eval). Real people read the model's responses and rate them against clear criteria: accuracy, helpfulness, tone, safety. Well-known model leaderboards like LMArena (formerly Chatbot Arena) are built on pairwise human comparisons too.

In this guide, we'll cover when human evaluation is really needed, which methods exist, how to write instructions and calculate sample size, how to check the raters themselves, how to analyze the results and what it all costs.

When You Need Human Evaluation and When an LLM Judge Is Enough

Today, part of the evaluation is done by the models themselves: one LLM reads another's response and scores it. This approach is called LLM-as-a-judge. In a well-known study by the authors of MT-Bench and Chatbot Arena, GPT-4's judgments matched human preferences in more than 80% of cases — about the same level at which humans agree with each other. But the same authors also described systematic biases in the judge model: it favors the answer shown first, prefers longer answers and may rate its own outputs higher.

In practice, the two approaches don't compete — they complement each other. An LLM judge is cheap and fast and works well for daily runs. Humans are needed where the cost of a mistake is high, and to verify that the judge itself isn't wrong.

Don't skip humans in these cases:

1. Before a release. The decision "do we ship the new version or not" is better made on judgments from people who resemble your users.
2. New languages and markets. Judge models are weaker in low-resource languages and have a poor sense of local politeness norms. A native speaker will judge how natural an answer in Hindi or Swahili sounds more reliably.
3. Validating the LLM judge itself. Before trusting automated scoring, compare it with human judgments on a few hundred examples. If agreement is low, the judge needs tuning.
4. Subjective criteria. Tone, empathy, "does it sound like a real person," how much the answer helped solve the problem.
5. Safety and sensitive topics. Here the cost of a missed error is too high to rely on automation alone.

Evaluation Methods: Pairwise Comparison, Rating Scales and Rubrics

Human evaluation comes in a few standard formats. The format you choose determines how stable your results will be and how many judgments you'll need.

Pairwise comparison (A/B)

The rater sees one prompt and two responses and picks the better one, or marks a tie. This is the most reliable format for the question "is the new version better than the old one": it's easier for people to compare two options than to give an absolute score. The result is calculated as a win rate. Model leaderboards based on Elo or Bradley–Terry ratings are built on comparisons like these.

Two rules are mandatory: randomize the order of responses (people also tend to pick the option on the left or on top more often), and don't reveal which model wrote which response.

Absolute rating scale

The rater gives a single response a score on a scale, for example from 1 to 5. This works for tracking quality over time or finding weak spots. The main problem is that everyone has their own idea of a "4." One rater is strict, another gives almost everything a 5. Anchored levels help: for each score, describe what it means and give an example.

Rubric or checklist

Instead of one overall score, the rater answers several specific yes/no questions: was the instruction followed, are there factual errors, is the format correct, is there unnecessary filler. Specific questions produce noticeably more consistent answers than "rate the quality from 1 to 5," and they immediately show what exactly went wrong.

Ranking

The rater sees three to five responses and orders them from best to worst. This is how preference training data is collected. But with every added response, the task gets longer and harder, and attention drops.

A combination that works for most teams: pairwise comparison for the final decision, plus a short checklist of two or three questions to understand why one response won.

How to Design an Evaluation: Criteria, Instructions and Sample Size

A good evaluation doesn't start with finding raters. It starts with four decisions: what exactly we're measuring, on which prompts, how to explain it to the rater, and how many judgments we need for the result to be trustworthy.

1. Pick 2–4 criteria

A typical set for an assistant: factual accuracy, helpfulness (did the answer solve the task), instruction following and tone. For customer support, add "the model doesn't promise things the company doesn't do." For translation, add "the text sounds natural to a native speaker." A rater can't hold ten criteria in a single task: attention gets scattered, and all the scores drift toward the middle.

2. Build a prompt set

Prompts should resemble real ones: take them from logs (after removing personal data) or ask people from your target audience to write their own. Add hard cases: ambiguous questions, long instructions, requests the model should politely decline. If the set contains only easy questions, both versions will handle them equally well, and you won't see any difference.

3. Write instructions with examples

The rater doesn't know your product and can't ask you questions. Your instructions should include:

1. two sentences of context: who the user is and what they need;
2. a plain-language definition of each criterion;
3. 2–3 worked examples: which response is better and why;
4. a rule for borderline cases: when to mark a tie, and whether accuracy or completeness matters more;
5. a request to explain the choice in one sentence. This keeps raters disciplined and gives you valuable comments.

4. Calculate how many comparisons you need

Sample size depends on how small a difference you want to detect. For pairwise comparison, a simple rule of thumb for proportions works: at 95% confidence, the margin of error of the win rate is roughly 1/√n. In numbers:

In practice, 300–500 comparisons are usually enough for a "should we ship the new version" decision. So that one careless judgment doesn't decide the outcome, give each pair to three different raters and use the majority vote. This also shows you how much raters agree with each other.

A Sample Rubric You Can Adapt

Here's what a compact rubric might look like for a customer support assistant. The pairwise question comes first, followed by three checklist questions for each response.

Main question: "Imagine you are the customer who sent this message. Which response would help you more? Choose A, B or Tie."

Checklist for each response:

Tie rule: choose Tie only if both responses are equally useful or equally bad. If one response is slightly better, choose it.

Explanation: one sentence on what decided your choice.

Give raters the reference notes (policies, prices, product facts) inside the task. Without them, they'll judge accuracy by guesswork, and you'll measure how confident the response sounds rather than whether it's correct.

Who Should Evaluate, and How to Check the Raters

Who you pick as raters depends on what knowledge it takes to tell a good response from a bad one.

Most product questions — "is the answer clear," "did it help," "which of the two is nicer to read" — don't require a degree. They're best asked of ordinary people who resemble your users. Keep experts for fact-checking in specialized fields.

How to filter out careless raters

1. Control pairs. Mix 1–2 pairs into every batch where one response is obviously better because the other is off-topic or has a glaring error. Anyone who gets these wrong isn't reading.
2. Time on task. If reading two responses takes at least a minute and someone spends 10 seconds, their judgments can't be trusted.
3. A comment on the choice. A one-sentence "why" quickly shows whether the person read the responses. Identical or meaningless comments are a reason to reject the work.
4. Overlap. Have several people rate some of the pairs. A rater who consistently disagrees with the majority most likely misunderstood the instructions.

Inter-rater agreement

If even humans can't agree on which response is better, the problem is usually in the instructions, not the raters. Agreement is measured with coefficients that correct for chance: Cohen's kappa for two raters and Krippendorff's alpha for any number. A value of 1 means perfect agreement; 0 means agreement no better than chance. Krippendorff himself recommended relying on data with an alpha of 0.8 or higher and treating values from 0.667 as acceptable only for tentative conclusions. In subjective tasks like rating tone, that level is harder to reach. More judgments per pair and more specific criteria help.

A practical sequence: rate the first 50 pairs yourself and give the same pairs to a pilot group of raters. Where the answers diverge, clarify the instructions, and only then launch the main batch. For more on accuracy control when working with external contributors, see our article on outsourcing data labeling.

Common Mistakes That Make Human Evals Useless

Most failed evaluations don't fail because of bad raters. They fail because of the setup. Here are the mistakes we see most often:

1. Showing model names or versions. Raters who see "GPT" or "new version" form an opinion before reading. Always label responses neutrally as A and B.
2. No tie option. Forcing a choice between two equally good responses adds noise to your data. Allow ties, but define when to use them.
3. Too many criteria in one task. Ten sliders per response guarantee average scores everywhere. Split the work into separate tasks or cut criteria down to the ones that drive your decision.
4. Only easy prompts. If 90% of your prompt set is simple, any two decent models will tie, and you'll conclude there's no difference when there is one on hard cases.
5. One rater per item. A single judgment can't be checked. With three, you get a majority vote and an agreement score for free.
6. Skipping the pilot. Unclear instructions surface in the first 20 tasks. Finding that out after 1,000 tasks means paying for data you can't use.
7. The wrong audience. Testing a product for Brazilian users with raters from other countries tells you little about how it will be received in Brazil.
8. Ignoring the comments. The one-sentence explanations are often more valuable than the scores: they tell you what to fix in the next version.

How to Analyze the Results

Once the judgments are in, the analysis takes an hour, not a week. Here's a simple sequence.

1. Aggregate each pair. With three raters per pair, take the majority vote. If all three disagree (A, B and Tie), count the pair as a tie.

2. Calculate the win rate. The common convention is to count a tie as half a win for each side. For example, out of 400 pairs, version B won 212, version A won 148, and 40 were ties. B's win rate is (212 + 20) / 400 = 58%.

3. Check the margin of error. With 400 comparisons, the margin is about ±5 percentage points, so B's true win rate is roughly between 53% and 63%. The whole range is above 50%, so B is genuinely better. If the range crossed 50%, you would need more data before deciding.

4. Break the results down by category. An overall win can hide a loss in an important segment. Split the prompts by type (billing questions, technical issues, complaints) or by language, and check the win rate in each group.

5. Read the comments on the losses. Sort the pairs where the new version lost and read the raters' explanations. Patterns appear quickly: "too long," "didn't answer the second question," "made up a price."

6. Check agreement. If agreement in some category is much lower than elsewhere, the criteria for that category are probably unclear, and its results deserve less weight.

7. Compare with your LLM judge. Run the same pairs through your automated judge and see how often it agrees with the human majority. This tells you how much you can rely on it between human evaluation rounds.

How to Run an Evaluation on RapidWorkers and What It Costs

You don't need a dedicated in-house team or an agency contract to run human evaluation. You can show response pairs in any tool you like — Google Forms, Typeform, Label Studio or your own form — and recruit raters through a RapidWorkers campaign. The worker opens the link, rates a batch of pairs and submits a completion code as proof. You match the code against the answers in your form and accept or reject the work.

Here's how to launch:

1. Package pairs into tasks. A convenient size is 10 pairs per task, including 1–2 control pairs. That's 10–15 minutes of work: long enough for the worker to get into the instructions, short enough that they don't get tired.
2. Create a campaign. Paste the instructions with examples and the link to your form, and specify what to submit as proof.
3. Select countries. If your product is in Spanish for Mexico, the raters should be people living in Mexico. For more on country targeting, see our guide to AI training data collection.
4. Run a pilot of 10 tasks. Check the control pairs, completion time and comments, and refine the instructions if needed.
5. Scale up and review the work. Rejected work isn't paid, and if you end a campaign early, the unused budget returns to your balance.

Here's the math for a typical comparison of two model versions (the reward is for illustration — you set it yourself):

A pilot of 10 tasks at the same reward costs $9.20. For that money, you get an answer that no automated metric can give you: which version real people prefer, and why.

Keep the results together with the raters' comments. Human-judged pairs pay off twice: as a reference set for validating your LLM judge, and as a test set for future regression checks.

FAQ: Human Evaluation of LLMs

What is human evaluation in machine learning?

It's the assessment of a model's outputs by real people against predefined criteria: accuracy, helpfulness, tone, safety. For language models, the most common formats are pairwise comparison of two responses and scoring on a scale or checklist.

Can an LLM judge fully replace human raters?

For routine checks, often yes — but only after you've compared the judge's scores with human judgments on your own data. Judge models are sensitive to response order and length and are weaker in low-resource languages. Release decisions and safety reviews are better confirmed by humans.

How many raters and comparisons do I need?

To decide which of two versions is better, 300–500 pairs with three judgments each is usually enough. 400 comparisons give a win rate margin of error of about ±5 percentage points. To detect a very small difference, you'll need around a thousand.

Do I need experts to evaluate model responses?

Only if the evaluation requires specialized knowledge: medicine, law, complex code. Clarity, helpfulness and tone are better judged by ordinary people who resemble your users, since they're the ones who will read these responses.

How much does human evaluation cost?

It depends on complexity and volume. In the example in this article, comparing two model versions on 400 pairs with three raters per pair costs about $110 on RapidWorkers, including the fee. Expert evaluation in specialized fields costs significantly more.

How do I evaluate a model in a language other than English?

Find raters who are native speakers from the country where the product will be used. Translate the instructions and examples, add a "natural language" criterion and restrict the campaign to the relevant countries.

How long does a human evaluation take?

Preparing the prompt set, instructions and form usually takes a day or two. A pilot of 10 tasks typically completes within hours, and the main batch depends on volume and how narrowly you target countries. Analysis takes about an hour once the data is in.

How is human evaluation different from RLHF?

Human evaluation measures how good a model already is. RLHF (reinforcement learning from human feedback) uses human preferences to train the model further. The raw material can look similar — people compare or rank responses — but the goal is different: a verdict on quality versus training data.

The Bottom Line: Humans Belong Where Releases Are Decided

Automated metrics and LLM judges give you speed, but they don't answer the main question: did the product actually get better for real people? Reliable human evaluation is built from simple parts: pairwise comparison instead of abstract scores, 2–4 clear criteria, instructions with examples, control pairs and three judgments per item. You calculate the sample size in advance, and the evaluation itself costs tens of dollars, not thousands.

Start small: take 100 real prompts, collect responses from both versions and run a pilot. Create a campaign on RapidWorkers, select your users' countries, and the first judgments will already show you where your model is heading.