Human Evaluation

Human Evaluation Platform for AI Models

Find out how your model performs for real people. Evaluators in the countries you choose rate, compare and check your model's outputs, from chatbot answers to generated images and synthetic voices.

Text, image and audio 路 Many evaluators per item 路 Any language

Person comparing two model answers side by side

200+

Countries

13,487

Tasks completed

195,000+

Registered workers

~4 Hours

Avg time to results

How It Works

From Test Set to Human Scores

Prepare your outputs, write the questions, and collect independent judgments from many people.

Person preparing AI model outputs for a human evaluation task
01 路 Prepare

Prepare your test set

Put a numbered set of outputs into the task: text in the instructions, images or audio as reference files.

  • Text in the task instructions
  • Images or audio as reference files
  • Numbered item set
Laptop showing evaluation criteria and rating rules
02 路 Rules

Write evaluation rules

Set the criteria, the rating scale and examples of good and bad outputs so evaluators judge the same way.

  • Criteria you define
  • Rating scale or comparison
  • Examples of good and bad outputs
Person comparing two model answers side by side on a screen
03 路 Collect

Collect independent judgments

Each evaluator completes your task once, so every item gets opinions from different people.

  • One submission per evaluator
  • Many opinions per item
  • Countries and languages you choose
Person reviewing human evaluation submissions on a computer
04 路 Review

Review and analyze

Approve careful submissions, reject careless ones and export the answers to compare scores across model versions.

  • Approve or reject each submission
  • Export answers for analysis
  • Compare scores across versions
How We Work

Run It Yourself or Let Us Manage It

Launch evaluation tasks on the platform, or hand the project to our team for end-to-end delivery.

Self-serve

Launch evaluation tasks yourself

Create a task, add your outputs and rating rules, choose countries and review results yourself.

Create an evaluation task

Fully managed

Let our team run the evaluation

We design the evaluation, recruit and screen evaluators, run quality checks and deliver clean results in your format.

Talk to us
Pricing

Estimate Your Evaluation Cost

You set the pay per task. One submission is one evaluator's judgments on your item set. The 15% platform fee is included.

?
?
min
?
$
FAQ

Frequently Asked Questions

You set the pay for each task, starting from $0.10, plus a 15% platform fee. One submission is one evaluator's judgments on your item set, and a campaign starts from $0.30, so you can run a 20 to 50 item pilot first.

Any model whose output a person can read, see or hear: chatbots and language models, image generators, speech synthesis, speech recognition, translation, search and recommendations.

Side-by-side comparison, rating scales from 1 to 5, mean opinion score for voice and audio, pass or fail checks, error tagging and open feedback. Side-by-side works well for A/B tests between two model versions.

Put a numbered set of text outputs into the task instructions and upload images or audio as reference files. Evaluators review them and submit their scores and comments as their answer.

For clear-cut checks one or two may be enough; for subjective quality like naturalness or preference, use three or more and compare answers. Add control items, ask for short reasons and reject careless submissions.

Everyday users from many countries, best for judging clarity, usefulness, naturalness and language rather than specialist facts. Choose countries where your language is spoken and state it in your instructions; task descriptions are written in English.

Yes. We design the evaluation, recruit and screen evaluators, run quality checks and deliver clean results in your format.