
Prepare your test set
Put a numbered set of outputs into the task: text in the instructions, images or audio as reference files.
Text in the task instructions
Images or audio as reference files
Numbered item set
Find out how your model performs for real people. Evaluators in the countries you choose rate, compare and check your model's outputs, from chatbot answers to generated images and synthetic voices.
Text, image and audio 路 Many evaluators per item 路 Any language
200+
Countries
13,487
Tasks completed
195,000+
Registered workers
~4 Hours
Avg time to results
Prepare your outputs, write the questions, and collect independent judgments from many people.

Put a numbered set of outputs into the task: text in the instructions, images or audio as reference files.

Set the criteria, the rating scale and examples of good and bad outputs so evaluators judge the same way.

Each evaluator completes your task once, so every item gets opinions from different people.

Approve careful submissions, reject careless ones and export the answers to compare scores across model versions.
Launch evaluation tasks on the platform, or hand the project to our team for end-to-end delivery.
Self-serve
Create a task, add your outputs and rating rules, choose countries and review results yourself.
Create an evaluation taskFully managed
We design the evaluation, recruit and screen evaluators, run quality checks and deliver clean results in your format.
Talk to usYou set the pay per task. One submission is one evaluator's judgments on your item set. The 15% platform fee is included.
You set the pay for each task, starting from $0.10, plus a 15% platform fee. One submission is one evaluator's judgments on your item set, and a campaign starts from $0.30, so you can run a 20 to 50 item pilot first.
Any model whose output a person can read, see or hear: chatbots and language models, image generators, speech synthesis, speech recognition, translation, search and recommendations.
Side-by-side comparison, rating scales from 1 to 5, mean opinion score for voice and audio, pass or fail checks, error tagging and open feedback. Side-by-side works well for A/B tests between two model versions.
Put a numbered set of text outputs into the task instructions and upload images or audio as reference files. Evaluators review them and submit their scores and comments as their answer.
For clear-cut checks one or two may be enough; for subjective quality like naturalness or preference, use three or more and compare answers. Add control items, ask for short reasons and reject careless submissions.
Everyday users from many countries, best for judging clarity, usefulness, naturalness and language rather than specialist facts. Choose countries where your language is spoken and state it in your instructions; task descriptions are written in English.
Yes. We design the evaluation, recruit and screen evaluators, run quality checks and deliver clean results in your format.