Get prompts, ideal answers and human feedback on your model's responses from everyday people in the countries and languages you choose. Set the task, set the pay and approve only what fits.
No subscription · Pay per approved answer · 12,000+ workers online
190+
Countries
13,487
Tasks completed
12,000+
Registered workers
~4 Hours
Avg time to results
Launch a prompt writing or answer evaluation task yourself and start receiving human feedback, without long negotiations.
Paste the prompts or model responses you want people to work with into the task. Explain what to write, compare or rate, and add an example of a good submission.
Prompts or model responsesClear rating rulesExample of a good answer
Select the countries your contributors should come from and state the language they should work in. See how your model performs for real users in each market.
Any number of countries in one taskNative speakers by countryYou set the total volume
Your task becomes available to workers in the selected countries. Each person completes it once, so you get independent opinions from many different people.
Independent opinionsShort reasons for every choiceNew person in every submission
Read every submission, approve the answers and ratings that follow your rules and reject the rest. Keep your dataset clean from the start.
Read every submissionApprove or reject each oneClean data for trainingSet up a task for the human input your language model needs. Contributors follow your instructions and work in their own language.
Contributors write realistic prompts and the answer they would want to get. Use the pairs for supervised fine-tuning and to set a quality bar for your model.
Prompt and answer in one submissionTopics and tone you defineWritten without AI tools
Workers see two responses to the same prompt and choose the better one with a short reason. Collect human preference data for RLHF and model selection.
A versus B choicesShort reason for every pickMany opinions per pair
Contributors rate model answers on a scale you set, such as helpfulness, clarity or tone. Track how your model improves from version to version.
Your own rating scaleSeveral criteria per answerComments on weak points
Workers read model answers on everyday topics and mark mistakes, made-up facts or confusing parts. Spot weak areas before your users do.
Everyday topics and factsMistakes marked and explainedClear yes or no checks
Native speakers in different countries check if answers are correct, natural and polite in their language. Find translation errors and awkward phrasing early.
Native speakers by countryGrammar, tone and wordingLocal cultural context
Contributors write the questions and requests they would type into a chatbot in daily life. Build realistic test sets that match how real users talk.
Everyday requests and questionsNatural wording and typosTopics by country and language
Put your prompt and two model responses into the task. Every worker picks the better one and explains why, so you get preference data with reasons.
Your setup
Choose Text collection, paste the prompt and two model responses into the instructions and explain how to choose the better one.
Text collection task typePrompt and responses in instructionsRules for choosing and explainingYour view
Each submission arrives with the worker's choice and a short reason, ready to review and approve.
Choice and reason in one submissionWorker country on every submissionApprove or reject each answerChoose where your contributors come from and which language they work in. Every submission comes from a different person.
Accept workers from all countries or pick specific ones by continent or individually. Test your model with native speakers in each market.
Describe who should take part in your instructions, such as language level, age group or field of interest. Reject submissions that do not match.
Add a sample of a good comparison or rating, so contributors follow the same logic and your data stays consistent.
Each worker can complete your task only once. Many different people rate the same answers, so you see real agreement and disagreement.
You decide what counts as a good answer or rating and review every submission before it joins your dataset.
Define rating scales, criteria and what a good reason looks like.
Show good and bad examples so contributors know exactly what you expect.
Read each submission and accept only answers and ratings that follow your rules.
Add a few answers with an obvious correct choice to spot careless submissions.
Every answer and rating comes from a real person who chose to take your task and knew what it was for.
Your task description explains that the prompts, answers and ratings will be used to train or evaluate AI.
Nobody is assigned to your task. Each worker decides to take it and submits their work on their own.
Each submission comes from a specific worker in the country you selected, written specifically for your task.
You set the pay per task. Adjust the numbers to see what your project will cost, with the 15% platform fee included.
How long it takes a worker to complete one task.
Minimum pay is $0.10 per task. Need more feedback later? Extend the same task instead of creating a new one.
LLM training data is the text and human feedback used to train and evaluate large language models: prompts, ideal answers, comparisons between responses and ratings. Data from real people helps models answer the way users expect.
It is data where people compare two or more model answers and choose the better one. These choices are used in RLHF and similar methods to teach models which answers people prefer.
Our contributors are everyday users from many countries, not certified experts. They are best for judging clarity, helpfulness, tone and language. You can describe the skills you need in your instructions and reject submissions that do not meet them.
You paste the prompts and responses into your task instructions. Workers read them, then submit their choice, rating or comment as text.
You set the pay for each task, starting from $0.10, plus a 15% platform fee. Writing tasks usually need a higher pay rate than quick comparisons. A campaign starts from $0.30.
Yes. Every worker completes your task once, so the same answers get independent opinions from as many people as you need.
Any language spoken by workers in the countries you select. Choose the countries and state the required language in your instructions.
Ask workers in your instructions to use their own words without AI tools. Review submissions carefully, reject anything that looks generated, and add control questions to spot careless work.
Yes. Extend your existing task instead of creating a new one. Workers who already took part still cannot submit again.
If you need a special format, a review flow of your own or help writing your rating rules, contact us through the form below and we will set it up with you.
Need many languages, high volume or help designing your rating rules? Send a message and we will set it up with you.