First-Person Video Data for AI: How to Collect Egocentric Video

Robots already work almost flawlessly on factory assembly lines, yet they still struggle to fold a T-shirt. The problem isn't the hardware — it's the data: models lack examples of how people do ordinary household tasks with their own hands. That's why robotics startups now pay people around the world to film themselves doing chores with a head-mounted camera. According to the same report, Nvidia said that adding more than 20,000 hours of first-person video improved robots' success rates on some tasks by more than 50%.

This kind of footage is called egocentric video, or first-person video: the camera sees the world the way a person sees it, with their hands, the objects they handle and the movement of their head. Below, we cover who needs this data, how it differs from regular video, how to write filming scenarios and instructions, how to check quality and what collection costs.

Who Needs First-Person Video

What all these use cases share: the model needs to see not a scene from the outside but what the acting person sees. That's exactly how a robot with cameras in its head, or an assistant in a pair of glasses, will "see."

How Egocentric Video Differs from Regular Video, and Why Public Datasets Aren't Enough

Regular video films a person from the outside. Egocentric video films the world through their eyes, and that changes everything:

1. Hands at the center of the frame. For robotics, this is the key point: the model learns how fingers grab, hold, turn and put down objects.
2. The camera is always moving. It moves with the head or body, so the data captures where the person is looking and where they're going.
3. Continuous actions. Instead of short clips, you get long sequences of steps: take out, wash, chop, put away. The model learns not just the actions but their order.
4. Real homes instead of studios. The value of the data lies in the variety of kitchens, dishes, lighting and clutter.

What's already publicly available

These datasets became the foundation for research, but they're usually not enough for a commercial product. There are three reasons. First, licenses often restrict commercial use — you need to check the terms for each dataset. Second, they don't contain the specific tasks you need: if your robot folds laundry, you need hundreds of hours of laundry folding specifically. Third, the volume falls short: companies collecting this data for robots already talk about hundreds of thousands of hours a month.

We covered the general principles of collecting video for AI in our article on video data collection for AI training. Here, we'll focus on what's specific to first-person filming.

Which Scenarios to Film and How to Build an Action List

The most common mistake is asking people to "film how you spend your day." Half the footage ends up being a walk to the store and a phone in someone's hands, with almost none of the actions you need. Start with what exactly your model should learn to do.

Step 1. Define the target skills

If your robot needs to clean a kitchen, you don't need "household scenes" — you need specific manipulations: pick up a plate from the table, open the dishwasher, put the plate in the rack, wipe the table. Make a list of 10–30 such actions. This will be the backbone of your tasks.

Step 2. Group actions into 5–15-minute scenarios

One video, one complete task with a beginning and an end. This makes it easier for the worker to understand what to film and easier for you to check and annotate the result.

Step 3. Build in variety

A model trained in one kitchen will be lost in another. That's why many different people matter more than many hours from one person. Cap the number of videos per worker and collect data in several countries: stoves, sinks, dishes, food and the very ways people do ordinary things differ a lot between India, Brazil and Poland. For how to split collection across countries, see our guide to AI training data collection.

Pro Tip: include a few "imperfect" situations in your tasks: an object falls, a lid won't open, something has to be redone. A robot in a real home will have to deal with exactly these moments.

Equipment and Recording Requirements

First-person filming doesn't require special equipment. Companies collecting this data for robots often get by with a smartphone on a head strap and a list of tasks.

For most projects, a smartphone on the head or chest is the most practical choice: you don't depend on whether workers own special gear. The key is to pick one mounting option for the entire dataset so the angle stays consistent.

Requirements to include in the task

1. Quality: at least 1080p and 30 frames per second, no filters or effects.
2. Both wrists in frame most of the time: the worker checks this with a short test recording before filming.
3. Lighting: enough light to see fingers and small objects.
4. Continuity: one scenario, one recording, with no cuts or pauses.
5. Natural behavior: do everything at your normal pace, don't slow down or perform for the camera.
6. A short description: the list of actions in order and the country where you filmed.

Note: attach a short example of a good recording to the task. A single 30-second video explains the angle, pace and hand position better than a page of text.

Consent and Privacy: What You Can't Overlook

Video from inside people's homes is sensitive data. Family members, children, documents, screens with private messages and addresses easily end up in the frame. Mistakes here hurt both people and your right to use the dataset.

1. Explicit consent from the worker. The task should state honestly what the recordings will be used for and obtain consent to share the video for AI training.
2. No bystanders in the frame. Ask workers to film when no one else is around. Children in the frame are an automatic reason to reject the video.
3. No personal information. Remove documents, envelopes with addresses, and phone and computer screens from the frame.
4. No audio unless you need it. Background conversations are personal data too. If audio doesn't matter for your model, strip the track.
5. Review before use. Even with good instructions, review videos manually or automatically for faces and text, and blur anything that slips through.

If you collect data in the EU or for a European product, consult a lawyer about GDPR requirements for consent and storage.

Quality Control and Annotation

Without quality control, a noticeable share of recordings will be useless. The founder of one company that collects this kind of video for robotics says that only about half of the submitted footage is usable. The most common reasons videos get rejected:

How egocentric video is annotated

Raw video is useful, but annotated video is far more valuable. The main types of annotation:

1. Action timestamps: the start and end of each step with a label ("picked up the knife," "chopped the onion"). This is how the roughly 90,000 actions in EPIC-KITCHENS-100 were annotated.
2. Short text descriptions of what's happening in the frame, for multimodal models.
3. Hand and object bounding boxes on key frames: which hand is holding which object, and where.
4. Success labels: whether the action was completed on the first try.

Timestamp annotation is often run as a separate campaign with different workers. It doubles as a check: if the annotator can't find the stated actions in the video, the video should be reshot.

What Egocentric Video Collection Costs and How to Run It on RapidWorkers

In this market, workers are reportedly paid roughly $5 to $20 per hour, depending on the country. On RapidWorkers, you don't pay by the hour but per accepted video: recordings that fail review aren't paid. That matters when a significant share of footage doesn't make the cut.

For details on how first-person video collection works on our platform, see the RapidWorkers egocentric video data collection page.

Here's an approximate budget for a first project (the rewards are for illustration — you set them yourself):

A $3 reward for 10 minutes of video plus setup amounts to 15–20 minutes of work for the worker. For high-income countries, raise the reward, or collection will be slow. The easiest approach is to run a separate campaign for each group of countries, each with its own reward.

Here's how to launch:

1. Fund your balance by card or crypto, with a $20 minimum.
2. Create a campaign in the AI Tasks section (beta). Describe the scenario and action list, the recording requirements and the privacy rules, and attach an example of a good video.
3. Select countries. RapidWorkers workers live in 190+ countries, so you can build variety in homes and habits into the dataset from the start.
4. Run a pilot of 20 videos. See what workers misunderstood and refine the instructions.
5. Scale up and review. Accept only videos that meet the requirements. If you end a campaign early, the unused budget returns to your balance.
6. Run annotation as a separate campaign on the accepted videos. We covered how to control annotation accuracy in our article on outsourcing data labeling.

FAQ: First-Person Video for AI

What is egocentric video?

It's video shot from a person's own point of view: the camera is mounted on their head or chest, or built into glasses. The frame shows their hands, the objects they handle and where they're looking. This data is used to train robots, AR assistants and multimodal models.

Why do robots need videos of people?

Robots meant to work in homes need examples of how people handle objects in real environments. Collecting those examples on the robots themselves is slow and expensive, while first-person video can be shot quickly in thousands of different homes. According to Nvidia, adding more than 20,000 hours of such video improved success rates on some robotic tasks by more than 50%.

Can I record egocentric video on a regular phone?

Yes. A smartphone on an inexpensive head or chest mount is the most common option for large-scale collection. What matters is one mount type for the whole dataset, a resolution of at least 1080p and both wrists in the frame.

How many hours of video do I need?

It depends on the task. The research dataset EPIC-KITCHENS-100 has 100 hours of kitchen activity, and Ego4D has 3,670 hours of daily life. To test a hypothesis on a single skill, teams usually start with tens of hours from many different people, while companies building general-purpose robots collect hundreds of thousands of hours.

Can I use Ego4D for a commercial product?

Every public dataset has its own license, and it often restricts commercial use. Before using one in a product, read the license terms and consult a lawyer if needed. Collecting your own data with workers' consent removes this question.

How much does first-person video collection cost?

Workers in this market are paid roughly $5–20 per hour, depending on the country. In the example in this article, 500 accepted 10-minute videos (about 83 hours), plus annotation and the RapidWorkers fee, cost about $2,650.

The Bottom Line: Robots Learn from What People See

First-person video has become one of the most in-demand types of data for robotics and AR. Public datasets give you a starting point, but a specific product needs its own scenarios, its own countries and clean rights to the data. Dataset quality comes down to simple things: a clear action list, a consistent camera angle, hands in the frame, strict privacy rules and a review of every video.

Start with one skill and a pilot of 20 videos — that's about $70. Launch egocentric video collection on RapidWorkers, select your countries and get your first recordings from real homes to test your instructions before you invest in large-scale collection.