Define the outcome explicitly
A task can be executed as instructed yet fail because an object slips or a tool does not work. Another attempt can end successfully after a wrong intermediate step. Decide whether the dataset labels final outcome, procedural correctness, or both. Without this distinction, success labels can obscure the behavior your team is trying to study.
Retain the evidence of failure
A clip ending at contact may omit the dropped object or incomplete closure that establishes failure. Request enough follow-through for a reviewer to judge the outcome. Describe whether failures may be naturally occurring or deliberately staged. Keep those sources distinguishable, since a person pretending to make an error can behave differently from someone encountering an unexpected problem.
Document recovery and repeated attempts
When a participant retries, decide whether each attempt becomes its own unit or remains part of one longer task. Preserve links between attempts and identify any reset of objects or instructions. A recovery sequence can contain useful decisions that disappear if the recording is reduced to isolated successful actions. Specify whether pauses and assistance should remain visible.
Avoid inventing certainty
Some outcomes cannot be confirmed from the participant’s view. Allow unknown or unverified outcomes instead of forcing a binary label. If another record supplies the result, identify that source separately from visual evidence. Review the distribution of outcomes by task and environment so failures do not become a proxy for one location, participant, or capture configuration.
Review checklist
- Separate outcome from procedure
- Show enough follow-through
- Link retries and resets
- Allow unverified outcomes