Name the change being tested
A different room, a new workplace, and a rearranged table are distinct conditions. State which one the evaluation is intended to measure. If several conditions change at once, acknowledge that the result will combine their effects. A useful sourcing brief describes the deployment uncertainty instead of asking only for footage from more locations.
Control source overlap
Keep related sessions and derived clips together according to your evaluation design. The same room can reappear with a different camera or participant and still provide familiar visual cues. Request location and session references at an appropriate level of detail. These identifiers support grouping without requiring public disclosure of sensitive or unnecessary location information. Keep the meaning of each environment identifier documented for reviewers.
Check task comparability
If one environment contains only assembly and another contains only cleaning, task and environment become entangled. Review whether the intended actions and outcomes are represented across the relevant settings. Sparse combinations may be unavoidable, but they should be explicit. Do not interpret a single aggregate result as evidence about every environment or task independently.
Keep evaluation annotations usable
Use the same definitions across familiar and unfamiliar conditions, while allowing unknown objects or unobservable outcomes where necessary. Review hard examples before finalizing acceptance criteria. A valid challenge should come from the intended scene variation, not from broken media or inconsistent labels. Include the required held-out conditions and annotation policy when discussing feasibility with Fiund.
Review checklist
- Define the environment distinction
- Group related sources consistently
- Review task coverage by setting
- Separate intended difficulty from errors