Zovda
Screen recordings look like agent data and are not. A recording gives you pixels over time. An agent needs the same pixels plus the decision, the target, and the verification that the action worked.
Without those three, a recording is a video of someone using software.
Pixels plus actions, or nothing
The minimum unit is a step: the screen state, the action taken, the element the action addressed, and the resulting state. Steps chain into a task when they share a goal and a starting state.
Teams often keep the frames and drop the action log. The frames are then unusable for training and unusable for replay, because the click that made the difference is a few pixels wide.
What a trajectory record holds
- Task statement, in the user's words
- Starting screen state, with resolution and scale factor
- Each step: action type, target element, coordinates, and the text typed if any
- Screen state after each step
- The condition that says the task is done
- Where the run failed, if it failed
The scale factor matters more than people expect. The same layout at a different zoom is a different set of coordinates.
Where trajectories break
Three failures show up again and again. The task statement describes the goal after the fact, so it leaks the answer. The done condition is a human judgment rather than a checkable state. The failure rows are deleted, leaving a set where every run succeeded on the first attempt.
That last one is the most expensive, because the model never sees a recovery.
Evaluation without replay
If a trajectory cannot be replayed, it cannot be graded. Store enough to rebuild the screen and the action, and you can score a model's step against a reference step instead of eyeballing a video.
Zovda builds these trajectories through Datuva, where each step keeps its target and its resulting state, so a run can be replayed months later. The video and multimodal collection describes the recording side of the same problem.
Tell us the application and the tasks your agent must handle. We reply within one business day.