Zovda
A pilot exists to answer one question: would a full run of this change my model in the direction I want. Most pilots are designed to answer a different one, which is whether the vendor is pleasant to work with.
Pleasantness is worth knowing. It is not worth six weeks.
Define the decision up front
Write down, before the first row arrives: what result would make you say yes, what would make you say no, and what would make you extend.
If the three answers are the same as last time, this is a renewal, not a pilot.
Test the hard slice, not the easy one
A pilot with clean, well-formed examples will pass. That tells you the pipeline works, which you already knew.
Run the pilot on the cases the current model fails. That is where a data source either has something or does not.
Keep the baseline frozen
Evaluate the pilot model against the same held-out set, the same prompt, the same decoding settings, and the same evaluator as the baseline. Any difference in those four makes the comparison unreadable.
Time-box the setup
If the first usable batch takes longer than the evaluation, the pilot is measuring logistics. That is a legitimate finding, and it should be reported as one.
Write the memo either way
A failed pilot that documents why is worth more than a passed pilot that documents nothing. The failure is usually about a mismatch between the task and the data, and that is reusable knowledge.
Zovda runs pilots through Datuva, where the batch arrives with the same structure as a full delivery. The question bank collection shows the format at small scale.
Tell us what result would change your mind. We reply within one business day.