Zovda
Preference collection is easy to run and hard to run well, because the easiest pairs to collect are the ones with an obvious winner. Those pairs carry almost no information about taste.
A model trained mostly on obvious pairs learns the shortcuts that separate a good answer from a bad one. Length. Confidence. Formatting. All three correlate with winning and none of them is quality.
The tie problem
Raters are asked to choose. Many pairs have no clear winner, and forcing a choice converts noise into a label.
Keep the tie as an option, and keep the ties in the set. A model that never sees a tie learns that one answer is always better, which is not what a careful rater believes.
What a pair should differ in
A useful pair differs in one thing: the reasoning, the completeness, the handling of an edge case. A pair that differs in five things teaches a preference you cannot name.
Write the difference down when you collect. If the raters cannot say what made one better, the row is a vibe.
Position and length are confounds
Swapping the order of two answers changes the result more than most teams expect. So does padding one answer with a summary paragraph.
Randomize position, record it, and check the win rate by length before you trust any preference claim.
Keep the ties and the losses
The rows where the model's own answer lost are the most informative, and the first to be dropped when the file gets messy.
Zovda builds these sets through Datuva, where each pair keeps the reason and the position it was shown in. The corpus catalogue shows where pairs sit in the pipeline.
Tell us what "better" means for your product. We reply within one business day.