Zovda
Distillation is sold as compression: same behavior, smaller model. What actually transfers is the output distribution, including the parts nobody wanted to keep.
A student model that sounds exactly like its teacher also fails exactly where the teacher fails, and it fails more often.
What transfers
- The format and the register
- The hedging patterns, including when they are inappropriate
- The systematic errors, because they are consistent and therefore learnable
- The refusal style, including over-refusal
Consistency is what makes a teacher useful and what makes its errors reproducible.
What does not transfer
The teacher's ability to notice it does not know. A model that guesses rarely knows it is guessing, and the student learns the guess without the doubt.
Filter before you distill
Take the teacher's outputs, run them against a check where one exists, and drop the ones that fail. A distilled set built from verified outputs is a different product from one built from all outputs.
Add the corrections
Where the teacher was wrong and someone fixed it, keep both. The pair teaches the student something the teacher never had.
Zovda builds these collections through Datuva, where each row records which model produced it. The corpus catalogue shows where generated rows sit in each stage.
Tell us what you are compressing. We reply within one business day.