Zovda
A team ships a product in ten languages and trains on one. The corpus is English, converted nine times. It scores well, and users in nine markets still find the assistant slightly off.
That gap is not a vocabulary gap. It is a difference in how much is said, how directly, and what a polite answer looks like.
Parallel text measures one thing
Aligned sentences across languages are useful for matching tasks. They are a poor proxy for natural usage, because the aligned version inherits the source sentence's rhythm.
A corpus that only contains converted text teaches a model to speak every language like the one it was written in.
Native rows are the expensive half
Native rows cost more to collect and carry the signal nobody can generate: idiom, abbreviation, regional spelling, and the length people actually write.
A practical target is a native share per language large enough to hold the register, with converted rows used for coverage rather than style.
Record the register, not just the language
Formal and informal are different corpora inside one language. A support answer and a social post in the same file teach the model to blur them.
We keep a register label per row and a language tag that distinguishes regional variants, so a mix can be built rather than guessed.
The checklist we run
- What share of each language is native rather than converted?
- Does every row carry a register label?
- Are regional variants separated instead of pooled?
- Can a language be evaluated on its own, not only in aggregate?
- Is length distribution per language recorded?
If a supplier cannot answer these, ten languages is one language with ten spellings. Zovda keeps native and converted shares separate in the Datuva catalogue so buyers can see the difference before they train.
Where to start
Start from the markets that matter and the register your product uses there. The corpus catalogue shows how language families sit across stages.
Tell us the languages, registers and volume. We reply within one business day.