阅读中文版

Your Code Corpus Is Mostly Duplicate Imports

A decision tree with one successful path

Most code datasets are counted in repositories. The unit that matters is an example a model can be graded on: one request, one expected behavior, one way to check it. Repositories are an input format. Examples are the product.

The gap between the two is where most of the work hides.

A sample you cannot run teaches imitation

If a sample cannot be executed, it teaches style. That is still worth something, and it is not what you want when the model will be trusted with a patch.

What makes a sample executable is not the language. It is the context: imports that resolve, a fixture or a small input, and an assertion that fails before the fix and passes after it.

What a usable code record holds

  • The task, written as a request rather than a description of the answer
  • The starting state of the files that matter
  • The patch, as a diff rather than a rewritten file
  • The check that decides whether the result is right
  • The runtime the check was run on

The last field is the one people skip. A test that passes on one interpreter and fails on another is a different example.

Patching is not the same task as writing

Writing a function on a blank page and editing a module inside a large repository are two skills, and they need two distributions. Mix them and the model mostly sees greenfield code, then gets judged on edits where the hard part is not the logic but the surrounding conventions.

Keep the diff small enough that a reviewer can follow it. Keep the file context large enough that the diff is possible.

The dedup pass nobody runs

Two files that differ by one import line are the same example. So are four copies of a utility function pasted into four repositories. Exact-match dedup catches almost none of that.

What catches it is a signature over normalized tokens plus a similarity pass over the surrounding context. It is cheap, and it changes the effective size of the set by more than most teams expect.

Where to start

The code and agent collection shows how these records are structured, and the corpus catalogue shows which stage each one serves.

Zovda builds these collections through Datuva, where a task, its starting state, and its check stay in one row, so a sample re-run next year still means the same thing.

Tell us the language, the task shape, and the volume you need. We reply within one business day.

Show us what your model has to patch

Related insights

Trajectories that survive the second step

Tell us the environment and the task, not just the tool list. We reply within 1 business day.

Start a trialSee the corpus catalogue

Request a trial

Tell us what you need. We reply within 1 business day.