Skip to main content
A dataset is a durable snapshot of usable model interactions. Model Distillation stores entries in Postgres for inspection and evaluation, then exports immutable artifacts when training starts.

Create your first dataset

Follow the step-by-step guide from source selection through row inspection.

​
Choose a source

​
Task traffic

Use the current task’s traces when the proxy recorded the traffic. You can filter by:
  • task version;
  • resolved provider and model;
  • scalar trace metadata;
  • fixed start and end timestamps.
Task-source reads use task-attributed Agent Spans and reconstruct the exact provider-shaped Chat Completions request and response.

​
Existing W&B project

Choose an explicit entity and project to import compatible self-logged Weave calls that do not have Model Distillation task identity.

​
Sampling and split

  • Maximum traces caps the random sample after filtering.
  • Validation data reserves a fraction for evaluations; the remaining rows are training data.
  • Split assignment is deterministic and grouped by scenario, agent run, conversation, or trace. Related examples never leak across train and validation.
Use a fixed date range. A moving window would make the same dataset configuration produce different training data later.

​
Inspect and filter rows

The dataset workbench shows messages, original output, relabeled output, comparison-model output, token estimates, split, and per-row evaluation results. Filters support:
  • text in input or original output;
  • train or validation split;
  • relabel status;
  • whether an evaluation was applied.
Delete clearly invalid entries before training. Deleting a row does not alter the source trace.

​
Append new traffic

Use Append with a later fixed range to grow a dataset incrementally. Model Distillation deduplicates entries by trace and turn. Appending increments the dataset revision and makes older relabel or evaluation results stale when their coverage no longer matches.

​
What is excluded

Errored traces, incomplete streams, invalid Chat Completions shapes, and traces whose fidelity mapping is incomplete are not eligible for training.
Agents can discover ready dataset IDs before starting relabeling, training, or evaluation:
To create or append data, follow the copyable request in Datasets Quick Start. See List datasets for this operation.
Last modified on August 25, 2026