Skip to main content
A dataset is a durable snapshot of usable model interactions. Model Distillation stores entries in Postgres for inspection and evaluation, then exports immutable artifacts when training starts.

Create your first dataset

Follow the step-by-step guide from source selection through row inspection.

​
Choose a source

The UI builds datasets from the project’s own traffic. The API also accepts an existing W&B project as a source, or a JSONL file that you upload. The following sections describe all three.

​
Project traffic

When the proxy recorded the traffic, use the current project’s traces. In the UI, the Source data section of the Create Dataset dialog lets you choose the following settings:
  • Project version: Pins the prompt and routing regime used to select traces.
  • Served by: Keeps only traces that one model served. To keep training data teacher-only, select the teacher. All models mixes the outputs of every model that served the version.
  • After (UTC) and Before (UTC): A fixed time window. The UI prefills the last 14 days as fixed timestamps.
  • Maximum traces: The random sample size after filtering. The default is 20,000 and the maximum is 100,000.
Through the API, filters also accepts scalar trace metadata. Project-source reads use project-attributed Agent Spans and reconstruct the exact provider-shaped Chat Completions request and response.

​
Existing W&B project

To import compatible self-logged Weave calls that don’t have Model Distillation project identity, set source to an object with entity and project in the API request instead of the string "task". This source isn’t available in the UI dialog.

​
Uploaded file

To bring in examples that were never logged to W&B, upload a JSONL file through the API. Model Distillation validates the rows and creates the dataset for you. See Upload a dataset.

​
Sampling and split

Whichever source you choose, the following settings control how many rows the dataset contains and how those rows are divided between training and test.
  • Maximum traces caps the random sample after filtering.
  • Train / test split reserves test rows for evaluations and model comparisons. The remaining rows are training data. The default is 90% training and 10% test.
Split assignment is deterministic and grouped by scenario, agent run, conversation, or trace. Related examples don’t leak across train and test, and Model Distillation deduplicates inputs automatically.
Use a fixed date range. A moving window would make the same dataset configuration produce different training data later.

​
Inspect and filter rows

The dataset workbench shows messages, original output, relabeled output, comparison-model output, token estimates, split, and per-row evaluation results. Filters support the following criteria:
  • Text in input or original output
  • Train or test split
  • Relabel status
  • Whether an evaluation was applied
Delete invalid entries before training. Deleting a row doesn’t alter the source trace.

​
Append new traffic

To grow a dataset incrementally, use Append with a later fixed range. Model Distillation deduplicates entries by trace and turn. Appending increments the dataset revision and makes older relabel or evaluation results stale when their coverage no longer matches.

​
What is excluded

Errored traces, incomplete streams, invalid Chat Completions shapes, and traces whose fidelity mapping is incomplete aren’t eligible for training.

​
Delete a dataset

You can delete a dataset in the UI or through the API. If fine-tunes or evaluations depend on a dataset, you can delete it together with its dependents, but you can’t delete it by itself. Queued or running fine-tunes, models with active routing, and active evaluations block deletion until they finish or you remove them. An import or relabel run in progress doesn’t block deletion. It stops at its next batch, and Model Distillation discards any results that arrive after deletion begins.
If you delete a dataset together with its dependents, the deletion also destroys the dependent fine-tunes’ training artifacts and any results that use those models.
To delete a dataset in the UI, follow these steps:
  1. Open the dataset’s Settings and select Delete dataset.
  2. If the dataset has dependents, select Also delete the dependent fine-tunes and affected evaluations and review the listed impact.
  3. To confirm, type the dataset name.
  4. If the dataset has dependents, select Delete Dataset and Dependents. Otherwise, select Delete Dataset.
After you confirm, deletion runs in the background. If it fails before destructive work begins, the dataset becomes available again. If it fails afterward, the dataset stays locked until a retry succeeds. To retry, confirm the deletion again. Retries stay within the scope that the original request recorded.
Deletion takes one request, and the server determines the dependents and blockers itself. To also delete the dependent fine-tunes and affected evaluations that exist when the server accepts the request, set cascade=true. Without it, a dataset with dependents returns 409 Conflict with type dataset_in_use. The following request deletes a dataset together with its dependents:
The request returns 202 Accepted with the deletion operation and its status. If you repeat the request, it returns the same operation while that operation is queued, running, or completed. A repeated request also retries failed cleanup within the scope that the original request recorded. When the dataset is already gone and no operation exists, the request returns 204 No Content, and GET on the dataset returns 404 Not Found.To preview the dependents and blockers before deleting, call GET /tasks/{alias}/datasets/{datasetId}/deletion-plan. The preview is informational and also reports the status of a running deletion. For reference details, see Delete a dataset and Preview dataset deletion.
Agents can discover ready dataset IDs before starting relabeling, training, or evaluation:
To create or append data, follow the copyable request in Datasets Quick Start. For reference details about this operation, see List datasets.
Last modified on September 21, 2026