Skip to main content
Model Distillation adds an improvement loop around an AI feature you already have. Your application keeps serving requests while training and evaluation happen separately in the background.

​
1. Connect an existing feature

Your application calls Model Distillation through the familiar Chat Completions interface. Give the feature a stable task name, and that name remains unchanged as the model behind it improves.

​
2. Learn from real examples

Model Distillation collects the inputs and successful answers your application already produces. You choose which examples represent the behavior you want the new model to learn. If the existing answers are not ideal, you can use a stronger model to create improved training answers before training.

​
3. Train and compare candidates

Train one model or a sweep of different models. Every candidate is tested on the same examples so you can see whether it preserves the quality of the model you use today.

​
4. Deploy without a risky switch

Send a small amount of traffic to the winner first, or deploy it fully after evaluation. The previous model can remain available for a quick rollback.

​
5. Repeat automatically

Automation can wait for enough new examples, prepare the next dataset, train several candidates, compare them, and deploy the best model that clears your threshold.

​
What you control

  • which product behavior becomes a task;
  • which examples and answers the model learns from;
  • which models are trained and compared;
  • how good a candidate must be before deployment;
  • how much traffic the winner receives.

Follow the complete workflow

Connect a task and take it from real usage to a deployed model.
An agent can perform the same loop through these public resources:
  1. PUT /tasks/{alias} and POST /chat/completions to connect traffic;
  2. POST /tasks/{alias}/datasets to freeze training and validation data;
  3. POST /tasks/{alias}/finetunes for each candidate;
  4. POST /evals to compare the candidates;
  5. PUT /tasks/{alias}/versions/{version}/routing to deploy a winner.
Use the Management OpenAPI specification and Inference proxy OpenAPI specification for exact schemas.
Last modified on August 25, 2026