Skip to main content
Model Distillation adds an improvement loop around an AI feature you already have. Your application keeps serving requests while training and evaluation happen separately in the background.

​
1. Connect an existing feature

Your application calls Model Distillation through the familiar Chat Completions interface. Give the feature a stable proxy model name, and that name remains unchanged as the model behind it improves.

​
2. Learn from real examples

Model Distillation collects the inputs and successful answers your application already produces. You choose which examples represent the behavior you want the new model to learn. If the existing answers are not ideal, you can use a stronger model to create improved training answers before training.

​
3. Train and compare candidates

Train one model or a sweep of different models. Every candidate is tested on the same examples so you can see whether it preserves the quality of the model you use today.

​
4. Deploy without a risky switch

Send a small amount of traffic to the winner first, or deploy it fully after evaluation. The previous model can remain available for a quick rollback.

​
What you control

  • which product behavior becomes a project;
  • which examples and answers the model learns from;
  • which models are trained and compared;
  • how good a candidate must be before you deploy it;
  • how much traffic the winner receives.

Follow the complete workflow

Connect a project and take it from real usage to a deployed model.
An agent can perform the same loop through these public resources:
  1. PUT /tasks/{alias} and POST /chat/completions to connect traffic;
  2. POST /tasks/{alias}/datasets to freeze training and validation data;
  3. POST /tasks/{alias}/finetunes for each candidate;
  4. POST /evals to compare the candidates;
  5. PUT /tasks/{alias}/versions/{version}/routing to deploy a winner.
Use the Management OpenAPI specification and Inference proxy OpenAPI specification for exact schemas.
Last modified on September 2, 2026