Skip to main content
Model Distillation is designed for teams that already have an LLM feature in production and want a specialized model that can deliver similar or better behavior at lower cost and latency.

​
What Model Distillation provides

  • Connect your application once: Keep the familiar Chat Completions interface and switch between models without changing how your application sends requests or reads responses.
  • Learn from real usage: Automatically collect the examples your application already produces, so you can improve a model using behavior that matters to your customers.
  • Build better training data: Choose the examples you want a model to learn from, review them, remove poor examples, and improve their answers when needed.
  • Train models for your task: Create specialized models that can match the behavior of a much larger model while reducing cost and latency.
  • Compare quality before deployment: Test models on the same examples, see where each one wins or fails, and make deployment decisions with evidence.
  • Deploy safely: Test a new model with part of your traffic, increase its share gradually, and keep the previous model ready for rollback.
  • Automate continuous improvement: Let Model Distillation prepare data, train several candidates, compare them, and deploy the best model that meets your quality goal.

​
A typical customer journey

  1. Connect one AI feature to Model Distillation.
  2. Collect examples from normal product usage.
  3. Choose the examples and answers worth learning from.
  4. Train several candidate models.
  5. Compare them with the model you use today.
  6. Deploy the winner gradually and monitor the result.
Your current model remains available throughout the process, so there is no risky all-at-once migration.
Send the agent to the Quick Start for the full sequence. Studio actions use the public Management API, while application traffic uses the Inference proxy API. Each actionable guide includes the equivalent request under API.
Last modified on August 25, 2026