Quick start - CoreWeave Docs

This guide shows the complete path from an existing AI feature to a specialized model running in production. It’s for developers who already have an application that calls a model through an OpenAI client and want to replace that model without changing the application each time. You connect your application to Model Distillation, collect real traffic into a dataset, fine-tune and evaluate candidate models, and then route production traffic to the winner. By the end, your application calls a stable proxy model name, and a candidate you chose for its balance of quality, speed, and cost serves the share of traffic you assign to it. Open Model Distillation Studio, sign in with your W&B API key, and choose the team that owns the application.

We recommend a team service-account key for a production application. It keeps your application’s availability from depending on one employee’s account.

Step 2: Register your current model provider

Open Providers and connect the service that runs your current model. This lets Model Distillation keep serving the same model while you collect data and train alternatives. If you use the built-in wandb-inference provider, you can skip this step. The provider must expose an OpenAI-compatible Chat Completions endpoint for routed application traffic. Enter the exact API base URL, including any required path prefix such as /v1. The proxy appends /chat/completions. For Studio relabeling and evaluations, the provider named openai uses OpenAI’s Responses API instead. After you save the provider, if the model ID isn’t already available, add or enable it. Wait until the provider deployment is Applied at the current revision before continuing.

Step 3: Create a project

From Projects, select Create or import a project. Create a new W&B project or import one you already have, give the feature a stable proxy model name such as ticket-classifier, and select the model that serves it today. The proxy model name becomes the model value used by your application. It stays the same as you train and deploy new implementations.

API: Create a project (PUT /tasks/{alias})

The API calls a project a task, and the proxy model name is the alias in the URL. Set the W&B project name in project, choose create or import in project_mode, and set the current model in targets:

curl --request PUT \
  --url "https://distillation.training.wandb.ai/v1/tasks/ticket-classifier" \
  --header "Authorization: Bearer $WANDB_API_KEY" \
  --header "Wandb-Entity: your-team" \
  --header "Content-Type: application/json" \
  --data '{
    "project": "ticket-classifier",
    "project_mode": "create",
    "targets": [
      {"model_ref": "openai/gpt-5.6-sol", "weight": 1}
    ]
  }'

For the complete request and response, see Create a Model Distillation project.

Routing is published asynchronously. Open the project’s Routing page and wait until the deployment is Applied at the current routing revision before sending application traffic.

Step 4: Connect your application

Keep your normal OpenAI client, but change its base URL and model:

from openai import OpenAI

client = OpenAI(
    base_url="https://proxy.training.wandb.ai/v1",
    api_key=wandb_api_key,
)

response = client.chat.completions.create(
    model="ticket-classifier",
    messages=messages,
    extra_body={
        "metadata": {
            "wandb.entity": "your-team",
            "wandb.thread_id": conversation_id,
        }
    },
)

Model Distillation now collects useful examples from normal application traffic. For Python, JavaScript, and curl examples, see Connect your application.

Step 5: Create a dataset

After the project has representative traffic, create a dataset. Review the examples and remove poor ones. If you want the new model to learn better behavior than the current model provides, improve the answers. Follow the Datasets Quick Start to create and inspect your first dataset.

Step 6: Fine-tune several models

Train one or more candidate models from the dataset. Trying several candidates gives you a better chance of finding the right balance of quality, speed, and cost. Follow the Fine-tuning Quick Start to start a training job.

Step 7: Evaluate the candidates

Compare every candidate with the answers you want to preserve. Review the overall result and individual examples before choosing a winner. Follow the Evaluations Quick Start to run your first comparison.

Step 8: Deploy the winner

Add the winner to project routing. You can start with a small share of traffic, increase it gradually, and keep the original model ready for rollback. Your application keeps calling the same proxy model name, and the new model now serves the share of traffic you assigned to it.

Train and deploy your first model

Follow the complete manual workflow with more detail about each decision.

​Step 1: Sign in with W&B

​Step 2: Register your current model provider

​Step 3: Create a project

​Step 4: Connect your application

​Step 5: Create a dataset

​Step 6: Fine-tune several models

​Step 7: Evaluate the candidates

​Step 8: Deploy the winner

Train and deploy your first model

Step 1: Sign in with W&B

Step 2: Register your current model provider

Step 3: Create a project

Step 4: Connect your application

Step 5: Create a dataset

Step 6: Fine-tune several models

Step 7: Evaluate the candidates

Step 8: Deploy the winner