Connect your application - CoreWeave Docs

Use this guide to send a request through the Model Distillation proxy and verify that it records the request. Most Chat Completions applications only need a new base URL, API key, and project, plus a model value of router. The rest of the request and response handling stays the same.

Before you begin

You need a Forge API key, a project with routing applied, and the project’s OpenAI-Project value. The value has the form [ENTITY]/[PROJECT], where [PROJECT] is the project’s Weights & Biases project name, not its project alias. Studio shows the value, ready to copy, in the project header and on the project’s settings page. The examples use your-team/ticket-classifier. Replace it with your project’s OpenAI-Project value.

Python
JavaScript
cURL

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://proxy.training.coreweave.com/v1",
    api_key=os.environ["WANDB_API_KEY"],
    project="your-team/ticket-classifier",  # sent as OpenAI-Project
)

response = client.chat.completions.create(
    model="router",
    messages=[
        {"role": "user", "content": "My package arrived damaged. What should I do?"}
    ],
    metadata={"gen_ai.conversation.id": "customer-4827"},
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://proxy.training.coreweave.com/v1",
  apiKey: process.env.WANDB_API_KEY,
  project: "your-team/ticket-classifier", // sent as OpenAI-Project
});

const response = await client.chat.completions.create({
  model: "router",
  messages: [
    { role: "user", content: "My package arrived damaged. What should I do?" },
  ],
  metadata: { "gen_ai.conversation.id": "customer-4827" },
});

console.log(response.choices[0].message.content);

curl https://proxy.training.coreweave.com/v1/chat/completions \
  -H "Authorization: Bearer $WANDB_API_KEY" \
  -H "Content-Type: application/json" \
  -H "OpenAI-Project: your-team/ticket-classifier" \
  -d '{
    "model": "router",
    "messages": [
      {"role": "user", "content": "My package arrived damaged. What should I do?"}
    ],
    "metadata": {
      "gen_ai.conversation.id": "customer-4827"
    }
  }'

The OpenAI SDKs send the project option as the OpenAI-Project header. They also read it from the OPENAI_PROJECT_ID environment variable. With the header, model: "router" uses the weighted routing of project version 1. To use another version, add "forge.project_version": "2" to metadata. Pass metadata directly to create(), as in the examples. Older releases of the openai Python package don’t have the metadata parameter. Upgrade the package, or pass the same object as extra_body={"metadata": {...}}.

Verify the integration

After you send a request, return to the project overview. Confirm the following:

The request count increased.
The expected model served the request.
The saved input and answer look correct.

If the proxy returns 404 with No routing configuration for project, check the OpenAI-Project value. It must use the Weights & Biases project name, not the project alias. Also check that routing is applied. For other failures, see Proxy errors.

Preserve conversation identity

Use the same gen_ai.conversation.id for every request in one conversation. This keeps a conversation on the same model during a gradual rollout and keeps related examples together when preparing data.

Chat Completions details

Learn about streaming, tools, structured output, provider parameters, and request precedence.