Inference proxy overview - CoreWeave Docs

The inference proxy is the only component on your application’s online path. It exposes:

POST https://proxy.training.coreweave.com/v1/chat/completions
Authorization: Bearer <wandb-api-key>
OpenAI-Project: [ENTITY]/[PROJECT]

The bearer token is your Forge API key. OpenAI-Project selects the Model Distillation project by its entity and Weights & Biases project name. Your existing OpenAI client continues to work: the OpenAI SDKs send the header from their project option or the OPENAI_PROJECT_ID environment variable. The model value determines whether the request uses the project’s weighted routing or overrides the model.

What the proxy adds

Project routing

Resolve a project version, choose a weighted target, apply its parameters, and keep conversations sticky.

Credential isolation

Replace the API key with the selected provider credential without forwarding client headers.

Training-grade traces

Record the complete dispatched request and response with project and provider identity.

OpenAI-compatible transport

Preserve non-streaming and streaming Chat Completions response behavior.

Supported routes

With the OpenAI-Project header, model takes one of these values:

`model` value	Result
`router`, blank, or omitted	Weighted routing for the selected project version
`provider/model-id`	Model override through a registered provider, within the project

metadata["forge.project_version"] selects the project version, such as "2". Omitting it selects version 1. Routed requests and model overrides are both project traffic. The proxy traces, samples, and caches them in the selected version. Non-streaming project requests can also opt into exact-response caching with the Forge-Cache-Mode header, which replays an identical earlier response instead of calling the provider.

API: Send a project-routed request (POST /chat/completions)

curl --request POST \
  --url "https://proxy.training.coreweave.com/v1/chat/completions" \
  --header "Authorization: Bearer $WANDB_API_KEY" \
  --header "Content-Type: application/json" \
  --header "OpenAI-Project: your-team/ticket-classifier" \
  --data '{
    "model": "router",
    "messages": [{"role": "user", "content": "Classify this request"}],
    "metadata": {"gen_ai.conversation.id": "conversation-4827"}
  }'

See Create a routed chat completion for the complete request and response schema.

Fail closed behavior

The proxy rejects unknown API keys, entities, projects, versions, and providers before calling an arbitrary endpoint. Provider URLs and credentials come only from server-managed materialization; callers cannot supply them in request headers or tokens.