Inference proxy overview - CoreWeave Docs

The inference proxy is the only component on your application’s online path. It exposes:

POST https://proxy.training.wandb.ai/v1/chat/completions
Authorization: Bearer <wandb-api-key>

Your existing OpenAI client continues to work. The model value decides whether the request uses project routing or calls a registered provider directly.

What the proxy adds

Project routing

Resolve a project version, choose a weighted target, apply its parameters, and keep conversations sticky.

Credential isolation

Replace the W&B bearer key with the selected provider credential without forwarding client headers.

Training-grade traces

Record the complete dispatched request and response with project and provider identity.

OpenAI-compatible transport

Preserve non-streaming and streaming Chat Completions response behavior.

Supported routes

`model` value	Result
`proxy-model-name`	Weighted routing for project version 1
`proxy-model-name@v2`	Weighted routing pinned to version 2
`provider/model-id`	Direct call through a registered provider

Project requests are traced automatically. Direct provider calls are not traced unless metadata["wandb.project"] selects a destination. Non-streaming project requests can also opt into exact-response caching with the wandb-cache-mode header, which replays an identical earlier response instead of calling the provider.

API: Send a project-routed request (POST /chat/completions)

curl --request POST \
  --url "https://proxy.training.wandb.ai/v1/chat/completions" \
  --header "Authorization: Bearer $WANDB_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "model": "ticket-classifier",
    "messages": [{"role": "user", "content": "Classify this request"}],
    "metadata": {"wandb.entity": "your-team"}
  }'

See Create a routed chat completion for the complete request and response schema.

Fail closed behavior

The proxy rejects unknown W&B keys, entities, projects, versions, and providers before calling an arbitrary endpoint. Provider URLs and credentials come only from server-managed materialization; callers cannot supply them in request headers or tokens.

Last modified on September 3, 2026

​What the proxy adds