Inference proxy overview - CoreWeave Docs

The inference proxy is the only component on your application’s online path. It exposes:

POST https://proxy.training.wandb.ai/v1/chat/completions
Authorization: Bearer <wandb-api-key>

Your existing OpenAI client continues to work. The model value decides whether the request uses task routing or calls a registered provider directly.

What the proxy adds

Task routing

Resolve a task version, choose a weighted target, apply its parameters, and keep conversations sticky.

Credential isolation

Replace the W&B bearer key with the selected provider credential without forwarding client headers.

Training-grade traces

Record the complete dispatched request and response with task and provider identity.

OpenAI-compatible transport

Preserve non-streaming and streaming Chat Completions response behavior.

Supported routes

`model` value	Result
`task-alias`	Weighted routing for task version 1
`task-alias@v2`	Weighted routing pinned to version 2
`provider/model-id`	Direct call through a registered provider

Task requests are traced automatically. Direct provider calls are not traced unless metadata["wandb.project"] selects a destination.

API: Send a task-routed request (POST /chat/completions)

curl --request POST \
  --url "https://proxy.training.wandb.ai/v1/chat/completions" \
  --header "Authorization: Bearer $WANDB_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "model": "ticket-classifier",
    "messages": [{"role": "user", "content": "Classify this request"}],
    "metadata": {"wandb.entity": "your-team"}
  }'

See Create a routed chat completion for the complete request and response schema.

Fail closed behavior

The proxy rejects unknown W&B keys, entities, tasks, versions, and providers before calling an arbitrary endpoint. Provider URLs and credentials come only from server-managed materialization; callers cannot supply them in request headers or tokens.

Last modified on August 25, 2026

​What the proxy adds