api.coreweave.com.
This page covers cross-cutting topics: authentication, protocols, status values, error formats, and the OpenAI-compatible inference endpoint. For per-endpoint request and response schemas, see the per-operation pages under each service in the left sidebar:
The Inference API is versioned as
v1alpha1. APIs may change before general availability.Authentication
The Inference API uses bearer token authentication to identify the caller and authorize each request. All API requests must include a CoreWeave API access token in theAuthorization header as a Bearer token. The token must belong to a user with the Inference Viewer or Inference Admin role, depending on the operation.
Replace [API-TOKEN] with your CoreWeave API access token.
curl "https://api.coreweave.com/v1alpha1/inference/gateways" \
-H "Authorization: Bearer [API-TOKEN]"
Protocol support
You can call the Inference API over several transport protocols, depending on your client tooling and performance needs:| Protocol | Description |
|---|---|
| REST/JSON | Standard HTTP/1.1 with JSON request and response bodies. All examples in this documentation use REST. |
| gRPC | Protocol buffers over HTTP/2 for high-performance programmatic access. |
| Connect | gRPC-compatible protocol with improved browser and HTTP/1.1 support. |
Schema and SDKs
The Inference API is defined in Protobuf and published to CoreWeave’s public Buf Schema Registry, so you can install a generated client instead of writing HTTP calls by hand.- Public BSR module: buf.build/coreweave/inference (module name
buf.build/coreweave/inference). - Generated SDKs: Inference SDKs on the Buf Schema Registry. Clients are available for the Connect, gRPC, and Protobuf ecosystems across languages including Go, Python, TypeScript, Java, Kotlin, Rust, and Swift.
- Services:
GatewayService,DeploymentService, andCapacityClaimService(packagecoreweave.inference.v1alpha1).
Generated clients cover the Inference management API, which is what you use to create and manage gateways, deployments, and capacity claims. To send inference requests to a deployed model, use any OpenAI-compatible client against your gateway endpoint. See OpenAI-compatible endpoint.
Query parameters
List endpoints support the following query parameter:| Parameter | Type | Description |
|---|---|---|
updatedAfter | date-time | Filter resources to those updated after the specified timestamp (ISO 8601 format). |
Status values
Use these status values to determine the lifecycle state of a resource when polling or reconciling state in your application. All resources share a common set of status values:| Status | Description |
|---|---|
STATUS_UNSPECIFIED | Status has not been set. |
STATUS_CREATING | Resource is being provisioned. |
STATUS_READY | Resource is active and operational. |
STATUS_UPDATING | Resource configuration is being updated. |
STATUS_DELETING | Resource is being removed. |
STATUS_ERROR | Resource encountered a recoverable error. |
STATUS_FAILED | Resource encountered a terminal error. |
conditions array in its status with detailed information about the current state, including timestamps, reasons, and human-readable messages. Each condition’s status field is STATUS_TRUE, STATUS_FALSE, or STATUS_UNSPECIFIED when it has not been set.
Error responses
When a request fails, the API returns a structured error body that your client can parse to surface details to users or trigger retries. Error responses follow the standard format:{
"code": 400,
"message": "name is required",
"details": []
}
OpenAI-compatible endpoint
Deployed models expose an OpenAI-compatible completions endpoint through their associated gateway. The endpoint URL depends on the gateway’s routing strategy. See Gateways for details on how each routing strategy constructs the request URL. For a complete walkthrough, see the Getting started guide.OpenAPI specification
The complete OpenAPI 3.0 specification for the Inference API is embedded below. Expand the code block by clicking “See all” at the bottom, or use the download and copy buttons in the title bar.Inference OpenAPI
openapi: 3.0.3
info:
title: CoreWeave Inference API
version: 0.0.1
description: The CoreWeave Inference API provides programmatic control over inference gateways, model
deployments, and capacity claims.
servers:
- url: https://api.coreweave.com
description: CoreWeave production API.
paths:
/v1alpha1/inference/capacityclaims:
get:
operationId: CapacityClaimService_ListCapacityClaims
summary: List capacity claims
description: Returns every CapacityClaim that the caller can read.
parameters:
- name: updatedAfter
in: query
description: Filter by those updated after this timestamp
schema:
type: string
format: date-time
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/ListCapacityClaimsResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- CapacityClaimService
post:
operationId: CapacityClaimService_CreateCapacityClaim
summary: Create capacity claim
description: Creates a new CapacityClaim that reserves GPU hardware for inference deployments.
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/CreateCapacityClaimRequest'
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/CreateCapacityClaimResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- CapacityClaimService
/v1alpha1/inference/capacityclaims/parameters:
get:
operationId: CapacityClaimService_GetCapacityClaimParameters
summary: Get capacity claim parameters
description: Returns the available instance types per Availability Zone for use when creating a
CapacityClaim.
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/GetCapacityClaimParametersResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- CapacityClaimService
/v1alpha1/inference/capacityclaims/{id}:
get:
operationId: CapacityClaimService_GetCapacityClaim
summary: Get capacity claim
description: Retrieves a single CapacityClaim by its UUID.
parameters:
- name: id
in: path
description: The unique identifier of the CapacityClaim to get, UUID format
required: true
schema:
type: string
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/GetCapacityClaimResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- CapacityClaimService
delete:
operationId: CapacityClaimService_DeleteCapacityClaim
summary: Delete capacity claim
description: Deletes a CapacityClaim by ID. Returns the deleted CapacityClaim.
parameters:
- name: id
in: path
description: The unique identifier of the CapacityClaim to delete, UUID format
required: true
schema:
type: string
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/DeleteCapacityClaimResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- CapacityClaimService
patch:
operationId: CapacityClaimService_UpdateCapacityClaim
summary: Update capacity claim
description: Updates an existing CapacityClaim. The full `resources` block is required and replaces
the existing reservation.
parameters:
- name: id
in: path
description: The unique identifier of the CapacityClaim to update, UUID format
required: true
schema:
type: string
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/UpdateCapacityClaimRequest'
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/UpdateCapacityClaimResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- CapacityClaimService
/v1alpha1/inference/deployments:
get:
operationId: DeploymentService_ListDeployments
summary: List deployments
description: Returns every deployment that the caller can read, optionally filtered by parent gateway
or update timestamp.
parameters:
- name: parentGatewayId
in: query
description: The ID of the parent gateway to filter deployments by
schema:
type: string
- name: updatedAfter
in: query
description: Filter by those updated after this timestamp
schema:
type: string
format: date-time
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/ListDeploymentsResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- DeploymentService
post:
operationId: DeploymentService_CreateDeployment
summary: Create deployment
description: Creates a new model deployment associated with one or more gateways.
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/CreateDeploymentRequest'
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/CreateDeploymentResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- DeploymentService
/v1alpha1/inference/deployments/parameters:
get:
operationId: DeploymentService_GetDeploymentParameters
summary: Get deployment parameters
description: Returns the available gateways, runtime engines and versions, runtime config keys,
and instance types for use when creating a deployment.
parameters:
- name: gatewayIds
in: query
description: "Optional set of gateway IDs to scope the returned parameters to. When provided,\
\ the response\n enumerates only the values valid across every listed gateway, so a deployment\
\ targeting this\n exact set and using only the returned values passes validation. Empty means\
\ all of the\n organization's gateways (the values are enumerated across them without the cross-gateway\n\
\ intersection)."
schema:
type: array
items:
type: string
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/GetDeploymentParametersResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- DeploymentService
/v1alpha1/inference/deployments/{id}:
get:
operationId: DeploymentService_GetDeployment
summary: Get deployment
description: Retrieves a single deployment by ID.
parameters:
- name: id
in: path
description: The ID of the deployment to get
required: true
schema:
type: string
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/GetDeploymentResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- DeploymentService
delete:
operationId: DeploymentService_DeleteDeployment
summary: Delete deployment
description: Deletes a deployment by ID. Returns the deleted deployment.
parameters:
- name: id
in: path
description: The ID of the deployment to delete
required: true
schema:
type: string
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/DeleteDeploymentResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- DeploymentService
patch:
operationId: DeploymentService_UpdateDeployment
summary: Update deployment
description: Updates an existing deployment. The request body replaces the deployment's spec; this
endpoint does not accept a field mask.
parameters:
- name: id
in: path
description: The ID of the deployment to update
required: true
schema:
type: string
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/UpdateDeploymentRequest'
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/UpdateDeploymentResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- DeploymentService
/v1alpha1/inference/gateways:
get:
operationId: GatewayService_ListGateways
summary: List gateways
description: Returns every inference gateway that the caller can read, optionally filtered by update
timestamp.
parameters:
- name: updatedAfter
in: query
description: Filter by those updated after this timestamp
schema:
type: string
format: date-time
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/ListGatewaysResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- GatewayService
post:
operationId: GatewayService_CreateGateway
summary: Create gateway
description: Creates a new inference gateway. Each gateway must be configured with exactly one authentication
type and exactly one routing strategy.
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/CreateGatewayRequest'
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/CreateGatewayResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- GatewayService
/v1alpha1/inference/gateways/parameters:
get:
operationId: GatewayService_GetGatewayParameters
summary: Get gateway parameters
description: Returns the available Availability Zones for use when creating a gateway.
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/GetGatewayParametersResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- GatewayService
/v1alpha1/inference/gateways/{id}:
get:
operationId: GatewayService_GetGateway
summary: Get gateway
description: Retrieves a single gateway by its UUID.
parameters:
- name: id
in: path
description: The unique identifier of the reservation to get, UUID format
required: true
schema:
type: string
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/GetGatewayResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- GatewayService
delete:
operationId: GatewayService_DeleteGateway
summary: Delete gateway
description: Deletes a gateway by ID. Returns the deleted gateway.
parameters:
- name: id
in: path
description: The unique identifier of the gateway to delete, UUID format
required: true
schema:
type: string
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/DeleteGatewayResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- GatewayService
patch:
operationId: GatewayService_UpdateGateway
summary: Update gateway
description: Updates an existing gateway. The request body replaces the gateway's spec; this endpoint
does not accept a field mask.
parameters:
- name: id
in: path
description: The unique identifier of the gateway to update, UUID format
required: true
schema:
type: string
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/UpdateGatewayRequest'
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/UpdateGatewayResponse'
default:
description: Default error response
content:
application/json:
schema:
$ref: '#/components/schemas/Status'
tags:
- GatewayService
components:
schemas:
BodyBasedRouting:
description: Body based routing configuration
type: object
properties:
apiType:
description: Well-known API type for routing. `API_TYPE_OPENAI` extracts the model name from
the `model` field in the JSON request body (OpenAI-compatible behavior). The zero value `API_TYPE_UNSPECIFIED`
is never valid to send.
type: string
format: enum
enum:
- API_TYPE_OPENAI
required:
- apiType
CapacityClaim:
description: CapacityClaim object with specification and status fields
type: object
properties:
spec:
description: The specification of the CapacityClaim
allOf:
- $ref: '#/components/schemas/CapacityClaimSpec'
readOnly: true
status:
description: The status of the CapacityClaim
allOf:
- $ref: '#/components/schemas/CapacityClaimStatus'
readOnly: true
CapacityClaimResources:
description: Resources to reserve within a CapacityClaim
type: object
properties:
instanceId:
description: The instance type to reserve, by ID specifier (for example, `gb200-4x`). Identifiers
are case-sensitive and must use the lowercase form. Must be a valid instance type within at
least one of the specified `zones`.
type: string
instanceCount:
description: The number of instances to reserve.
type: integer
format: uint32
capacityType:
description: The capacity type for the CapacityClaim. `CAPACITY_TYPE_MANAGED` reserves instances
from CoreWeave's managed capacity pool and is the only value accepted when creating or updating
a claim. `CAPACITY_TYPE_CUSTOMER` is deprecated; it is not accepted on create or update, but
can still appear when reading older claims. This field cannot be changed after the claim is
created.
type: string
format: enum
enum:
- CAPACITY_TYPE_CUSTOMER
- CAPACITY_TYPE_MANAGED
zones:
description: The Availability Zones the CapacityClaim may use resources from. Zone identifiers
are case-sensitive and must use the uppercase canonical form (for example, `US-WEST-04A`).
The order of `zones` implies an allocation preference; earlier entries are preferred.
type: array
items:
type: string
required:
- instanceId
- instanceCount
- capacityType
- zones
CapacityClaimSpec:
description: CapacityClaimSpec contains the specification for a CapacityClaim
type: object
properties:
id:
description: The unique identifier of the CapacityClaim, UUID format
type: string
readOnly: true
name:
description: The human readable name of the CapacityClaim
type: string
readOnly: true
resources:
description: The specification for the resources to reserve
allOf:
- $ref: '#/components/schemas/CapacityClaimResources'
readOnly: true
organizationId:
description: The organization ID that owns the CapacityClaim
type: string
readOnly: true
CapacityClaimStatus:
description: CapacityClaimStatus contains the status fields for a CapacityClaim
type: object
properties:
createdAt:
description: The time at which the CapacityClaim was created.
type: string
format: date-time
readOnly: true
updatedAt:
description: The time at which the CapacityClaim was last updated.
type: string
format: date-time
readOnly: true
status:
description: The overall status of the CapacityClaim. `STATUS_UNSPECIFIED` means the status
has not been set. See the [Inference API overview](https://docs.coreweave.com/products/inference/reference/api-overview#status-values)
for the meaning of each value.
type: string
format: enum
enum:
- STATUS_UNSPECIFIED
- STATUS_CREATING
- STATUS_UPDATING
- STATUS_DELETING
- STATUS_ERROR
- STATUS_FAILED
- STATUS_READY
readOnly: true
conditions:
description: List of conditions representing detailed status information
type: array
items:
$ref: '#/components/schemas/Condition'
readOnly: true
allocatedInstances:
description: The number of instances currently allocated and available.
type: integer
format: uint32
readOnly: true
pendingInstances:
description: The number of instances being provisioned to fulfill the capacity claim.
type: integer
format: uint32
readOnly: true
Condition:
description: Condition represents a detailed status condition for resources
type: object
properties:
type:
description: Type of condition
type: string
readOnly: true
status:
description: 'The status of the condition: `STATUS_TRUE` or `STATUS_FALSE`, or `STATUS_UNSPECIFIED`
when the status has not been set.'
type: string
format: enum
enum:
- STATUS_UNSPECIFIED
- STATUS_FALSE
- STATUS_TRUE
readOnly: true
lastUpdateTime:
description: Last time the condition was updated
type: string
format: date-time
readOnly: true
reason:
description: Reason for the condition's last transition
type: string
readOnly: true
message:
description: Human-readable message indicating details about the condition
type: string
readOnly: true
zone:
description: Zone associated with the condition
type: string
readOnly: true
CoreWeaveAuth:
description: Use CoreWeave IAM authentication, no additional configuration
type: object
properties: {}
CreateCapacityClaimRequest:
description: Request to create a CapacityClaim
type: object
properties:
id:
description: The unique identifier of the CapacityClaim, UUID format
type: string
name:
description: The human readable name of the CapacityClaim
type: string
resources:
description: The specification for the resources to reserve
allOf:
- $ref: '#/components/schemas/CapacityClaimResources'
required:
- resources
CreateCapacityClaimResponse:
description: Response for CreateCapacityClaim
type: object
properties:
capacityClaim:
description: The created CapacityClaim
allOf:
- $ref: '#/components/schemas/CapacityClaim'
readOnly: true
CreateDeploymentRequest:
description: Request for CreateDeployment
type: object
properties:
id:
description: The unique identifier of the deployment, UUID format
type: string
name:
description: The name of the deployment
type: string
gatewayIds:
description: The gateways to associate the deployment with
type: array
items:
type: string
runtime:
description: Runtime selection and configuration
allOf:
- $ref: '#/components/schemas/DeploymentRuntime'
resources:
description: Resource configuration for the deployment
allOf:
- $ref: '#/components/schemas/DeploymentResources'
model:
description: The model configuration
allOf:
- $ref: '#/components/schemas/DeploymentModel'
autoscaling:
description: The autoscaling configuration
allOf:
- $ref: '#/components/schemas/DeploymentAutoscaling'
traffic:
description: The traffic configuration for the deployment
allOf:
- $ref: '#/components/schemas/DeploymentTraffic'
disabled:
description: Disable the deployment
type: boolean
required:
- name
- gatewayIds
- runtime
- resources
- model
- autoscaling
- traffic
CreateDeploymentResponse:
description: Response for CreateDeployment
type: object
properties:
deployment:
description: The deployment that was created
allOf:
- $ref: '#/components/schemas/Deployment'
readOnly: true
CreateGatewayRequest:
description: Request for CreateGateway
type: object
properties:
id:
description: The unique identifier of the gateway, UUID format
type: string
name:
description: "The human readable name of the gateway. Capped at 38 characters so the derived\n\
\ production FQDN (<name>.<orgId>.gw.cwinference.com) fits within the 64-character\n X.509\
\ Common Name limit."
type: string
zones:
description: "The zones to make the gateway available in, limits where deployments associated\
\ with the gateway may exist, no\n zones means all may be used."
type: array
items:
type: string
coreWeaveAuth:
description: CoreWeave IAM
allOf:
- $ref: '#/components/schemas/CoreWeaveAuth'
weightsAndBiasesAuth:
description: Weights & Biases
allOf:
- $ref: '#/components/schemas/WeightsAndBiasesAuth'
endpointConfiguration:
description: Additional endpoint configuration options
allOf:
- $ref: '#/components/schemas/EndpointConfiguration'
bodyBasedRouting:
description: Body based routing
allOf:
- $ref: '#/components/schemas/BodyBasedRouting'
headerBasedRouting:
description: Header based routing
allOf:
- $ref: '#/components/schemas/HeaderBasedRouting'
pathBasedRouting:
description: Path based routing
allOf:
- $ref: '#/components/schemas/PathBasedRouting'
required:
- name
CreateGatewayResponse:
description: Response for CreateGateway
type: object
properties:
gateway:
description: The gateway that was created
allOf:
- $ref: '#/components/schemas/Gateway'
readOnly: true
DeleteCapacityClaimResponse:
description: Response for DeleteCapacityClaim
type: object
properties:
capacityClaim:
description: The deleted CapacityClaim
allOf:
- $ref: '#/components/schemas/CapacityClaim'
readOnly: true
DeleteDeploymentResponse:
description: Response for DeleteDeployment
type: object
properties:
deployment:
description: The deployment that was deleted
allOf:
- $ref: '#/components/schemas/Deployment'
readOnly: true
DeleteGatewayResponse:
description: Response for DeleteGateway
type: object
properties:
gateway:
description: The gateway that was deleted
allOf:
- $ref: '#/components/schemas/Gateway'
readOnly: true
Deployment:
description: Deployment object with specification and status fields
type: object
properties:
spec:
description: The specification of the deployment
allOf:
- $ref: '#/components/schemas/DeploymentSpec'
readOnly: true
status:
description: The status of the deployment
allOf:
- $ref: '#/components/schemas/DeploymentStatus'
readOnly: true
DeploymentAutoscaling:
description: Autoscaling configuration for the deployment
type: object
properties:
min:
description: The minimum number of replicas. Must be at least `1`; scale-to-zero is not supported.
type: integer
format: uint32
max:
description: The maximum number of replicas. Must be greater than or equal to `min`. Set equal
to `min` to disable autoscaling.
type: integer
format: uint32
priority:
description: Scaling priority relative to other deployments, from `0` to `1000`. Higher values
receive scaling preference during resource contention.
type: integer
format: uint32
capacityClasses:
description: The capacity classes the autoscaler may use for this deployment. Set `CAPACITY_CLASS_RESERVED`
to schedule replicas onto capacity reserved by a CapacityClaim; set `CAPACITY_CLASS_ON_DEMAND`
to use shared on-demand capacity. Omit the zero value `CAPACITY_CLASS_UNSPECIFIED` rather
than sending it.
type: array
items:
type: string
format: enum
enum:
- CAPACITY_CLASS_RESERVED
- CAPACITY_CLASS_ON_DEMAND
concurrency:
description: Target concurrent requests per replica. Lower values reduce latency; higher values
increase throughput. Must be at least `1`.
type: integer
format: uint32
required:
- min
- max
DeploymentModel:
description: Configuration of the model
type: object
properties:
name:
description: The model name used in inference requests (and returned by the gateway's `/models`
endpoint). When multiple deployments on the same gateway share a name, traffic is split between
them by `traffic.weight`.
type: string
bucket:
description: The CoreWeave AI Object Storage (CAIOS) bucket the model is stored in.
type: string
path:
description: The path within `bucket` to the model and its configuration files.
type: string
required:
- name
- bucket
- path
DeploymentResourceParameters:
description: The available parameters for resource configuration
type: object
properties:
instanceTypes:
description: "Available instance types. May be empty — e.g. when a gateway_ids filter selects\
\ gateways that\n share no common instance type, or when the organization has no gateways."
type: array
items:
type: string
readOnly: true
DeploymentResources:
description: Resource configuration for the deployment
type: object
properties:
instanceType:
description: The instance type to use for the deployment. The available values are returned
by `GET /v1alpha1/inference/deployments/parameters`.
type: string
gpuCount:
description: Number of GPUs needed per instance. CPU and RAM are automatically assigned based
on the chosen GPU count.
type: integer
format: uint32
required:
- instanceType
- gpuCount
DeploymentRuntime:
description: Runtime selection and configuration for deployments
type: object
properties:
engine:
description: The inference engine to use (for example, `vllm`). The available engines are returned
by `GET /v1alpha1/inference/deployments/parameters`.
type: string
version:
description: The version of the engine. Defaults to `latest` when unset. Must be one of the
engine's available versions returned by the parameters endpoint.
type: string
engineConfig:
description: Engine-specific configuration arguments. The allowed keys per engine are returned
under `runtimeParameters.runtimeConfigOptions` by the parameters endpoint.
type: object
additionalProperties:
type: string
engineEnv:
description: Map of engine-specific environment variables to inject into the model runtime container
type: object
additionalProperties:
type: string
required:
- engine
- version
DeploymentRuntimeParameters:
description: The available parameters for runtime configuration
type: object
properties:
runtimeVersions:
description: The available runtime versions per engine
type: object
additionalProperties:
$ref: '#/components/schemas/DeploymentRuntimeParameters_RuntimeVersions'
readOnly: true
runtimeConfigOptions:
description: The available runtime config options per engine
type: object
additionalProperties:
$ref: '#/components/schemas/DeploymentRuntimeParameters_RuntimeConfigOptions'
readOnly: true
engineEnvOptions:
description: The available engine environment variable options per engine
type: object
additionalProperties:
$ref: '#/components/schemas/DeploymentRuntimeParameters_EngineEnvOptions'
readOnly: true
DeploymentRuntimeParameters_EngineEnvOptions:
description: The available engine environment variable options
type: object
properties:
allowedNames:
description: The allowed engine environment variable names.
type: array
items:
type: string
readOnly: true
DeploymentRuntimeParameters_RuntimeConfigOptions:
description: The available runtime config options
type: object
properties:
allowedKeys:
description: The allowed engine config keys
type: array
items:
type: string
readOnly: true
DeploymentRuntimeParameters_RuntimeVersions:
description: List of runtime versions
type: object
properties:
versions:
description: The version strings
type: array
items:
type: string
readOnly: true
DeploymentSpec:
description: DeploymentSpec object with configuration fields
type: object
properties:
id:
description: The ID of the deployment
type: string
readOnly: true
name:
description: The name of the deployment
type: string
readOnly: true
gatewayIds:
description: The IDs of the gateways the deployment is associated with. A deployment may be
attached to multiple gateways; traffic from each gateway is routed by that gateway's strategy.
type: array
items:
type: string
readOnly: true
runtime:
description: Runtime selection and configuration. The available engines, versions, and per-engine
config keys are returned by `GET /v1alpha1/inference/deployments/parameters`.
allOf:
- $ref: '#/components/schemas/DeploymentRuntime'
readOnly: true
resources:
description: Resource configuration for the deployment. CPU and RAM are automatically assigned
based on `gpuCount` for the chosen `instanceType`.
allOf:
- $ref: '#/components/schemas/DeploymentResources'
readOnly: true
model:
description: The model configuration. The model is loaded from a path in a CoreWeave AI Object
Storage (CAIOS) bucket.
allOf:
- $ref: '#/components/schemas/DeploymentModel'
readOnly: true
autoscaling:
description: Autoscaling configuration. CoreWeave manages scaling within the configured `min`
and `max` bounds; scale-to-zero is not supported.
allOf:
- $ref: '#/components/schemas/DeploymentAutoscaling'
readOnly: true
traffic:
description: Traffic configuration for the deployment. When multiple deployments share a model
name on the same gateway, `traffic.weight` controls the share of traffic each deployment receives.
allOf:
- $ref: '#/components/schemas/DeploymentTraffic'
readOnly: true
organizationId:
description: The organization ID that owns the deployment.
type: string
readOnly: true
disabled:
description: When `true`, the deployment does not receive traffic and is not scaled.
type: boolean
readOnly: true
DeploymentStatus:
description: DeploymentStatus object for status fields
type: object
properties:
createdAt:
description: The time at which the deployment was created.
type: string
format: date-time
readOnly: true
updatedAt:
description: The time at which the deployment was last updated.
type: string
format: date-time
readOnly: true
status:
description: The overall status of the deployment. `STATUS_UNSPECIFIED` means the status has
not been set. See the [Inference API overview](https://docs.coreweave.com/products/inference/reference/api-overview#status-values)
for the meaning of each value.
type: string
format: enum
enum:
- STATUS_UNSPECIFIED
- STATUS_CREATING
- STATUS_UPDATING
- STATUS_DELETING
- STATUS_ERROR
- STATUS_FAILED
- STATUS_READY
readOnly: true
conditions:
description: List of conditions representing detailed status information
type: array
items:
$ref: '#/components/schemas/Condition'
readOnly: true
DeploymentTraffic:
description: Configuration for traffic control to the deployment
type: object
properties:
weight:
description: The relative weight of traffic routed to this deployment compared to other deployments
with the same model name on the same gateway. Weights are normalized into percentages.
type: integer
format: uint32
EndpointConfiguration:
description: Configuration options for the gateway endpoint, DNS, TLS, etc.
type: object
properties:
additionalDns:
description: Additional DNS configurations, these DNS need to be configured to point to the
gateway endpoint manually
type: array
items:
type: string
edgeProxyMode:
description: 'How client traffic reaches the gateway. `EDGE_PROXY_MODE_MANAGED` proxies traffic
through CoreWeave''s managed edge network: DDoS mitigation and WAF, with TLS terminated at
the edge and the origin address kept out of public DNS. `EDGE_PROXY_MODE_DIRECT` connects
clients straight to the gateway''s load balancer, with no edge filtering and the load balancer
address published in DNS. `EDGE_PROXY_MODE_UNSPECIFIED` (or omitting the field) applies the
platform default, currently managed.'
type: string
format: enum
enum:
- EDGE_PROXY_MODE_UNSPECIFIED
- EDGE_PROXY_MODE_MANAGED
- EDGE_PROXY_MODE_DIRECT
allowedSourceIpRanges:
description: "Client source IP ranges permitted to reach the gateway, in CIDR notation (IPv4\
\ or IPv6), up to\n 50 entries. An empty list applies no source restriction. Use a full-length\
\ prefix for a single\n address (e.g. \"203.0.113.5/32\")."
type: array
items:
type: string
Gateway:
description: Gateway object with specification and status fields
type: object
properties:
spec:
description: The specification of the gateway
allOf:
- $ref: '#/components/schemas/GatewaySpec'
readOnly: true
status:
description: The status of the gateway
allOf:
- $ref: '#/components/schemas/GatewayStatus'
readOnly: true
GatewaySpec:
description: GatewaySpec contains the specification for a Gateway
type: object
properties:
id:
description: The unique identifier of the gateway, UUID format
type: string
readOnly: true
name:
description: "The human readable name of the gateway. Combined with organization_id and the\n\
\ deployment base domain it forms the gateway's public FQDN\n (<name>.<orgId>.gw.<baseDomain>).\
\ See CreateGatewayRequest.name and\n UpdateGatewayRequest.name for the length cap applied\
\ on write."
type: string
readOnly: true
zones:
description: The Availability Zones the gateway is deployed in. The field accepts an array,
but gateways are currently limited to one zone. The available zones are returned by `GET /v1alpha1/inference/gateways/parameters`.
type: array
items:
type: string
readOnly: true
coreWeaveAuth:
description: CoreWeave IAM authentication. Validates inference requests using the same CoreWeave
API access tokens used to call the management API.
allOf:
- $ref: '#/components/schemas/CoreWeaveAuth'
weightsAndBiasesAuth:
description: Weights & Biases authentication. Validates inference requests using W&B SaaS credentials.
W&B self-hosted is not currently supported.
allOf:
- $ref: '#/components/schemas/WeightsAndBiasesAuth'
organizationId:
description: The organization ID that owns the gateway
type: string
readOnly: true
endpointConfiguration:
description: Optional endpoint configuration. Use `additionalDns` to register additional DNS
names that you point at the gateway endpoint manually.
allOf:
- $ref: '#/components/schemas/EndpointConfiguration'
bodyBasedRouting:
description: Body based routing
allOf:
- $ref: '#/components/schemas/BodyBasedRouting'
headerBasedRouting:
description: Header based routing
allOf:
- $ref: '#/components/schemas/HeaderBasedRouting'
pathBasedRouting:
description: Path based routing
allOf:
- $ref: '#/components/schemas/PathBasedRouting'
GatewayStatus:
description: GatewayStatus contains the status fields for a Gateway
type: object
properties:
createdAt:
description: The time at which the gateway was created.
type: string
format: date-time
readOnly: true
updatedAt:
description: The time at which the gateway was last updated.
type: string
format: date-time
readOnly: true
status:
description: The overall status of the gateway. `STATUS_UNSPECIFIED` means the status has not
been set. See the [Inference API overview](https://docs.coreweave.com/products/inference/reference/api-overview#status-values)
for the meaning of each value. Once `STATUS_READY`, the gateway exposes one or more endpoint
URLs in `endpoints`.
type: string
format: enum
enum:
- STATUS_UNSPECIFIED
- STATUS_CREATING
- STATUS_UPDATING
- STATUS_DELETING
- STATUS_ERROR
- STATUS_FAILED
- STATUS_READY
readOnly: true
conditions:
description: List of conditions representing detailed status information
type: array
items:
$ref: '#/components/schemas/Condition'
readOnly: true
endpoints:
description: The endpoint URLs for the gateway. The default endpoint follows the pattern `https://api.[GATEWAY-ID].gw.cwinference.com`;
additional entries appear when `endpointConfiguration.additionalDns` is set.
type: array
items:
type: string
readOnly: true
GetCapacityClaimParametersResponse:
description: Response for GetCapacityClaimParameters
type: object
properties:
zoneInstanceTypes:
description: Map keyed by Availability Zone (for example, `US-WEST-04A`) whose values list the
`instanceId` strings that are valid for a CapacityClaim in that zone.
type: object
additionalProperties:
$ref: '#/components/schemas/GetCapacityClaimParametersResponse_InstanceTypes'
readOnly: true
GetCapacityClaimParametersResponse_InstanceTypes:
description: List of available instance types
type: object
properties:
instanceIds:
description: List of instance type IDs
type: array
items:
type: string
readOnly: true
GetCapacityClaimResponse:
description: Response for GetCapacityClaim
type: object
properties:
capacityClaim:
description: The CapacityClaim
allOf:
- $ref: '#/components/schemas/CapacityClaim'
readOnly: true
GetDeploymentParametersResponse:
description: Response for GetDeploymentParameters
type: object
properties:
gatewayIds:
description: The gateway IDs in the caller's organization that can be referenced by a new deployment.
type: array
items:
type: string
runtimeParameters:
description: The available runtime parameters for the deployments
allOf:
- $ref: '#/components/schemas/DeploymentRuntimeParameters'
readOnly: true
resourceParameters:
description: The available resource parameters for the deployments
allOf:
- $ref: '#/components/schemas/DeploymentResourceParameters'
readOnly: true
required:
- gatewayIds
GetDeploymentResponse:
description: Response for GetDeployment
type: object
properties:
deployment:
description: The deployment
allOf:
- $ref: '#/components/schemas/Deployment'
readOnly: true
GetGatewayParametersResponse:
description: Response for GetGatewayParameters
type: object
properties:
zones:
description: The Availability Zone identifiers that are valid in a gateway's `zones` field.
type: array
items:
type: string
required:
- zones
GetGatewayResponse:
description: Response for GetGateway
type: object
properties:
gateway:
description: The gateway
allOf:
- $ref: '#/components/schemas/Gateway'
readOnly: true
GoogleProtobufAny:
description: Contains an arbitrary serialized message along with a @type that describes the type
of the serialized message.
type: object
properties:
'@type':
description: The type of the serialized message.
type: string
additionalProperties: true
HeaderBasedRouting:
description: Header based routing configuration, uses a header value to route requests to the model
by name
type: object
properties:
headerName:
description: The HTTP header to read the model name from on each inference request. Must be
1-100 characters.
type: string
required:
- headerName
ListCapacityClaimsResponse:
description: Response for ListCapacityClaims
type: object
properties:
capacityClaims:
description: The list of CapacityClaims
type: array
items:
$ref: '#/components/schemas/CapacityClaim'
readOnly: true
ListDeploymentsResponse:
description: Response for ListDeployments
type: object
properties:
items:
description: The list of deployments
type: array
items:
$ref: '#/components/schemas/Deployment'
readOnly: true
ListGatewaysResponse:
description: Response for ListGateways
type: object
properties:
items:
description: The list of gateways
type: array
items:
$ref: '#/components/schemas/Gateway'
readOnly: true
PathBasedRouting:
description: Path-based routing configuration. The model name is read from the first segment of
the URL path (for example, `/my-model/v1/chat/completions`). URL-encode model names that contain
special characters.
type: object
properties: {}
Status:
description: Standard error response. `code` is a [`google.rpc.Code`](https://cloud.google.com/apis/design/errors#error_codes);
`message` is human-readable English; `details` carries machine-readable error details when present.
type: object
properties:
code:
description: The status code, which should be an enum value of [google.rpc.Code][google.rpc.Code].
type: integer
format: int32
message:
description: A developer-facing error message, which should be in English. Any user-facing error
message should be localized and sent in the [google.rpc.Status.details][google.rpc.Status.details]
field, or localized by the client.
type: string
details:
description: A list of messages that carry the error details. There is a common set of message
types for APIs to use.
type: array
items:
$ref: '#/components/schemas/GoogleProtobufAny'
UpdateCapacityClaimRequest:
description: Request to update a CapacityClaim
type: object
properties:
id:
description: The unique identifier of the CapacityClaim to update, UUID format
type: string
resources:
description: The specification for the resources to reserve
allOf:
- $ref: '#/components/schemas/CapacityClaimResources'
required:
- id
- resources
UpdateCapacityClaimResponse:
description: Response for UpdateCapacityClaim
type: object
properties:
capacityClaim:
description: The updated CapacityClaim
allOf:
- $ref: '#/components/schemas/CapacityClaim'
readOnly: true
UpdateDeploymentRequest:
description: Request for UpdateDeployment
type: object
properties:
id:
description: The ID of the deployment to update
type: string
name:
description: The name of the deployment
type: string
gatewayIds:
description: The gateways to associate the deployment with
type: array
items:
type: string
runtime:
description: Runtime selection and configuration
allOf:
- $ref: '#/components/schemas/DeploymentRuntime'
resources:
description: Resource configuration for the deployment
allOf:
- $ref: '#/components/schemas/DeploymentResources'
model:
description: The model configuration
allOf:
- $ref: '#/components/schemas/DeploymentModel'
autoscaling:
description: The autoscaling configuration
allOf:
- $ref: '#/components/schemas/DeploymentAutoscaling'
traffic:
description: The traffic configuration for the deployment
allOf:
- $ref: '#/components/schemas/DeploymentTraffic'
disabled:
description: Disable the deployment
type: boolean
required:
- id
- name
- gatewayIds
- runtime
- resources
- model
- autoscaling
- traffic
UpdateDeploymentResponse:
description: Response for UpdateDeployment
type: object
properties:
deployment:
description: The updated deployment
allOf:
- $ref: '#/components/schemas/Deployment'
readOnly: true
UpdateGatewayRequest:
description: Request for UpdateGateway
type: object
properties:
id:
description: The unique identifier of the gateway to update, UUID format
type: string
name:
description: "The human readable name of the gateway. Capped at 38 characters, matching\n CreateGatewayRequest.name."
type: string
zones:
description: "The zones to make the gateway available in, limits where deployments associated\
\ with the gateway may exist, no\n zones means all may be used."
type: array
items:
type: string
coreWeaveAuth:
description: CoreWeave IAM
allOf:
- $ref: '#/components/schemas/CoreWeaveAuth'
weightsAndBiasesAuth:
description: Weights & Biases
allOf:
- $ref: '#/components/schemas/WeightsAndBiasesAuth'
endpointConfiguration:
description: Additional endpoint configuration options
allOf:
- $ref: '#/components/schemas/EndpointConfiguration'
bodyBasedRouting:
description: Body based routing
allOf:
- $ref: '#/components/schemas/BodyBasedRouting'
headerBasedRouting:
description: Header based routing
allOf:
- $ref: '#/components/schemas/HeaderBasedRouting'
pathBasedRouting:
description: Path based routing
allOf:
- $ref: '#/components/schemas/PathBasedRouting'
required:
- id
- name
UpdateGatewayResponse:
description: Response for UpdateGateway
type: object
properties:
gateway:
description: The updated gateway
allOf:
- $ref: '#/components/schemas/Gateway'
readOnly: true
WeightsAndBiasesAuth:
description: Use W&B auth
type: object
properties:
apiKey:
description: The W&B organization API key. Required if `serverUrl` is set.
type: string
serverUrl:
description: Optional W&B server URL. Defaults to the W&B shared SaaS instance.
type: string
enableUsageReports:
description: Send inference usage data to W&B for tracking.
type: boolean
enableRateLimiting:
description: Enable W&B-controlled rate limiting for inference requests.
type: boolean
securitySchemes:
bearerAuth:
type: http
scheme: bearer
bearerFormat: JWT
description: CoreWeave API access token sent as a bearer token.
x-default: Bearer {API_ACCESS_TOKEN}
tags:
- name: CapacityClaimService
description: Endpoints for creating, listing, getting, updating, and deleting CapacityClaim reservations
of GPU hardware for inference deployments.
- name: DeploymentService
description: Endpoints for creating, listing, getting, updating, and deleting model deployments. Each
deployment associates a model with one or more gateways and configures runtime, resources, autoscaling,
and traffic.
- name: GatewayService
description: Endpoints for creating, listing, getting, updating, and deleting inference gateways. Gateways
provide authentication, request routing, load balancing, and traffic splitting for one or more deployments.
security:
- bearerAuth: []