Run a GPU sandbox - CoreWeave Docs

This guide shows how to create a sandbox with one or more GPUs, confirm the GPU is visible from inside the sandbox, and set the CPU and memory requests that go with it. GPU sandboxes run in either placement mode: on serverless capacity that CoreWeave operates, or on a CoreWeave Kubernetes Service (CKS) cluster you own. The two modes differ in which GPUs you can get and who sets the limits, so this page covers them in separate sections. For CPU-only sandboxes and the two placement modes, see Get started with CoreWeave sandboxes.

GPU sandboxes are in private preview and are enabled per organization. This is separate from CPU-only sandboxes, which are in public preview: an organization that can already run CPU sandboxes still needs GPUs enabled. To request access, contact your account team or email support@wandb.ai. Until GPUs are enabled for your organization, a request that includes a GPU fails with CWSANDBOX_GPU_NOT_ALLOWED and the message this organization is not entitled to create GPU sandboxes.

Before you begin

You need the following:

GPU sandboxes enabled for your organization.
A credential. Serverless accepts a CoreWeave API access token or a Weights & Biases API key. CKS placement requires a CoreWeave API access token. See Choose a credential.
The Python client, cwsandbox 1.14.2 or later. For a W&B API key, install the wandb extra:
uv pip install 'cwsandbox[wandb]>=1.14.2'
For a CoreWeave access token, the base package is enough:
uv pip install 'cwsandbox>=1.14.2'

The TypeScript client doesn’t expose GPU resources yet. Use the Python client for GPU sandboxes.

How GPU requests work

The same rules apply in both placement modes:

A GPU request reserves GPUs and nothing else. It doesn’t add CPU or memory to the sandbox. Set those explicitly, sized for the work you’ll run alongside the GPU. If you leave them out, the sandbox gets the defaultCpu and defaultMemory values from the policy that governs the runner.
Sandboxes get whole GPUs. GPUs aren’t shared, partitioned, or time-sliced between sandboxes.
A sandbox runs on a single Node. The largest sandbox is one full Node, and how many GPUs that is depends on the GPU type. On a Node with 8 GPUs, a sandbox can request anywhere from 1 GPU up to 8. Requests for several GPUs need that many free GPUs, plus the CPU and memory you asked for, on one Node at the same time.
The GPU type is a filter, not a menu. The optional type key must match exactly, including case, one of the GPU types the runner advertises. Omit it to accept any GPU the runner has.

Run a GPU sandbox on serverless capacity

Serverless placement needs no runner and no policy of your own. CoreWeave owns the policy and the hardware.

Available GPUs

Serverless capacity runs one GPU model, the NVIDIA RTX PRO 6000 Blackwell Server Edition.

GPU	GPU memory	GPUs per sandbox
NVIDIA RTX PRO 6000 Blackwell Server Edition	96 GB	1 to 8

Two instance types carry it: High Memory and Standard Memory. They differ in host RAM, not in the GPU, and both present the same GPU type to a sandbox, so which one a sandbox lands on isn’t something you select. Leave the type key out of the GPU request so the platform assigns whichever GPU type the serverless pool runs. The field is still accepted here, but it filters rather than selects: a type the pool doesn’t have matches no runner and fails with CWSANDBOX_RUNNER_UNAVAILABLE, which reads like a capacity error rather than a configuration error. Set CPU and memory limits equal to requests. The flat resources form in the following example does that for you, and a runner can be configured to reject a sandbox whose limits exceed its requests rather than trim them. Disk is requested separately from CPU, memory, and GPU rather than alongside them: ResourceOptions has no disk field. The container’s root filesystem is Node-local ephemeral storage that the sandbox doesn’t reserve a share of, so df inside the sandbox reports the Node’s filesystem rather than a per-sandbox quota. For a dedicated writable path, declare a scratch volume:

from cwsandbox import AuthStrategy, Sandbox, ScratchVolumeOptions

with Sandbox.run(
    auth=AuthStrategy.WANDB,
    resources={"cpu": "2", "memory": "8Gi", "gpu": 1},
    volumes=[ScratchVolumeOptions(name="work", mount_path="/work", size="20Gi")],
    max_lifetime_seconds=3600,
) as sandbox:
    result = sandbox.exec(["df", "-h", "/work"]).result()
    print(result.stdout)

A disk-backed volume, the default, draws on the same Node-local storage as the root filesystem. Set medium="memory" for a tmpfs instead: a memory-backed volume must declare a size, and the memory-backed volumes on one container can’t total more than 80% of its memory request. Leave runtime_class unset too. A GPU request selects the GPU runtime class on its own, and a runtime class you pin is used exactly as given, so pinning the CPU class alongside a GPU request produces a sandbox that can’t reach the GPU.

Create the sandbox

Set your credential as described in Choose a credential, then run the following example. It creates a sandbox with 1 GPU, 2 CPUs, and 8 GiB of memory, then prints the GPU that nvidia-smi reports.

Python
TypeScript

Use AuthStrategy.WANDB for a W&B API key. For a CoreWeave access token, replace it with AuthStrategy.COREWEAVE_API_KEY, which reads CWSANDBOX_API_KEY.

from cwsandbox import AuthStrategy, Sandbox

with Sandbox.run(
    auth=AuthStrategy.WANDB,
    resources={"cpu": "2", "memory": "8Gi", "gpu": 1},
    max_lifetime_seconds=3600,
) as sandbox:
    result = sandbox.exec(
        ["nvidia-smi", "--query-gpu=name,memory.total", "--format=csv"]
    ).result()
    print(result.stdout)
    print(sandbox.resource_gpu)

The flat resources dict sets requests and limits to the same values. "gpu": 1 is shorthand for "gpu": {"count": 1}. To request more GPUs, raise the count.The resource_gpu property returns the GPU allocation the platform confirmed, such as {'count': 1}.

The @coreweave/cwsandbox package doesn’t accept GPU resources yet. Its resources option covers CPU and memory only, and a gpu key is ignored, so the sandbox starts without a GPU. Use the Python client to create GPU sandboxes.

Sample output:

name, memory.total [MiB]
NVIDIA RTX PRO 6000 Blackwell Server Edition, 97887 MiB

{'count': 1}

GPU sandboxes take longer to start than CPU-only sandboxes because the platform attaches the GPUs to the sandbox’s virtual machine. Allow several minutes if you set a request timeout.

Run a GPU sandbox on your CKS cluster

On CKS placement, your administrators decide which GPUs sandboxes can use and how many. The runner offers the GPU types present on the cluster’s Nodes, and the policy it carries bounds what a sandbox may request.

Prerequisites

Complete the following before you request a GPU on CKS:

A CKS cluster with GPU Nodes and a runner in the Ready state. See Deploy sandboxes on your own CKS cluster.
The cw-kata-nvidia-gpu runtime class installed on the cluster, and GPU Nodes configured for virtualization. CoreWeave manages both. To have GPU sandbox support enabled on your cluster, contact CoreWeave support or your account team and name the cluster and the NodePool you want it on. Confirm the runtime class is present before you write a policy:
kubectl get runtimeclass
A policy on that runner that permits GPUs. Set maxGpuCount under resources, include cw-kata-nvidia-gpu in allowedRuntimeClasses, and set defaultGpuRuntimeClass to cw-kata-nvidia-gpu under security. The GPU data science cluster example policy shows all three. For every field, see Configure a sandbox policy.
The SANDBOX_USER IAM action and a CoreWeave API access token, set as CWSANDBOX_API_KEY.

Find the GPU types on your runner

Each runner advertises the GPU types its Nodes carry. List them before you choose a type:

from cwsandbox import list_runners

for runner in list_runners():
    print(runner.runner_id, runner.supported_gpu_types, runner.max_gpu_count)

The values are the exact strings to pass as type. Matching is case-sensitive, so B200 and b200 aren’t the same type. To filter runners by GPU type instead, pass gpu_type to list_runners(). For more, see Discover runners.

Create the sandbox

The following example places a sandbox on CKS with 2 GPUs of a specific type, 8 CPUs, and 32 GiB of memory. Replace [GPU-TYPE] with one of the types your runner advertises, or drop the type key to accept any GPU on the runner.

from cwsandbox import ResourceOptions, Sandbox

with Sandbox.run(
    placement_mode="cks",
    resources=ResourceOptions(
        requests={"cpu": "8", "memory": "32Gi"},
        limits={"cpu": "8", "memory": "32Gi"},
        gpu={"count": 2, "type": "[GPU-TYPE]"},
    ),
    max_lifetime_seconds=3600,
) as sandbox:
    result = sandbox.exec(["nvidia-smi", "-L"]).result()
    print(result.stdout)
    print(sandbox.resource_gpu)

To target one runner rather than any CKS runner in your organization, add runner_ids=["[RUNNER-ID]"]. The confirmed allocation in resource_gpu reports the count only, such as {'count': 2}, even when you requested a type. The GPU count is capped by the policy’s maxGpuCount and by the number of free GPUs on a single Node. A request above the policy cap fails with CWSANDBOX_RESOURCE_CEILING_EXCEEDED. A type the runner doesn’t offer, or one with a different case, finds no eligible runner and fails with CWSANDBOX_RUNNER_UNAVAILABLE.

Container images

The platform provides the NVIDIA driver and the nvidia-smi tool inside a GPU sandbox, so the default image can already see the GPU. To run CUDA applications, use an image that ships the CUDA runtime and libraries your code needs, such as a pytorch/pytorch or nvidia/cuda image. Match the image to the GPU. The RTX PRO 6000 Blackwell Server Edition is compute capability 12.0 (sm_120), which needs CUDA 12.8 or later, and PyTorch 2.7 was the first release built for it. An older image still reports the GPU’s name correctly, because that reads device metadata through the driver, then fails at the first kernel launch with CUDA error: no kernel image is available for execution on the device. Check that your framework lists sm_120 rather than trusting the device name:

from cwsandbox import AuthStrategy, Sandbox

CHECK_GPU = """
import torch

print(torch.cuda.get_device_capability(0))
print(torch.cuda.get_arch_list())

x = torch.ones(32, device="cuda")
print((x + x).sum().item())
torch.cuda.synchronize()
"""

with Sandbox.run(
    auth=AuthStrategy.WANDB,
    container_image="pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtime",
    resources={"cpu": "2", "memory": "8Gi", "gpu": 1},
) as sandbox:
    result = sandbox.exec(["python", "-c", CHECK_GPU]).result()
    print(result.stdout)

Sample output:

(12, 0)
['sm_70', 'sm_75', 'sm_80', 'sm_86', 'sm_90', 'sm_100', 'sm_120']
64.0

A large framework image takes longer to pull than the default image, so allow a few minutes for the sandbox to become ready.

Common errors

Error	Cause	What to do
`CWSANDBOX_GPU_NOT_ALLOWED` (`this organization is not entitled to create GPU sandboxes`)	GPU sandboxes aren’t enabled for your organization.	Contact your account team or support@wandb.ai to request access.
`SandboxResourceExhaustedError` (`runner capacity exhausted`)	No Node has enough free GPUs, CPU, or memory for the request right now.	Retry after a short wait, or request fewer GPUs.
`CWSANDBOX_RUNNER_UNAVAILABLE` (`no eligible runner is available`)	No runner matches the request. One cause is a GPU `type` that no runner offers, including a type with the wrong case. The same code also appears when no eligible runner is connected right now.	On serverless, remove `type`. On CKS, use a type from `supported_gpu_types`, or drop `type`. If `type` is already correct, retry.
`CWSANDBOX_RESOURCE_CEILING_EXCEEDED` (`gpu count exceeds the policy maximum`)	The request asks for more GPUs than the policy’s `maxGpuCount`.	Lower the count, or ask an administrator to raise the cap.
`nvidia-smi: not found`	The sandbox has no GPU, or its image doesn’t expose the NVIDIA tools.	Check `sandbox.resource_gpu` first. If it reports a positive count, the GPUs are allocated and the image is what to change.

Next steps

Sandbox configuration covers every ResourceOptions field, QoS classes, and timeouts.
Configure a sandbox policy explains the GPU, runtime class, and resource constraints administrators set on a CKS runner.
Get started with CoreWeave sandboxes covers CPU-only sandboxes, credentials, and deploying a runner on your own cluster.

​Before you begin

​How GPU requests work

​Run a GPU sandbox on serverless capacity

​Available GPUs

​Create the sandbox

​Run a GPU sandbox on your CKS cluster

​Prerequisites

​Find the GPU types on your runner

​Create the sandbox

​Container images

​Common errors

​Next steps

Before you begin

How GPU requests work

Run a GPU sandbox on serverless capacity

Available GPUs

Create the sandbox

Run a GPU sandbox on your CKS cluster

Prerequisites

Find the GPU types on your runner

Create the sandbox

Container images

Common errors

Next steps