Run a GPU sandbox - CoreWeave Docs

This guide shows how to create a sandbox with one or more GPUs, confirm the GPU is visible from inside the sandbox, and set the CPU and memory requests that go with it. GPU sandboxes run in either placement mode: on serverless capacity that CoreWeave operates, or on a CoreWeave Kubernetes Service (CKS) cluster you own. The two modes differ in how the sandbox is isolated, which GPUs you can get, and who sets the limits, so this page covers them in separate sections. For CPU-only sandboxes, see Get started.

GPU sandboxes are in private preview and require your organization to be allowlisted, even if it already has access to CPU-only sandboxes in public preview. To request access, contact your account team or email forge-support@coreweave.com. Until your organization is allowlisted, requests to create GPU sandboxes fail with CWSANDBOX_GPU_NOT_ALLOWED.

For concurrent sandbox quotas and resource limits, see Limits and quotas.

Before you begin

You need the following:

GPU sandboxes enabled for your organization.
A W&B API key.
The Python client, cwsandbox 1.14.2 or later, with the wandb extra:
uv pip install 'cwsandbox[wandb]>=1.14.2'

The TypeScript client doesn’t expose GPU resources yet. Use the Python client for GPU sandboxes.

How GPU requests work

GPU requests follow these rules:

A GPU request reserves GPUs and nothing else. Set CPU and memory explicitly, sized for the work you’ll run alongside the GPU. Omitted CPU and memory values can use the policy’s defaultCpu and defaultMemory, but the resolved allocation must still satisfy the policy’s resource requirements.
Sandboxes get whole GPUs. GPUs aren’t shared, partitioned, or time-sliced between sandboxes.
A sandbox gets exactly 1 GPU. Serverless capacity supports only 1-GPU sandboxes.
The GPU type is a filter, not a menu. The optional type key must match exactly, including case, one of the GPU types the runner advertises. Omit it to accept any GPU the runner has.

Run a GPU sandbox on serverless capacity

Serverless placement needs no runner and no policy of your own. CoreWeave owns the policy and the hardware. GPU sandboxes run in a virtual machine, so they suit untrusted code.

Available GPUs

Serverless capacity runs one GPU model, the NVIDIA RTX PRO 6000 Blackwell Server Edition. Only 1-GPU sandboxes are supported on serverless capacity.

GPU	GPU memory	GPUs per sandbox
NVIDIA RTX PRO 6000 Blackwell Server Edition	96 GB	1

Two instance types carry it: High Memory and Standard Memory. They differ in host RAM, not in the GPU, and both present the same GPU type to a sandbox, so which one a sandbox lands on isn’t something you select. Leave the type key out of the GPU request so the platform assigns a GPU from CoreWeave-managed compute. The field is still accepted here, but it filters rather than selects: a type that matches no runner fails with CWSANDBOX_RUNNER_UNAVAILABLE. A runner that receives an unsupported type rejects it with CWSANDBOX_PLACEMENT_CONSTRAINT_UNSATISFIED. Use resources to choose CPU and memory alongside the GPU count, as shown in the following example. Disk is requested separately from CPU, memory, and GPU rather than alongside them: ResourceOptions has no disk field. The container’s root filesystem is Node-local ephemeral storage that the sandbox doesn’t reserve a share of, so df inside the sandbox reports the Node’s filesystem rather than a per-sandbox quota. For a dedicated writable path, declare a scratch volume:

from cwsandbox import AuthStrategy, Sandbox, ScratchVolumeOptions

with Sandbox.run(
    auth=AuthStrategy.WANDB,
    resources={"cpu": "2", "memory": "8Gi", "gpu": 1},
    volumes=[ScratchVolumeOptions(name="work", mount_path="/work", size="20Gi")],
    max_lifetime_seconds=3600,
) as sandbox:
    result = sandbox.exec(["df", "-h", "/work"]).result()
    print(result.stdout)

A disk-backed volume, the default, draws on the same Node-local storage as the root filesystem. Set medium="memory" for a tmpfs instead: a memory-backed volume must declare a size, and the memory-backed volumes on one container can’t total more than 80% of its memory request. Leave runtime_class unset too. A GPU request selects the GPU virtual machine runtime class on its own, and a runtime class you pin is used exactly as given, so pinning the CPU class alongside a GPU request produces a sandbox that can’t reach the GPU.

Create the sandbox

Set WANDB_API_KEY to your W&B API key, then run the following example. It creates a sandbox with 1 GPU, 2 CPUs, and 8 GiB of memory, then prints the GPU that nvidia-smi reports.

Python
TypeScript

This example uses AuthStrategy.WANDB for a W&B API key.

from cwsandbox import AuthStrategy, Sandbox

with Sandbox.run(
    auth=AuthStrategy.WANDB,
    resources={"cpu": "2", "memory": "8Gi", "gpu": 1},
    max_lifetime_seconds=3600,
) as sandbox:
    result = sandbox.exec(
        ["nvidia-smi", "--query-gpu=name,memory.total", "--format=csv"]
    ).result()
    print(result.stdout)
    print(sandbox.resource_gpu)

The flat resources dict sets requests and limits to the same values. "gpu": 1 is shorthand for "gpu": {"count": 1}. Serverless supports only 1 GPU per sandbox, so keep the count at 1.The resource_gpu property returns the GPU allocation the platform confirmed, such as {'count': 1}.

The @coreweave/cwsandbox package doesn’t accept GPU resources yet. Its resources option covers CPU and memory only, and a gpu key is ignored, so the sandbox starts without a GPU. Use the Python client to create GPU sandboxes.

Sample output:

name, memory.total [MiB]
NVIDIA RTX PRO 6000 Blackwell Server Edition, 97887 MiB

{'count': 1}

GPU sandboxes take longer to start than CPU-only sandboxes because the platform attaches the GPUs to the sandbox’s virtual machine. Allow several minutes if you set a request timeout.

Container images

The platform provides the NVIDIA driver and the nvidia-smi tool inside a GPU sandbox, so the default image can already see the GPU. To run CUDA applications, use an image that ships the CUDA runtime and libraries your code needs, such as a pytorch/pytorch or nvidia/cuda image. Match the image to the GPU. The RTX PRO 6000 Blackwell Server Edition is compute capability 12.0 (sm_120), which needs CUDA 12.8 or later, and PyTorch 2.7 was the first release built for it. An older image still reports the GPU’s name correctly, because that reads device metadata through the driver, then fails at the first kernel launch with CUDA error: no kernel image is available for execution on the device. Check that your framework lists sm_120 rather than trusting the device name:

from cwsandbox import AuthStrategy, Sandbox

CHECK_GPU = """
import torch

print(torch.cuda.get_device_capability(0))
print(torch.cuda.get_arch_list())

x = torch.ones(32, device="cuda")
print((x + x).sum().item())
torch.cuda.synchronize()
"""

with Sandbox.run(
    auth=AuthStrategy.WANDB,
    container_image="pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtime",
    resources={"cpu": "2", "memory": "8Gi", "gpu": 1},
) as sandbox:
    result = sandbox.exec(["python", "-c", CHECK_GPU]).result()
    print(result.stdout)

Sample output:

(12, 0)
['sm_70', 'sm_75', 'sm_80', 'sm_86', 'sm_90', 'sm_100', 'sm_120']
64.0

A large framework image takes longer to pull than the default image, so allow a few minutes for the sandbox to become ready.

Common errors

Error	Cause	What to do
`CWSANDBOX_GPU_NOT_ALLOWED`	Your organization is not allowlisted for GPU sandboxes.	Contact your account team or forge-support@coreweave.com to request access.
`SandboxResourceExhaustedError` (`runner capacity exhausted`)	No Node has enough free GPUs, CPU, or memory for the request right now.	Retry after a short wait.
`CWSANDBOX_RUNNER_UNAVAILABLE` (`no eligible runner is available`)	No capacity matches the request. One cause is a GPU `type` that serverless capacity doesn’t offer, including a type with the wrong case. The same code also appears when no capacity is connected right now.	Remove `type`. If the request is already correct, retry.
`nvidia-smi: not found`	The sandbox has no GPU, or its image doesn’t expose the NVIDIA tools.	Check `sandbox.resource_gpu` first. If it reports a positive count, the GPUs are allocated and the image is what to change.

Next steps

Sandbox configuration covers every ResourceOptions field, QoS classes, and timeouts.
Get started covers CPU-only serverless sandboxes and credentials.

​Before you begin

​How GPU requests work

​Run a GPU sandbox on serverless capacity

​Available GPUs

​Create the sandbox

​Container images

​Common errors

​Next steps

Before you begin

How GPU requests work

Run a GPU sandbox on serverless capacity

Available GPUs

Create the sandbox

Container images

Common errors

Next steps