Implement a warm pool - CoreWeave Docs

Use a warm pool when requests need a running sandbox without waiting for one to start. The warm pool recipe maintains a buffer of prepared sandboxes, assigns each workload its own sandbox, and replenishes the buffer in the background. Idle sandboxes consume compute while waiting for work.

How the pool works

The recipe’s WarmPool helper starts the configured number of sandboxes and runs a readiness check in each. Their main processes stay running between commands. No periodic ping is needed. Each workload claims one ready sandbox, triggering preparation of a replacement. Every exec() within that claim shares the same sandbox and files. When the workload finishes or fails, the helper stops its sandbox. Subsequent workloads receive separate sandboxes, so they don’t inherit previous workloads’ local state.

Run the recipe

You need Python 3.11+, uv, and serverless access and credentials.

Download or clone the recipe repository and open recipes/warm-pool. Install dependencies and create your environment file:
uv sync --frozen cp .env.example .env
Near the top of demo.py, select the matching AUTH setting. Add your key to .env, or use an existing W&B login:
Credential Credential source AUTH setting
CoreWeave API access token CWSANDBOX_API_KEY AuthStrategy.COREWEAVE_API_KEY (default)
W&B API key WANDB_API_KEY or an existing W&B login AuthStrategy.WANDB

Credential	Credential source	`AUTH` setting
CoreWeave API access token	`CWSANDBOX_API_KEY`	`AuthStrategy.COREWEAVE_API_KEY` (default)
W&B API key	`WANDB_API_KEY` or an existing W&B login	`AuthStrategy.WANDB`

Run the comparison with CoreWeave authentication:

uv run --frozen --env-file .env demo.py

For W&B authentication, include its optional dependency:

uv run --frozen --extra wandb --env-file .env demo.py

The demo compares four requests using on-demand creation with four using the pool. It reports initial pool preparation separately from request timings and checks that every workload receives a distinct sandbox. Expect this final message:

Verified 8 distinct workload sandboxes; all sandboxes stopped.

Adapt the pool to your workload

Edit the constants in demo.py:

POOL_SIZE = 2     # Target number of idle sandboxes.
CONCURRENCY = 2   # Maximum active workloads.
REQUESTS = 4      # Requests per comparison mode.

These defaults allow up to four sandboxes at once: two buffer sandboxes in addition to two active workloads. Each sandbox requests 2 vCPU and 4 GiB RAM. If the buffer empties, requests wait for replenishment. If the active limit is reached, they wait for a workload to finish. Measure with your own arrival pattern before increasing the buffer. Replace WORKLOAD in demo.py with your code. Add shared setup to prepare() in warm_pool.py and use a readiness check appropriate for your application. Within an asynchronous workload handler, use the recipe’s claim context for multiple commands:

async with pool.claim() as sandbox:
    await sandbox.exec(["sh", "-c", "echo hello > /tmp/result.txt"], check=True)
    result = await sandbox.exec(["cat", "/tmp/result.txt"], check=True)
    print(result.stdout)

Collect outputs before leaving the claim. Supply customer inputs and credentials after claiming, and scope shared external storage to the appropriate customer.

Keep the pool running

Keep the pool and enclosing Session contexts open while accepting requests. Each Python process owns a separate pool. The demo closes both after its comparison. The helper replaces idle sandboxes at 5 minutes. Each sandbox has a 10-minute lifetime. The pool continues replenishing while its context is open. Keep jobs comfortably under 5 minutes with these defaults: claiming a sandbox doesn’t reset its deadline. For longer-job settings and cleanup after interrupted runs, see the recipe. Transient preparation failures get up to three attempts. If those attempts are exhausted or preparation encounters a non-retryable error, all further claims fail, even if ready sandboxes remain. Exit the pool and Session contexts to clean up, then create a new pool when the underlying issue is resolved. To shorten replenishment, build stable dependencies into your image or restore prepared workspace files from a file system snapshot. Snapshots capture files in the configured mount. Application processes still start separately. A sandbox template can store the shared image and snapshot configuration.

​How the pool works

​Run the recipe

​Adapt the pool to your workload

​Keep the pool running

How the pool works

Run the recipe

Adapt the pool to your workload

Keep the pool running