웜 풀 구현하기 - CoreWeave Docs

Use a warm pool when requests need a running sandbox without waiting for one to start. The warm pool recipe maintains a buffer of prepared sandboxes, assigns each workload its own sandbox, and replenishes the buffer in the background. Idle sandboxes consume compute while waiting for work.

How the pool works

The recipe’s WarmPool helper starts the configured number of sandboxes and runs a readiness check in each. Their main processes stay running between commands. No periodic ping is needed. Each workload claims one ready sandbox, triggering preparation of a replacement. Every exec() within that claim shares the same sandbox and files. When the workload finishes or fails, the helper stops its sandbox. Subsequent workloads receive separate sandboxes, so they don’t inherit previous workloads’ local state.

Run the recipe

You need Python 3.11+, uv, and serverless access and credentials.

Download or clone the recipe repository and open recipes/warm-pool. Install dependencies and create your environment file:
uv sync --frozen cp .env.example .env

demo.py 상단 부근에서 AUTH를 AuthStrategy.WANDB로 설정하세요. W&B API 키를 .env에 WANDB_API_KEY로 추가하거나, 기존 W&B 로그인을 사용하세요.
W&B 인증으로 비교를 실행하세요. W&B 인증을 사용하려면 해당 선택 종속성이 필요합니다.
uv run --frozen --extra wandb --env-file .env demo.py

The demo compares four requests using on-demand creation with four using the pool. It reports initial pool preparation separately from request timings and checks that every workload receives a distinct sandbox. Expect this final message:

Verified 8 distinct workload sandboxes; all sandboxes stopped.

Adapt the pool to your workload

Edit the constants in demo.py:

POOL_SIZE = 2     # Target number of idle sandboxes.
CONCURRENCY = 2   # Maximum active workloads.
REQUESTS = 4      # Requests per comparison mode.

These defaults allow up to four sandboxes at once: two buffer sandboxes in addition to two active workloads. Each sandbox requests 2 vCPU and 4 GiB RAM. If the buffer empties, requests wait for replenishment. If the active limit is reached, they wait for a workload to finish. Measure with your own arrival pattern before increasing the buffer. Replace WORKLOAD in demo.py with your code. Add shared setup to prepare() in warm_pool.py and use a readiness check appropriate for your application. Within an asynchronous workload handler, use the recipe’s claim context for multiple commands:

async with pool.claim() as sandbox:
    await sandbox.exec(["sh", "-c", "echo hello > /tmp/result.txt"], check=True)
    result = await sandbox.exec(["cat", "/tmp/result.txt"], check=True)
    print(result.stdout)

Collect outputs before leaving the claim. Supply customer inputs and credentials after claiming, and scope shared external storage to the appropriate customer.

Keep the pool running

Keep the pool and enclosing Session contexts open while accepting requests. Each Python process owns a separate pool. The demo closes both after its comparison. The helper replaces idle sandboxes at 5 minutes. Each sandbox has a 10-minute lifetime. The pool continues replenishing while its context is open. Keep jobs comfortably under 5 minutes with these defaults: claiming a sandbox doesn’t reset its deadline. For longer-job settings and cleanup after interrupted runs, see the recipe. Transient preparation failures get up to three attempts. If those attempts are exhausted or preparation encounters a non-retryable error, all further claims fail, even if ready sandboxes remain. Exit the pool and Session contexts to clean up, then create a new pool when the underlying issue is resolved. 보충 시간을 줄이려면 자주 변경되지 않는 의존성을 이미지에 미리 포함하여 구축하거나, 파일 시스템 스냅샷에서 미리 준비해 둔 워크스페이스 파일을 복원하세요. 스냅샷은 설정된 마운트의 파일만 캡처하며, 애플리케이션 프로세스는 여전히 별도로 시작해야 합니다.

​How the pool works

​Run the recipe

​Adapt the pool to your workload

​Keep the pool running

How the pool works

Run the recipe

Adapt the pool to your workload

Keep the pool running