Implémenter un pool préchauffé

Use a warm pool when requests need a running sandbox without waiting for one to start. The warm pool recipe maintains a buffer of prepared sandboxes, assigns each workload its own sandbox, and replenishes the buffer in the background. Idle sandboxes consume compute while waiting for work.

How the pool works

The recipe’s WarmPool helper starts the configured number of sandboxes and runs a readiness check in each. Their main processes stay running between commands. No periodic ping is needed. Each workload claims one ready sandbox, triggering preparation of a replacement. Every exec() within that claim shares the same sandbox and files. When the workload finishes or fails, the helper stops its sandbox. Subsequent workloads receive separate sandboxes, so they don’t inherit previous workloads’ local state.

Run the recipe

You need Python 3.11+, uv, and serverless access and credentials.

Download or clone the recipe repository and open recipes/warm-pool. Install dependencies and create your environment file:
uv sync --frozen cp .env.example .env

Vers le début de demo.py, définissez AUTH sur AuthStrategy.WANDB. Ajoutez votre clé API W&B dans le fichier .env en tant que WANDB_API_KEY, ou utilisez une connexion W&B existante.
Exécutez la comparaison avec l’authentification W&B, qui requiert sa dépendance facultative :
uv run --frozen --extra wandb --env-file .env demo.py

The demo compares four requests using on-demand creation with four using the pool. It reports initial pool preparation separately from request timings and checks that every workload receives a distinct sandbox. Expect this final message:

Verified 8 distinct workload sandboxes; all sandboxes stopped.

Adapt the pool to your workload

Edit the constants in demo.py:

POOL_SIZE = 2     # Target number of idle sandboxes.
CONCURRENCY = 2   # Maximum active workloads.
REQUESTS = 4      # Requests per comparison mode.

These defaults allow up to four sandboxes at once: two buffer sandboxes in addition to two active workloads. Each sandbox requests 2 vCPU and 4 GiB RAM. If the buffer empties, requests wait for replenishment. If the active limit is reached, they wait for a workload to finish. Measure with your own arrival pattern before increasing the buffer. Replace WORKLOAD in demo.py with your code. Add shared setup to prepare() in warm_pool.py and use a readiness check appropriate for your application. Within an asynchronous workload handler, use the recipe’s claim context for multiple commands:

async with pool.claim() as sandbox:
    await sandbox.exec(["sh", "-c", "echo hello > /tmp/result.txt"], check=True)
    result = await sandbox.exec(["cat", "/tmp/result.txt"], check=True)
    print(result.stdout)

Collect outputs before leaving the claim. Supply customer inputs and credentials after claiming, and scope shared external storage to the appropriate customer.

Keep the pool running

Keep the pool and enclosing Session contexts open while accepting requests. Each Python process owns a separate pool. The demo closes both after its comparison. The helper replaces idle sandboxes at 5 minutes. Each sandbox has a 10-minute lifetime. The pool continues replenishing while its context is open. Keep jobs comfortably under 5 minutes with these defaults: claiming a sandbox doesn’t reset its deadline. For longer-job settings and cleanup after interrupted runs, see the recipe. Transient preparation failures get up to three attempts. If those attempts are exhausted or preparation encounters a non-retryable error, all further claims fail, even if ready sandboxes remain. Exit the pool and Session contexts to clean up, then create a new pool when the underlying issue is resolved. Pour accélérer le réapprovisionnement, intégrez les dépendances stables directement dans votre image ou restaurez les fichiers préparés de l’espace de travail à partir d’un instantané du système de fichiers. Les instantanés capturent les fichiers présents dans le point de montage configuré. Les processus applicatifs doivent néanmoins toujours être démarrés séparément.

​How the pool works

​Run the recipe

​Adapt the pool to your workload

​Keep the pool running

How the pool works

Run the recipe

Adapt the pool to your workload

Keep the pool running