Choose authentication
Choose the credential for the account that owns your sandboxes. To obtain a Forge API key or CoreWeave API access token, see Choose and get credentials. Run the installation commands in an active Python virtual environment. Save the Python code from your credential tab as asandbox_auth.py file beside the Python script that uses it. This file stores only the authentication choice. Import AUTH from that file. When you create a sandbox or session, pass AUTH explicitly or set it in SandboxDefaults.
- Forge API key
- CoreWeave API access token
Use your Forge API key with the
wandb integration in the software development kit (SDK). Replace [WANDB-API-KEY] with your API key:sandbox_auth.py
placement_mode="cks" when you create a sandbox or in SandboxDefaults.
To authenticate a session and the sandboxes it owns, create the session with auth=AUTH. When you reconnect, list, or clean up sandboxes, keep the same credential.
Lifecycle states
Every sandbox passes through a series of states as it starts, runs, and shuts down. The SDK represents these asSandboxStatus values.
The main-process exit and restart paths apply to ordinary command-based sandboxes and platform-controlled Compose sandboxes. For the Compose engine-mode exception, see Main command lifetime.
PENDING and CREATING are transient. The SDK polls through them automatically.
TERMINATING is a transient backend state that wait_until_complete() and stop().result()
drive through to a terminal state. RUNNING is the stable operational state.
COMPLETED and FAILED are terminal. UNSPECIFIED is mapped to COMPLETED by the SDK at
poll time.
You can start a workflow when a sandbox is created, completes, or fails. Create a sandbox status change automation in the sandbox’s Weights & Biases project. For example, post to a Slack channel or call a webhook.
Creation patterns
You can create a sandbox in two ways, differing in when the start RPC fires. The following sections describe each pattern and when to choose it.Sandbox.run(): immediate start
Sandbox.run() creates a sandbox and calls start().result() internally, blocking until the
backend accepts the request:
Sandbox.run() takes the command and its arguments as variadic positional strings. It does not accept command or args as keyword arguments.
session.sandbox(): deferred start
session.sandbox() creates a sandbox object without making any network call. The start RPC
fires on first use:
Main command lifetime
For an ordinary sandbox created withSandbox.run() or session.sandbox(), the command is
the sandbox’s main process. What happens when it exits depends on its exit status:
- Exit status 0: The sandbox transitions to
COMPLETED. - Nonzero exit status: The platform can restart the main process’s container. A sandbox
that has already reached
RUNNINGstaysRUNNINGduring retries, and one that hasn’t staysCREATING.wait_until_complete()keeps waiting through retries, up to itstimeout. - Retries exhausted: The sandbox transitions to
FAILEDwhen the backend exhausts its allowed retries. A runtime error or maximum-lifetime expiry can fail the sandbox sooner.
CONTROL_CONTAINER_ENGINE) have a different lifecycle. The
platform doesn’t restart the engine or retry the sandbox after an engine failure. The sandbox
transitions to FAILED. When a Compose service exits, that alone doesn’t end the sandbox.
Platform-controlled Compose sandboxes (CONTROL_PLATFORM, the default) follow the ordinary
main-command behavior for their primary container. See Compose support.
For an ordinary sandbox, a main process that exits successfully ends the sandbox:
exec() on a long-running
sandbox rather than making the short command the main process.
Start a sandbox
After a sandbox object exists, you can start it explicitly or rely on the SDK to start it on first use. The following sections describe both approaches and how context managers interact with them.Explicit start
start() sends the start RPC and returns an OperationRef[None]:
Auto-start
Most operations auto-start the sandbox if it hasn’t been started yet. In the common case, you create a sandbox and start using it.Context managers and start
Context managers (with/async with) call start() on entry but do not call wait().
To inspect the startup status before your first operation, call wait() and check status:
exec() waits for RUNNING internally. Explicit
wait() is useful when you want to separate startup failures from operation failures.
Wait for a sandbox
The SDK provides waiting methods to inspect startup status or wait for the main command to finish. The following sections describe each method and how startup and operation timeouts relate.wait(): wait for startup status
wait() polls through startup states such as PENDING and CREATING. For example, it can return when the sandbox is
RUNNING or TERMINATING, or after it completes successfully:
wait() returns the sandbox object. Check its status before calling an operation that
requires a running sandbox. A normal return doesn’t guarantee RUNNING.
If the sandbox reaches a terminal state during startup, wait() handles it:
wait_until_complete(): block until terminal
wait_until_complete() blocks until the sandbox reaches a terminal state, polling through
TERMINATING automatically. Use this for sandboxes where the main command does the work:
raise_on_termination parameter, True by default, controls whether
wait_until_complete() raises SandboxTerminatedError after this client called stop()
or a legacy backend reports TERMINATED. External stops reported as COMPLETED
do not raise this error, because the response does not distinguish them from normal completion. A sandbox that reaches its maximum lifetime ends
FAILED instead, so wait_until_complete() raises SandboxFailedError.
Timeout phases
Startup wait time and operation timeouts are separate phases, and the sandbox’s maximum lifetime bounds both:- Startup wait: Time spent waiting for startup or an early terminal state. Controlled by the
timeoutparameter onwait(). - Completion wait: Time spent waiting for a terminal state, including any remaining startup,
execution, restarts, and termination. Controlled by the
timeoutparameter onwait_until_complete(). - Operation timeout: Time for an individual exec, read, or write. Controlled by
timeout_secondsonexec(), orrequest_timeout_secondsinSandboxDefaults. Doesn’t include startup wait. - Maximum lifetime: Total wall-clock time the sandbox can exist, counted from start and
including the startup wait. Controlled by
max_lifetime_seconds. If you don’t set it, the runner-policy default applies, with a 10-minute platform fallback when no positive default is set.
max_lifetime_seconds to comfortably exceed the total work you expect. You can’t change it
after the sandbox starts. For the timeout settings and how they interact, see
Timeouts. For the 30-day limit and
configuration examples for Serverless and CKS, see Run long-running sandboxes.
Operations and lifecycle
Operations likeexec(), read_file(), and write_file() auto-start the sandbox if needed,
then wait for RUNNING before proceeding:
timeout_seconds) applies only after the sandbox is RUNNING. Startup
time is not counted against it.
Stop a sandbox and end of life
This section covers how to shut down a sandbox, what happens after a stop, and when to usestop() versus delete().
stop()
stop() sends a stop request and returns OperationRef[None]. The sandbox transitions through
TERMINATING (grace period draining) before reaching a terminal state. The returned
OperationRef resolves when the backend confirms the terminal state, not only when the stop
RPC succeeds:
Snapshot-on-stop requires a sandbox started with a
file_system_snapshot mount and an
organization enabled for File System Snapshots (FSS). It captures that mount, not the entire
sandbox filesystem. See File system snapshots
for setup.
Choose one stop option for a fresh running sandbox. These options are alternatives, and each
ends that sandbox. For complete examples that create separate sandboxes, see
Manual cleanup.
- Plain stop
- Snapshot on stop
Set a longer grace period and tolerate a sandbox that is already gone:
start().result() to finish.
Don’t rely on stop() to wait for a pending creation request. If the client hasn’t received a
sandbox ID, stopping can return before that request finishes. If a creation response is lost
or a stop fails, a sandbox can remain running. See
Cleanup patterns for recovery options.
Concurrent plain stop() calls share one stop operation; repeated
plain stops do not issue duplicate stop RPCs. A snapshot_on_stop=True call raises
SnapshotOnStopConflictError if it would join or observe a stop that is not capturing a
snapshot. Request snapshot-on-stop before a plain stop begins.
Post-stop behavior
When the SDK observes completion through polling, a stopped sandbox can report its main process’s real exit code throughreturncode. Typical signal exit codes are 143 for SIGTERM and 137 for
SIGKILL. A process that handles SIGTERM can exit with its own code, including 0.
returncode can still be None with older backends that lack exit-code support.
It can also be None if the container never started or if the SDK observed terminal status
before the runner’s exit-code report arrived.
Status-only lookups through get_status(), list(), or from_id() don’t populate returncode.
After stop() is called, the sandbox transitions through TERMINATING (the grace period
draining state) and then reaches a terminal state (COMPLETED or FAILED). After stop() has
been called, the sandbox is unusable. Further operations raise SandboxNotRunningError:
status property is cached from the last API call. For fresh data before stopping, use
get_status():
Context manager exit
Context managers callstop() automatically on exit:
stop() compared to delete()
Use
stop() for a running sandbox, including one reconnected with Sandbox.from_id().
Use Sandbox.delete() for orphan cleanup by ID:
Under the hood
The SDK runs all gRPC operations on a background daemon thread with its own asyncio event loop. This design means:- The sync API (
.result()) blocks the calling thread while the background loop handles the network call. - The async API (
await) bridges to the same background loop, so both patterns use the same underlying implementation. - The background loop starts lazily on first use. gRPC channels are also created lazily.
- Auto-start works by checking if
sandbox_idis None before each operation and triggeringstart()if so. - On process exit, cleanup handlers (atexit + signal handlers) stop all sandboxes in registered sessions. A second Ctrl+C during cleanup forces immediate exit.
nest_asyncio. See the Sync compared to async patterns guide for usage patterns.
Common patterns
The following examples show end-to-end patterns that combine the lifecycle steps described above.Quick one-off
Run a command and get the result. Context manager handles cleanup:Controlled startup
Separate start errors from operation errors:Long-running sandbox
Run training withexec() in a keep-alive sandbox so you can retrieve the output before stopping the sandbox. Completing the training command doesn’t complete the sandbox.
Replace [TRAINING-IMAGE] with a container image that includes Python, /app/train.py, and its dependencies. The training script must write its model to /output/model.pt. Adjust the resources and timeouts for your workload.
Sandbox.run() without a command uses a keep-alive main process. The exec().result() call waits for training to finish, and check=True raises an exception if training exits with a nonzero code. The example saves the artifact to model.pt on your local machine while the sandbox remains running.
When the block exits, the context manager calls stop() unless the sandbox is already terminal. This cleanup also runs when training or artifact retrieval raises an exception.
Retrieve files before the sandbox’s main command exits. For more execution patterns, see Run commands.