File system snapshots - CoreWeave Docs

File System Snapshots (FSS) let a sandbox write to a local working directory, capture that directory into object storage as an immutable snapshot, and later restore it into new sandboxes. Use FSS to suspend and resume a sandbox’s filesystem across runs, or to fork one snapshot into several sandboxes that then diverge independently.

CoreWeave sandboxes are in public preview. For access, contact your CoreWeave account team, CoreWeave Support, or email support@coreweave.com.

How it works

A sandbox with FSS mounts a writable scratch filesystem at a path you choose. The runner backs this mount with an EmptyDir, not a direct object storage mount. Snapshot and restore operations copy a tarball between that local filesystem and an organization-scoped object storage bucket. Each snapshot is a FileSystemSnapshot resource identified by a server-assigned fileSystemSnapshotId. Snapshots have three defining properties:

Immutable: creating a snapshot writes one archive. Restoring a snapshot never changes the source snapshot, so a later restore always sees the original bytes.
Organization-scoped: you can restore only snapshots created in your own organization. A snapshot ID from another organization is treated as not found.
Independent lifecycle: snapshots are managed separately from sandboxes. Deleting a sandbox does not delete its snapshots, and deleting a snapshot does not affect sandboxes already running from it.

By default, CoreWeave provisions and manages the object storage bucket that holds your snapshots. To store snapshots in a bucket you own and control instead, see Bring your own bucket.

Enable file system snapshots

FSS is enabled per organization. If your organization is not enabled, snapshot calls fail: the Python client raises SnapshotNotSupportedError, and the HTTP API rejects the request before any snapshot work begins. To request access, contact your CoreWeave account team or CoreWeave Support. Once FSS is enabled, CoreWeave manages a bucket for your organization automatically. You don’t have to configure storage unless you want to bring your own bucket.

from cwsandbox import Sandbox, FileSystemSnapshotOptions
from cwsandbox.exceptions import SnapshotNotSupportedError

try:
    snapshot_id = sandbox.snapshot().result()
except SnapshotNotSupportedError:
    print("File system snapshots are not enabled for this organization.")

Start a sandbox with a snapshot mount

A sandbox can start from one of two filesystem sources: a fresh, empty scratch filesystem, or a restore of an existing snapshot.

Fresh scratch

A fresh scratch filesystem starts empty and can be snapshotted later. Use it for workflows that need writable local state during a run.

Python
HTTP API

from cwsandbox import Sandbox, FileSystemSnapshotOptions

with Sandbox.run(
    container_image="ubuntu:22.04",
    file_system_snapshot=FileSystemSnapshotOptions(mount_path="/work", size="10Gi"),
) as sandbox:
    sandbox.exec(["sh", "-c", "echo hello > /work/data.txt"]).result()

Set fileSystem.mountPath and fileSystem.size. Leave fileSystemSnapshot unset to start empty.

curl -X POST https://api.cwsandbox.com/v1beta2/sandboxes \
  -H "Authorization: Bearer $CWSANDBOX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "containerImage": "ubuntu:22.04",
    "command": "sleep",
    "args": ["3600"],
    "fileSystem": {
      "mountPath": "/work",
      "size": "10Gi"
    }
  }'

Restore from a snapshot

A restore starts the filesystem from an existing snapshot. Set the snapshot ID on the mount’s snapshot source.

Python
HTTP API

from cwsandbox import Sandbox, FileSystemSnapshotOptions

with Sandbox.run(
    container_image="ubuntu:22.04",
    file_system_snapshot=FileSystemSnapshotOptions(
        mount_path="/work",
        file_system_snapshot_id="fss_...",
    ),
) as sandbox:
    contents = sandbox.exec(["cat", "/work/data.txt"]).result()
    print(contents.stdout)

curl -X POST https://api.cwsandbox.com/v1beta2/sandboxes \
  -H "Authorization: Bearer $CWSANDBOX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "containerImage": "ubuntu:22.04",
    "command": "sleep",
    "args": ["3600"],
    "fileSystem": {
      "mountPath": "/work",
      "size": "10Gi",
      "fileSystemSnapshot": {
        "fileSystemSnapshotId": "fss_..."
      }
    }
  }'

A snapshot must be READY before you restore it. Restoring a snapshot that is still CREATING fails. See Snapshot status and failures.

Take a snapshot

You can capture a snapshot in two ways: on stop, when a sandbox shuts down, and mid-life, from a running sandbox.

Snapshot on stop

To preserve a sandbox’s filesystem when it shuts down, stop the sandbox with snapshot-on-stop enabled. This is the pattern for suspend and resume: stop with a snapshot, then later start a new sandbox that restores it.

Python
HTTP API

sandbox.stop(snapshot_on_stop=True).result()
print(sandbox.file_system_snapshot_id)

StopSandbox returns a fileSystemSnapshotId when the snapshot request is accepted. With waitForReady set to true, the call waits until the snapshot reaches READY or FAILED, bounded by maxTimeoutSeconds. Without it, the call can return while the snapshot is still CREATING.

curl -X POST https://api.cwsandbox.com/v1beta2/sandboxes/[SANDBOX-ID]/stop \
  -H "Authorization: Bearer $CWSANDBOX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "sandboxId": "[SANDBOX-ID]",
    "fileSystemSnapshotOnStop": true,
    "waitForReady": true,
    "idempotencyKey": "stop-[SANDBOX-ID]-snapshot-1",
    "maxTimeoutSeconds": 600
  }'

Mid-life snapshot

A mid-life snapshot captures a running sandbox without stopping it. The sandbox keeps running afterward.

Python
HTTP API

snapshot() waits until the snapshot is READY by default and returns the snapshot ID.

snapshot_id = sandbox.snapshot().result()
print(f"Created snapshot {snapshot_id}")

CreateFileSystemSnapshot accepts the same idempotencyKey, waitForReady, and maxTimeoutSeconds controls as snapshot-on-stop. Unlike StopSandbox, waitForReady defaults to true.

curl -X POST https://api.cwsandbox.com/v1beta2/sandboxes/[SANDBOX-ID]/file-system-snapshots \
  -H "Authorization: Bearer $CWSANDBOX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "sandboxId": "[SANDBOX-ID]",
    "waitForReady": true,
    "maxTimeoutSeconds": 600
  }'

With waitForReady set to false, a successful response means the snapshot row was created and the snapshot command was dispatched. The archive may still be CREATING, and it can later become FAILED. Poll the snapshot before you restore it.

Fork a snapshot

The same snapshot can be restored into more than one sandbox. Because each snapshot is immutable, the restored sandboxes start from identical bytes and then diverge as they write.

Create a READY snapshot, with either snapshot-on-stop or a mid-life snapshot.
Start sandbox A with fileSystemSnapshotId set to that snapshot.
Start sandbox B with the same fileSystemSnapshotId.
Writes in sandbox A and sandbox B diverge independently. The source snapshot is unchanged, so a later restore from the same ID still sees the original snapshot, not the writes from either fork.

from cwsandbox import Sandbox, FileSystemSnapshotOptions

# Restore the same snapshot into two independent sandboxes.
with Sandbox.run(
    file_system_snapshot=FileSystemSnapshotOptions(mount_path="/work", file_system_snapshot_id=snapshot_id),
) as fork_a, Sandbox.run(
    file_system_snapshot=FileSystemSnapshotOptions(mount_path="/work", file_system_snapshot_id=snapshot_id),
) as fork_b:
    fork_a.exec(["sh", "-c", "echo a >> /work/data.txt"]).result()
    fork_b.exec(["sh", "-c", "echo b >> /work/data.txt"]).result()

Manage snapshots

List, fetch, and delete snapshots independently of the sandboxes that created them.

Python
HTTP API

from cwsandbox import Sandbox

# List all snapshots in your organization.
snapshots = Sandbox.list_snapshots().result()

# Fetch one snapshot's details.
snapshot = Sandbox.get_snapshot(snapshot_id).result()
print(f"{snapshot.file_system_snapshot_id}: {snapshot.size_bytes} bytes")

# Delete a snapshot. missing_ok makes the call idempotent.
Sandbox.delete_snapshot(snapshot_id, missing_ok=True).result()

# List org-scoped snapshots.
curl https://api.cwsandbox.com/v1beta2/file-system-snapshots \
  -H "Authorization: Bearer $CWSANDBOX_API_KEY"

# Get one snapshot.
curl https://api.cwsandbox.com/v1beta2/file-system-snapshots/[SNAPSHOT-ID] \
  -H "Authorization: Bearer $CWSANDBOX_API_KEY"

# Delete one snapshot.
curl -X DELETE https://api.cwsandbox.com/v1beta2/file-system-snapshots/[SNAPSHOT-ID] \
  -H "Authorization: Bearer $CWSANDBOX_API_KEY"

Snapshot status and failures

Snapshots are created asynchronously. A snapshot request can succeed and return a fileSystemSnapshotId while the archive is still being written, and the snapshot can fail afterward. This happens most often when waitForReady is false, or when a client timeout occurs while the runner is still archiving. To check progress, fetch the snapshot and read two fields:

status: the lifecycle state. The terminal values are READY and FAILED. The HTTP API returns the fully qualified form, such as FILE_SYSTEM_SNAPSHOT_STATUS_READY.
statusReason: populated when status is FAILED, explaining why.

Poll GetFileSystemSnapshot until the snapshot reaches READY before you restore it. The Python client’s snapshot() and get_snapshot() handle this polling for you when you wait for the result. The following table lists the common failure reasons and what to do about each.

Reason	What it means	What to do
`CWSANDBOX_FSS_AUTH_FAILED`	Credential exchange or bucket authorization failed.	Retry after the bucket or WIF policy is fixed. Contact support if this is a CoreWeave-managed bucket.
`CWSANDBOX_FSS_TRANSPORT_FAILED`	The runner could not reach object storage, or the archive transfer failed.	Retry. If it repeats, contact support with the snapshot ID.
`CWSANDBOX_FSS_BACKEND_THROTTLED`	Object storage returned sustained throttling or backend errors.	Retry later.
`CWSANDBOX_FSS_CREATE_TIMED_OUT`	The snapshot did not finish inside the configured timeout.	Retry with a higher `maxTimeoutSeconds`, or reduce the data size.
`CWSANDBOX_FSS_CANCELED`	The snapshot was canceled, usually by force-deleting the sandbox during in-flight work.	Retry the snapshot if you still need the data.
`CWSANDBOX_FSS_QUOTA_EXCEEDED`	The organization hit its snapshot count or storage cap.	Delete unused snapshots, or request a quota increase.
`CWSANDBOX_FSS_NOT_READY`	A restore referenced a snapshot that is not `READY`.	Poll until the snapshot is `READY`, or choose a different snapshot.
`CWSANDBOX_FSS_NOT_FOUND`	The snapshot does not exist in your organization.	Check the snapshot ID and the organization.
`CWSANDBOX_FSS_SIZE_EXCEEDED`	The requested filesystem size exceeds the supported scratch size.	Request a smaller filesystem, or contact support about limits.

If a runner finishes uploading the archive but cannot record the result, the snapshot row can become FAILED even though an object exists in the bucket. Retry the snapshot in this case. For asynchronous failures, the platform also emits a sandbox.file_system_snapshot.async_fail event, so downstream notification systems can react to snapshots that fail after the original call returned.

Limitations

FSS version 1 provides snapshot-backed local scratch storage. It is not a shared filesystem and does not provide live, multi-writer mounts. Sandboxes do not read or write each other’s filesystems while running. They share state only by snapshotting and restoring.

SDK example

For a complete, runnable Python example that starts a sandbox, takes a mid-life snapshot, forks it, captures a snapshot on stop, and manages snapshots, see file_system_snapshots.py in the cwsandbox-client repository.

Bring your own bucket for file system snapshots: store snapshots in a bucket you own.
Sandbox lifecycle: how stop() and the sandbox states work.
Python client: install the SDK and explore the API.

​How it works

​Enable file system snapshots

​Start a sandbox with a snapshot mount

​Fresh scratch

​Restore from a snapshot

​Take a snapshot

​Snapshot on stop

​Mid-life snapshot

​Fork a snapshot

​Manage snapshots

​Snapshot status and failures

​Limitations

​SDK example

​Related resources

How it works

Enable file system snapshots

Start a sandbox with a snapshot mount

Fresh scratch

Restore from a snapshot

Take a snapshot

Snapshot on stop

Mid-life snapshot

Fork a snapshot

Manage snapshots

Snapshot status and failures

Limitations

SDK example

Related resources