Skip to main content
CoreWeave GPU and CPU Nodes provide high-performance, Node-local ephemeral storage on NVMe RAID, mounted at /mnt/local. Use this storage for temporary data such as intermediate training artifacts, caches, render outputs, or logs. This page is for workload authors who need fast scratch space close to compute and want to understand how to request it safely on CoreWeave Kubernetes Service (CKS). Pod container filesystems (the writable layer created by the container runtime) and Kubernetes emptyDir volumes are stored on this NVMe-backed storage by default, through kubelet and containerd directories under /mnt/local. As a result, most workloads benefit from local storage without any additional configuration.
Local storage applies to workloads running on CoreWeave Kubernetes Service (CKS). Data is non-persistent and may be lost when a Pod is deleted or when the Node reboots (for example, during maintenance or failure).
For most scratch-storage use cases, use emptyDir volumes. emptyDir is the standard Kubernetes mechanism for Node-local scratch space. An emptyDir volume is created when a Pod is scheduled onto a Node and is deleted automatically when the Pod is removed. On CoreWeave, emptyDir volumes have the following characteristics:
  • Backed by NVMe: Data is stored under the kubelet Pod directories on /mnt/local, an encrypted NVMe RAID array dedicated to ephemeral workload storage. This provides high throughput and low latency for scratch workloads.
  • Better for heavy writes: Writes to an emptyDir volume go directly to the underlying filesystem, avoiding the copy-on-write overhead of the container image overlay filesystem. For large or write-heavy scratch data, use emptyDir instead of writing to arbitrary paths in the container root filesystem.

​
Set the amount of local ephemeral storage

Size your emptyDir volume to match the scratch space your workload needs. This protects the Node from runaway usage and helps the scheduler place your Pod correctly. The amount of available local ephemeral storage depends on the Node type. For information on ephemeral storage size per instance type, see GPU instances and CPU-only instances. For ephemeral storage above 20Gi, include the size in the workload’s resource request. This lets the scheduler place the Pod on a Node with enough capacity. The following example shows where to set the size limit for an emptyDir volume: 
volumes:
  - name: scratch
    emptyDir:
      sizeLimit: 50Gi  # Recommended: set a limit matching your expected usage

​
Advanced hostPath volumes

hostPath volumes mount a specific directory on the Node’s filesystem directly into a Pod. Usually, emptyDir is the right choice for scratch storage. Use hostPath only when you explicitly need to access a specific path on the Node’s filesystem (for example, a legacy application that expects a hard-coded directory under /mnt/local). The following example shows where to set a hostPath volume: 
volumes:
  - name: specialized-scratch
    hostPath:
      path: /mnt/local/my-unique-workload-id  # Always use a unique subdirectory
      type: DirectoryOrCreate
Restrict paths to /mnt/local/. Do not mount volumes outside of /mnt/local/ (such as /tmp/ or /var/). Paths outside /mnt/local/ are on the Node’s root filesystem, which is small and not intended for workload data. Filling it can destabilize the Node. The backing for a path like /tmp also varies by environment. For details, see What backs the temporary directory.
When you use hostPath, keep the following points in mind:
  • Understand the trade-offs: hostPath bypasses Kubernetes storage isolation and couples your Pod to a particular Node layout. Volumes aren’t cleaned up automatically, and you’re responsible for managing path uniqueness, cleanup, and avoiding conflicts with other workloads.
  • Remember data is ephemeral: Data on /mnt/local does not survive a Node reboot. For details on encryption and the data lifecycle, see RAID layout and throughput.

​
RAID layout and throughput

Local storage is assembled from the Node’s local NVMe drives into a software RAID array, formatted as XFS, and mounted at /mnt/local. The RAID layout depends on how many local NVMe drives the Node has, which varies by instance type:
Local NVMe drivesRAID layoutThroughput characteristics
2 to 3 drivesRAID1 (mirror)Reads can split across the drives. Writes are about the speed of a single drive, because data is mirrored rather than striped.
4 or more drivesRAID10 (striped mirror)High parallel read and write throughput that scales with the number of drives and the concurrency of the workload.
Most GPU Nodes have eight local NVMe drives in an 8-drive RAID10 (RAID 1+0) array, which delivers high parallel throughput. Many CPU Nodes and some GPU instance types have fewer drives and use RAID1. Some instance types provide a single local NVMe drive, which is presented as one device with no RAID. Because the drive count differs by instance type, local-storage capacity and throughput also differ by instance type. For the local-storage capacity of a specific instance type, see GPU instances and CPU-only instances. The capacity listed for an instance is the raw sum of its drives. Usable space on a mirrored array (RAID1 or RAID10) is about half of that. To check the layout on a running Node, run cat /proc/mdstat or lsblk.
Local storage is encrypted at rest with a key held only in memory. The key is discarded when the Node reboots, which crypto-shreds the data and makes it unrecoverable. Combined with the non-persistent nature of local storage, this means data never survives a reboot.

​
What backs the temporary directory

The directory that backs /tmp depends on the environment, so do not assume /tmp is fast NVMe scratch space. The backing differs across CKS Pods and SUNK Slurm jobs:
EnvironmentWhat backs /tmp
Generic CKS PodThe container’s writable layer (an overlay), which sits on the NVMe RAID under /mnt/local.
SUNK Slurm, bare srun (no container)The Node’s local NVMe RAID array (/dev/md127).
SUNK Slurm, inside an enroot or Pyxis container (srun --container-image)tmpfs (RAM), sized to a fraction of Node memory. Writing large amounts of data to /tmp consumes RAM, not disk, and can fail with ENOSPC even when disk-usage metrics look clean.
To confirm what backs /tmp on a Node, run findmnt -no SOURCE,FSTYPE /tmp (or df -h /tmp). A tmpfs source means /tmp is RAM-backed. For deterministic, NVMe-backed scratch space in a CKS Pod, use an emptyDir volume rather than /tmp. For SUNK Slurm jobs, including how to redirect temporary files off tmpfs, see Node-local storage and /tmp on Slurm nodes.

​
Benchmark local storage

When you benchmark local storage, target the actual NVMe-backed volume and use a concurrent workload. Two common mistakes make local storage look slower than it is:
  • Benchmarking /tmp instead of NVMe. In a SUNK enroot or Pyxis container, /tmp is tmpfs (RAM), which inflates results and does not represent NVMe. Point your benchmark at a default emptyDir (not medium: Memory) or a known path under /mnt/local instead. See What backs the temporary directory.
  • Using a single stream. A single-stream tool such as dd, or fio with numjobs=1, is latency-bound and understates a striped RAID10 array, which needs many parallel I/Os to reach full bandwidth.
The following fio command measures write bandwidth with 12 concurrent jobs against a mounted scratch directory. Replace /scratch with the mount path of your emptyDir or NVMe-backed volume. 
fio -name=cwtest -directory=/scratch -direct=1 -ioengine=sync \
  -bs=4M -size=20G -numjobs=12 -runtime=60 -time_based \
  -group_reporting -refill_buffers -randrepeat=0 -rw=write
Expect lower single-stream numbers on Nodes with fewer drives (a single drive or a 2-drive RAID1 mirror) than on GPU Nodes with an 8-drive RAID10 array. For a fuller reference benchmark, see the CoreWeave runtests.sh script, and point its target path at your NVMe-backed volume.

​
Non-persistent storage

Because Node-local ephemeral storage is non-persistent, it doesn’t persist through Node reboots and doesn’t require a Persistent Volume. If you need data to survive Node reboots or be shared across Nodes, use a persistent storage option instead. To create a persistent filesystem volume that is independent of Node status, see Create Distributed File Storage volumes.
Last modified on June 26, 2026