Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt

Use this file to discover all available pages before exploring further.

You can delete files from Distributed File Storage using either the VAST trash directory or the rm command. The VAST trash directory allows asynchronous deletion of files and folders in bulk. Serial rm -rf on large directories is a slow and possibly blocking operation; moving files marked for removal to the trash folder (.vast_trash) is much faster. This guide covers using the VAST trash directory.
.vast_trash is only accessible from a full volume mount. Pods using a Kubernetes subPath mount cannot access .vast_trash. For details and migration options, see Working with subPath mounts.

​
Prerequisites

  • You have an active CKS cluster with a Distributed File Storage volume mounted.
  • You have root user access to the cluster.

​
Accessing VAST trash

The VAST trash directory (.vast_trash) is located at the root of each volume’s mount point. For example, if your volume is mounted at /mnt/vast, the VAST trash directory is at /mnt/vast/.vast_trash. .vast_trash does not appear in directory listings (ls). To verify it exists: Replace [VOLUME-MOUNT-PATH] with the path where your volume is mounted (for example, /mnt/vast). 
cd /[VOLUME-MOUNT-PATH]/.vast_trash
If this is successful, you have trash access. If you get an error like No such file or directory, you likely have a subPath mount, which means there is no .vast_trash directory. You need to use the full volume mount instead.

​
Moving files to VAST trash

The root user can delete files by moving them to the VAST trash directory: 
sudo mv /[PATH-TO-REMOVE] /mnt/vast/.vast_trash
If you need to update the ownership of the trash directory, contact CoreWeave Support. If you see an error like mv: cannot overwrite non-directory '.vast_trash/<name>' with directory '<name>', this can occur if the same directory name was previously deleted and VAST is still finalizing metadata cleanup. Try a different directory name or wait a few seconds before retrying. Once moved to .vast_trash, files are permanently deleted and cannot be recovered. Attempting to list the contents of the trash directory will return an empty list: 
ls -la /mnt/vast/.vast_trash
Example output
total 0

​
Working with subPath mounts

.vast_trash is not available when using Kubernetes subPath mounts, because subPath only exposes a subdirectory of the volume, not the root where .vast_trash exists. If you need .vast_trash access but are currently using a subPath mount, you can:
  • Mount the full volume at a different location and then update your workloads to use the new mount point.
  • Use symlinks to organize subdirectories within a full volume mount instead of using subPath.
To check if you’re using a subPath mount, run the following command: Replace [NAMESPACE] and [POD-NAME] with your namespace and pod name. 
kubectl describe -n [NAMESPACE] pod [POD-NAME] | grep -A6 "Mounts:"
In the Mounts section, look for entries with path="...": this indicates a subPath mount.

​
Diagnosing mixed subPath and full volume mounts

You may have a configuration where some directories are full volume mounts and others are subPath mounts. This can cause confusion when trying to use .vast_trash. In this example, you have a full volume mount at /mnt/datasets and a subPath mount at /mnt/experiments: 
/mnt/datasets          # Full volume mount
├── .vast_trash/       # ✅ Trash accessible here
├── training-data/
└── validation/

/mnt/experiments       # subPath mount (pointing to experiments/ in another volume)
├── run-001/           # ❌ No .vast_trash visible
├── run-002/
└── checkpoints/
To verify your mount configuration, run the following command: 
kubectl describe -n [NAMESPACE] pod [POD-NAME] | grep -A6 "Mounts:"
You’ll see output like this:
Example output
    Mounts:
      /mnt/datasets from my-volume (rw)
      /mnt/experiments from my-volume (rw,path="experiments")
In the Mounts section, look for entries with path="...": this indicates a subPath mount. In this example, /mnt/experiments is a subPath mount, so it does not have its own VAST trash directory. When you try to delete files in this setup, you will see different behavior depending on the mount type:
  • Deleting from the full volume mount at /mnt/datasets works: 
    sudo mv /mnt/datasets/training-data/old-dataset /mnt/datasets/.vast_trash/
  • Deleting from the subPath mount at /mnt/experiments fails: 
    sudo mv /mnt/experiments/run-001 /mnt/experiments/.vast_trash/
# Error: No such file or directory
If you want to delete files from a subPath, like the experiments directory, you have two options:
  1. Use standard deletion: rm -rf /mnt/experiments/run-001
  2. Migrate to a full volume mount to gain trash access (see Working with subPath mounts above).
Last modified on April 29, 2026