This guide explains how to manage objects stored in CoreWeave AI Object Storage buckets using S3-compatible tools, including the AWS CLI, s3cmd, Boto3, and s5cmd. For high-performance or bulk transfers, see Migrate data to AI Object Storage for s5cmd (use the CoreWeave fork for AI Object Storage). Alternatively, you can use Cyberduck to manage your buckets and objects in a graphical interface. The Cloud Console doesn’t support managing objects.
To manage versioned buckets with rclone, see Versioned buckets.
Prerequisites
This guide presumes you have the following:
Add objects
Upload objects to a bucket so your data is available for downstream applications, sharing, or processing. Choose the tab that matches the tool you use.
AWS CLI
s3cmd
Boto3
s5cmd
Cloud Console
Ensure you have the AWS CLI installed and configured.Use the s3 cp command to copy a file into a bucket addressed using the s3:// scheme. Ensure you have s3cmd installed, configured, and correctly credentialed.Use the put command to copy an object into the bucket.This returns the following output, confirming your object is stored in your bucket:
-
Set environment variables for your CoreWeave credentials:
Alternatively, configure your CoreWeave credentials to work with the AWS CLI.
We recommend using a separate profile for CoreWeave AI Object Storage to avoid conflicts with your other AWS profiles and S3-compatible services. If you don’t set up this configuration, you might encounter errors when using AI Object Storage. If you have no other AWS profiles, you can use the
default profile instead of the cw profile created in the following steps. In that case, omit --profile cw from the commands.
For production workloads, CoreWeave recommends Workload Identity Federation for automated, short-lived credentials instead of static access keys.
Configure CoreWeave credentials with static access keys
Configure CoreWeave credentials with Workload Identity Federation (OIDC)
-
Use the following Python script to upload a file to your S3 bucket. Replace the following placeholders with the appropriate values for your request:
[BUCKET-NAME]: The name of the bucket to upload the file to.
[LOCAL-FILE-PATH]: The path to the local file to upload.
[OBJECT-NAME]: The key (name) of the object in the bucket.
[AVAILABILITY-ZONE]: Your CoreWeave Availability Zone (for example, US-EAST-04A). See Regions and Availability Zones.
Upload a file to a bucket
If the upload succeeds, a confirmation message appears.Use the CoreWeave fork of s5cmd with AI Object StorageThe upstream s5cmd uses path-style addressing with custom endpoint URLs. AI Object Storage doesn’t support path-style addressing and requires virtual-hosted style URLs. Use the CoreWeave fork of s5cmd, which is compatible with AI Object Storage and can safely replace any existing s5cmd installation. For setup and usage, see Migrate data to AI Object Storage. Ensure you have s5cmd installed and configured.Use the cp command to upload a file to a bucket. Replace [LOCAL-FILE-PATH] with the path to your file and [BUCKET-NAME] with your bucket name.The Cloud Console lets you upload objects to a bucket without installing a client.
- In the Cloud Console, navigate to Object Storage > Buckets and select the target bucket. The bucket opens to display its contents.
- Navigate to the folder where you want to upload objects. To create a new folder, click Create Folder, enter a folder name, and confirm. The Cloud Console adds the corresponding prefix to the bucket.
- Click Upload and select one or more files, or drag the files directly into the Cloud Console from your local file manager.
- Monitor upload progress in the Cloud Console. When the upload completes, the new objects appear in the file list.
The Cloud Console uses multipart upload and supports resumable uploads, so interrupted transfers can be retried without starting over. For large bulk transfers (hundreds of GB or more), use the AWS CLI, Boto3, or s5cmd for better throughput.
Verify object integrity
CoreWeave AI Object Storage supports checksum algorithms for verifying object integrity on upload and download. For the full list of supported algorithms, see Checksum algorithms.
Quota limit errors
If you try to upload an object to a bucket in an Availability Zone where capacity quota limits have been reached, you receive an error message:
<Message>The account is write suspended.</Message>
To resolve this, you can request a quota increase.
List buckets and their contents
Listing buckets and their contents helps you confirm which buckets exist in your account and verify which objects are stored in a given bucket. You can list buckets and their contents using S3-compatible tools such as the AWS CLI, s3cmd, Boto3, or s5cmd. If you’re working with versioned buckets, you can use rclone to list buckets and their contents, including delete markers.
AWS CLI
s3cmd
Boto3
s5cmd
Cloud Console
To see all your available buckets, use the ls command:To list all the objects in a bucket, use the ls command to target a bucket path.The terminal returns a listing for your selected bucket showing each object, its size, and its last modified date.Example file listing output
To list all your available buckets, use the ls command:To see the contents of a specific bucket, target the bucket’s path using the s3:// scheme.This command outputs a list of file paths including the bucket name and file name. For example:
-
Set environment variables for your CoreWeave credentials:
Alternatively, configure your CoreWeave credentials to work with the AWS CLI.
We recommend using a separate profile for CoreWeave AI Object Storage to avoid conflicts with your other AWS profiles and S3-compatible services. If you don’t set up this configuration, you might encounter errors when using AI Object Storage. If you have no other AWS profiles, you can use the
default profile instead of the cw profile created in the following steps. In that case, omit --profile cw from the commands.
For production workloads, CoreWeave recommends Workload Identity Federation for automated, short-lived credentials instead of static access keys.
Configure CoreWeave credentials with static access keys
Configure CoreWeave credentials with Workload Identity Federation (OIDC)
-
List all your buckets programmatically using the following script:
-
To view the contents of a specific bucket, replace
[BUCKET-NAME] with the name of your bucket:
The resulting output looks similar to the following:Use the CoreWeave fork of s5cmd with AI Object StorageThe upstream s5cmd uses path-style addressing with custom endpoint URLs. AI Object Storage doesn’t support path-style addressing and requires virtual-hosted style URLs. Use the CoreWeave fork of s5cmd, which is compatible with AI Object Storage and can safely replace any existing s5cmd installation. For setup and usage, see Migrate data to AI Object Storage. To list all your available buckets:To list the objects in a specific bucket, replace [BUCKET-NAME] with the name of your bucket:
- In the Cloud Console, navigate to Object Storage > Buckets to see all the buckets in your organization.
- Select a bucket to open it. The Cloud Console displays objects and folder prefixes in the bucket, along with each object’s key, size, and created date.
- Click a folder to navigate into it, or use the prefix path at the top of the page to navigate back to parent folders or the bucket root.
To find a bucket by name, or to find an object by name within the current bucket, use the search bar in the Cloud Console.To download an object, select it and click Download. The Cloud Console can download objects in any storage class, including the Archive tier; no restore operation is required.
Delete an object from a bucket
Delete objects you no longer need to free up capacity and keep your buckets organized. Deletions are permanent on buckets that don’t have versioning enabled, so confirm the target object before you run these commands.
AWS CLI
s3cmd
Boto3
s5cmd
Cloud Console
To delete specific objects from a bucket, use the rm command with the AWS CLI.When this succeeds, a confirmation message like this one appears: To delete an object from a bucket, use the del command.If the deletion succeeds, a confirmation message appears:
-
Set environment variables for your CoreWeave credentials:
Alternatively, configure your CoreWeave credentials to work with the AWS CLI.
We recommend using a separate profile for CoreWeave AI Object Storage to avoid conflicts with your other AWS profiles and S3-compatible services. If you don’t set up this configuration, you might encounter errors when using AI Object Storage. If you have no other AWS profiles, you can use the
default profile instead of the cw profile created in the following steps. In that case, omit --profile cw from the commands.
For production workloads, CoreWeave recommends Workload Identity Federation for automated, short-lived credentials instead of static access keys.
Configure CoreWeave credentials with static access keys
Configure CoreWeave credentials with Workload Identity Federation (OIDC)
-
Use the following script to delete an object from a bucket. Replace
[BUCKET-NAME] with the name of the bucket and [OBJECT-NAME] with the key of the object to delete.
Delete an object from a bucket
If successful, a success message appears:Use the CoreWeave fork of s5cmd with AI Object StorageThe upstream s5cmd uses path-style addressing with custom endpoint URLs. AI Object Storage doesn’t support path-style addressing and requires virtual-hosted style URLs. Use the CoreWeave fork of s5cmd, which is compatible with AI Object Storage and can safely replace any existing s5cmd installation. For setup and usage, see Migrate data to AI Object Storage. To delete an object from a bucket, use the rm command. Replace [BUCKET-NAME] with the bucket name and [OBJECT-NAME] with the object key.Delete an object with s5cmd
To delete multiple objects matching a pattern, replace [PREFIX] with the object key prefix to match:Delete objects matching a pattern
- In the Cloud Console, open the bucket and navigate to the object’s folder.
- Select one or more objects or folders to delete. To delete every object under a prefix, select the folder itself.
- Click Delete. The Cloud Console displays a confirmation dialog listing the items to delete.
- Confirm the deletion. The objects are permanently removed from the bucket.
Deleting a folder in the Cloud Console removes all objects under that prefix. Review the confirmation dialog before proceeding.
To safely identify and remove objects your workloads no longer access, for example to reclaim capacity in a shared bucket, see Delete cold objects.
Rename an object
Rename an object to change its key without re-uploading the underlying data. This is useful when you need to correct a typo, restructure a key prefix, or align object names with a new naming convention.
Make sure you have the following to rename an object:
s3:PutObject and s3:DeleteObject permissions.
- A bucket that does not have versioning enabled, now or in the past.
- A request scoped to renaming a single object within the same bucket. The source and destination keys must be in the same bucket.
For more information, see Rename Objects.
s5cmd does not support the atomic RenameObject API. Use the AWS CLI or Boto3 for atomic renames.
AWS CLI
s3cmd
Boto3
Cloud Console
To rename an object, use the aws s3api rename-object command with the AWS CLI.Replace the following placeholders with the appropriate values for your request:
[BUCKET-NAME]: The name of the bucket containing the object.
[SOURCE-OBJECT-NAME]: The current name of the object.
[DESTINATION-OBJECT-NAME]: The new name for the object.
When this succeeds, a confirmation message appears: To rename an object, use the mv command with s3cmd.Copy and delete operationThe s3cmd mv command performs a copy and delete operation instead of an atomic rename operation. For atomic rename operations, use the AWS CLI or Boto3 with the RenameObject API.
To prevent accidental overwrites when uploading objects, use the If-None-Match: * header with PutObject. For details, see Conditional writes. Replace the following placeholders with the appropriate values for your request:
[BUCKET-NAME]: The name of the bucket containing the source object.
[SOURCE-OBJECT-NAME]: The name of the source object to rename.
[DESTINATION-OBJECT-NAME]: The new name for the object.
-
Set environment variables for your CoreWeave credentials:
Alternatively, configure your CoreWeave credentials to work with the AWS CLI.
We recommend using a separate profile for CoreWeave AI Object Storage to avoid conflicts with your other AWS profiles and S3-compatible services. If you don’t set up this configuration, you might encounter errors when using AI Object Storage. If you have no other AWS profiles, you can use the
default profile instead of the cw profile created in the following steps. In that case, omit --profile cw from the commands.
For production workloads, CoreWeave recommends Workload Identity Federation for automated, short-lived credentials instead of static access keys.
Configure CoreWeave credentials with static access keys
Configure CoreWeave credentials with Workload Identity Federation (OIDC)
-
Use the
rename_object method to rename an object. Replace [BUCKET-NAME], [SOURCE-OBJECT-NAME], and [DESTINATION-OBJECT-NAME] with the appropriate values for your request.
- In the Cloud Console, open the bucket.
- Select the object you want to rename.
- Choose Rename and enter the new object key. The rename is performed within the same bucket.
The bucket must not have versioning enabled or have had versioning enabled in the past. See Rename objects for the full list of restrictions.