Upload, list, delete, and rename objects in AI Object Storage buckets
This guide explains how to manage objects stored in CoreWeave AI Object Storage buckets using S3-compatible tools, including the AWS CLI, s3cmd, Boto3, and s5cmd. For high-performance or bulk transfers, see Migrate data to AI Object Storage for s5cmd (use the CoreWeave fork for AI Object Storage). Alternatively, you can use Cyberduck to manage your buckets and objects in a graphical interface. The Cloud Console doesn’t support managing objects.To manage versioned buckets with rclone, see Versioned buckets.
Adequate permissions to manage objects in CoreWeave AI Object Storage (for example, s3:PutObject and s3:DeleteObject). For more information, see Object Storage S3 Permissions.
Upload objects to a bucket so your data is available for downstream applications, sharing, or processing. Choose the tab that matches the tool you use.
Alternatively, configure your CoreWeave credentials to work with the AWS CLI.We recommend using a separate profile for CoreWeave AI Object Storage to avoid conflicts with your other AWS profiles and S3-compatible services. If you don’t set up this configuration, you might encounter errors when using AI Object Storage. If you have no other AWS profiles, you can use the default profile instead of the cw profile created in the following steps. In that case, omit --profile cw from the commands.
For production workloads, CoreWeave recommends Workload Identity Federation for automated, short-lived credentials instead of static access keys.
Configure CoreWeave credentials with static access keys
Create a cw profile:
Create a new profile
aws configure --profile cw
When prompted, provide the following values:
AWS Access Key ID: The Access Key ID of your CoreWeave AI Object Storage Access Key.
AWS Secret Access Key: The Secret Key of your CoreWeave AI Object Storage Access Key.
Set the default endpoint URL to the appropriate endpoint for your use case:
The primary endpoint, https://cwobject.com, for use outside a CoreWeave cluster.
The LOTA endpoint, http://cwlota.com, for use inside a CoreWeave cluster. The LOTA endpoint routes to the LOTA path for best performance.
Set the primary endpoint for local development
aws configure set endpoint_url https://cwobject.com --profile cw
Set the S3 addressing_style to virtual:
Set virtual addressing style
aws configure set s3.addressing_style virtual --profile cw
To use this profile, pass --profile cw to your AWS CLI commands, or set AWS_PROFILE=cw in your environment.If you set endpoint_url and s3.addressing_style directly in your code (for example, in a Boto3 Config object), you can skip steps 3 and 4. The profile only needs the access key, secret key, and region.
Configure CoreWeave credentials with Workload Identity Federation (OIDC)
If you’ve configured Workload Identity Federation with OIDC, the AWS SDK automatically discovers temporary credentials. Set the following environment variables instead of configuring static access keys.Replace [ORG-ID] with your CoreWeave organization ID, [PATH-TO-JWT-TOKEN-FILE] with the path to your JWT token file, and [AVAILABILITY-ZONE] with your CoreWeave Availability Zone.
Configure WIF OIDC credentials
export AWS_CONTAINER_CREDENTIALS_FULL_URI=https://api.coreweave.com/v1/cwobject/temporary-credentials/oidc/[ORG-ID]export AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE=[PATH-TO-JWT-TOKEN-FILE]aws configure set s3.addressing_style virtualexport AWS_REGION="[AVAILABILITY-ZONE]"export AWS_ENDPOINT_URL_S3="https://cwobject.com"
With WIF configured, your S3-compatible tools and Boto3 code automatically obtain and refresh temporary credentials without passing access keys explicitly.Requires awscli >= 2.33.2 or boto3 >= 1.42.5. See Use Workload Identity Federation with OIDC for the full setup guide.
Use the following Python script to upload a file to your S3 bucket. Replace the following placeholders with the appropriate values for your request:
[BUCKET-NAME]: The name of the bucket to upload the file to.
[LOCAL-FILE-PATH]: The path to the local file to upload.
[OBJECT-NAME]: The key (name) of the object in the bucket.
If the upload succeeds, a confirmation message appears.
Example output
File uploaded to my-bucket-name/my-important-file.txt
Use the CoreWeave fork of s5cmd with AI Object StorageThe upstream s5cmd uses path-style addressing with custom endpoint URLs. AI Object Storage doesn’t support path-style addressing and requires virtual-hosted style URLs. Use the CoreWeave fork of s5cmd, which is compatible with AI Object Storage and can safely replace any existing s5cmd installation. For setup and usage, see Migrate data to AI Object Storage.
Ensure you have s5cmd installed and configured.Use the cp command to upload a file to a bucket. Replace [LOCAL-FILE-PATH] with the path to your file and [BUCKET-NAME] with your bucket name.
The Cloud Console lets you upload objects to a bucket without installing a client.
In the Cloud Console, navigate to Object Storage > Buckets and select the target bucket. The bucket opens to display its contents.
Navigate to the folder where you want to upload objects. To create a new folder, click Create Folder, enter a folder name, and confirm. The Cloud Console adds the corresponding prefix to the bucket.
Click Upload and select one or more files, or drag the files directly into the Cloud Console from your local file manager.
Monitor upload progress in the Cloud Console. When the upload completes, the new objects appear in the file list.
The Cloud Console uses multipart upload and supports resumable uploads, so interrupted transfers can be retried without starting over. For large bulk transfers (hundreds of GB or more), use the AWS CLI, Boto3, or s5cmd for better throughput.
CoreWeave AI Object Storage supports checksum algorithms for verifying object integrity on upload and download. For the full list of supported algorithms, see Checksum algorithms.
If you try to upload an object to a bucket in an Availability Zone where capacity quota limits have been reached, you receive an error message:<Message>The account is write suspended.</Message>To resolve this, you can request a quota increase.
Listing buckets and their contents helps you confirm which buckets exist in your account and verify which objects are stored in a given bucket. You can list buckets and their contents using S3-compatible tools such as the AWS CLI, s3cmd, Boto3, or s5cmd. If you’re working with versioned buckets, you can use rclone to list buckets and their contents, including delete markers.
AWS CLI
s3cmd
Boto3
s5cmd
Cloud Console
To see all your available buckets, use the ls command:
aws s3 ls
To list all the objects in a bucket, use the ls command to target a bucket path.
aws s3 ls s3://[BUCKET-NAME]
The terminal returns a listing for your selected bucket showing each object, its size, and its last modified date.
Alternatively, configure your CoreWeave credentials to work with the AWS CLI.We recommend using a separate profile for CoreWeave AI Object Storage to avoid conflicts with your other AWS profiles and S3-compatible services. If you don’t set up this configuration, you might encounter errors when using AI Object Storage. If you have no other AWS profiles, you can use the default profile instead of the cw profile created in the following steps. In that case, omit --profile cw from the commands.
For production workloads, CoreWeave recommends Workload Identity Federation for automated, short-lived credentials instead of static access keys.
Configure CoreWeave credentials with static access keys
Create a cw profile:
Create a new profile
aws configure --profile cw
When prompted, provide the following values:
AWS Access Key ID: The Access Key ID of your CoreWeave AI Object Storage Access Key.
AWS Secret Access Key: The Secret Key of your CoreWeave AI Object Storage Access Key.
Set the default endpoint URL to the appropriate endpoint for your use case:
The primary endpoint, https://cwobject.com, for use outside a CoreWeave cluster.
The LOTA endpoint, http://cwlota.com, for use inside a CoreWeave cluster. The LOTA endpoint routes to the LOTA path for best performance.
Set the primary endpoint for local development
aws configure set endpoint_url https://cwobject.com --profile cw
Set the S3 addressing_style to virtual:
Set virtual addressing style
aws configure set s3.addressing_style virtual --profile cw
To use this profile, pass --profile cw to your AWS CLI commands, or set AWS_PROFILE=cw in your environment.If you set endpoint_url and s3.addressing_style directly in your code (for example, in a Boto3 Config object), you can skip steps 3 and 4. The profile only needs the access key, secret key, and region.
Configure CoreWeave credentials with Workload Identity Federation (OIDC)
If you’ve configured Workload Identity Federation with OIDC, the AWS SDK automatically discovers temporary credentials. Set the following environment variables instead of configuring static access keys.Replace [ORG-ID] with your CoreWeave organization ID, [PATH-TO-JWT-TOKEN-FILE] with the path to your JWT token file, and [AVAILABILITY-ZONE] with your CoreWeave Availability Zone.
Configure WIF OIDC credentials
export AWS_CONTAINER_CREDENTIALS_FULL_URI=https://api.coreweave.com/v1/cwobject/temporary-credentials/oidc/[ORG-ID]export AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE=[PATH-TO-JWT-TOKEN-FILE]aws configure set s3.addressing_style virtualexport AWS_REGION="[AVAILABILITY-ZONE]"export AWS_ENDPOINT_URL_S3="https://cwobject.com"
With WIF configured, your S3-compatible tools and Boto3 code automatically obtain and refresh temporary credentials without passing access keys explicitly.Requires awscli >= 2.33.2 or boto3 >= 1.42.5. See Use Workload Identity Federation with OIDC for the full setup guide.
List all your buckets programmatically using the following script:
To view the contents of a specific bucket, replace [BUCKET-NAME] with the name of your bucket:
List objects in a bucket
import osimport boto3from botocore.client import Configboto_config = Config( region_name='[AVAILABILITY-ZONE]', s3={'addressing_style': 'virtual'})s3 = boto3.client( 's3', endpoint_url='https://cwobject.com', aws_access_key_id=os.environ['ACCESS_KEY_ID'], aws_secret_access_key=os.environ['SECRET_ACCESS_KEY'], config=boto_config)bucket_name = '[BUCKET-NAME]'response = s3.list_objects_v2(Bucket=bucket_name)if 'Contents' in response: print(f"Objects in {bucket_name}:") for obj in response['Contents']: print(f' {obj["Key"]} (Size: {obj["Size"]} bytes)')else: print(f"No objects found in {bucket_name}.")
The resulting output looks similar to the following:
Example output
Objects in my-bucket-name:my-first-file.txt (Size: 4500 bytes)another-file-of-mine.txt (Size: 5400 bytes)
Use the CoreWeave fork of s5cmd with AI Object StorageThe upstream s5cmd uses path-style addressing with custom endpoint URLs. AI Object Storage doesn’t support path-style addressing and requires virtual-hosted style URLs. Use the CoreWeave fork of s5cmd, which is compatible with AI Object Storage and can safely replace any existing s5cmd installation. For setup and usage, see Migrate data to AI Object Storage.
To list all your available buckets:
s5cmd --endpoint-url https://cwobject.com ls
To list the objects in a specific bucket, replace [BUCKET-NAME] with the name of your bucket:
s5cmd --endpoint-url https://cwobject.com ls 's3://[BUCKET-NAME]/*'
In the Cloud Console, navigate to Object Storage > Buckets to see all the buckets in your organization.
Select a bucket to open it. The Cloud Console displays objects and folder prefixes in the bucket, along with each object’s key, size, and created date.
Click a folder to navigate into it, or use the prefix path at the top of the page to navigate back to parent folders or the bucket root.
To find a bucket by name, or to find an object by name within the current bucket, use the search bar in the Cloud Console.To download an object, select it and click Download. The Cloud Console can download objects in any storage class, including the Archive tier; no restore operation is required.
Delete objects you no longer need to free up capacity and keep your buckets organized. Deletions are permanent on buckets that don’t have versioning enabled, so confirm the target object before you run these commands.
AWS CLI
s3cmd
Boto3
s5cmd
Cloud Console
To delete specific objects from a bucket, use the rm command with the AWS CLI.
aws s3 rm s3://[BUCKET-NAME]/[OBJECT-NAME]
When this succeeds, a confirmation message like this one appears:
Example output
delete: s3://my-bucket-name/my-important-file.txt
To delete an object from a bucket, use the del command.
Example command
s3cmd del s3://[BUCKET-NAME]/[OBJECT-NAME]
If the deletion succeeds, a confirmation message appears:
Alternatively, configure your CoreWeave credentials to work with the AWS CLI.We recommend using a separate profile for CoreWeave AI Object Storage to avoid conflicts with your other AWS profiles and S3-compatible services. If you don’t set up this configuration, you might encounter errors when using AI Object Storage. If you have no other AWS profiles, you can use the default profile instead of the cw profile created in the following steps. In that case, omit --profile cw from the commands.
For production workloads, CoreWeave recommends Workload Identity Federation for automated, short-lived credentials instead of static access keys.
Configure CoreWeave credentials with static access keys
Create a cw profile:
Create a new profile
aws configure --profile cw
When prompted, provide the following values:
AWS Access Key ID: The Access Key ID of your CoreWeave AI Object Storage Access Key.
AWS Secret Access Key: The Secret Key of your CoreWeave AI Object Storage Access Key.
Set the default endpoint URL to the appropriate endpoint for your use case:
The primary endpoint, https://cwobject.com, for use outside a CoreWeave cluster.
The LOTA endpoint, http://cwlota.com, for use inside a CoreWeave cluster. The LOTA endpoint routes to the LOTA path for best performance.
Set the primary endpoint for local development
aws configure set endpoint_url https://cwobject.com --profile cw
Set the S3 addressing_style to virtual:
Set virtual addressing style
aws configure set s3.addressing_style virtual --profile cw
To use this profile, pass --profile cw to your AWS CLI commands, or set AWS_PROFILE=cw in your environment.If you set endpoint_url and s3.addressing_style directly in your code (for example, in a Boto3 Config object), you can skip steps 3 and 4. The profile only needs the access key, secret key, and region.
Configure CoreWeave credentials with Workload Identity Federation (OIDC)
If you’ve configured Workload Identity Federation with OIDC, the AWS SDK automatically discovers temporary credentials. Set the following environment variables instead of configuring static access keys.Replace [ORG-ID] with your CoreWeave organization ID, [PATH-TO-JWT-TOKEN-FILE] with the path to your JWT token file, and [AVAILABILITY-ZONE] with your CoreWeave Availability Zone.
Configure WIF OIDC credentials
export AWS_CONTAINER_CREDENTIALS_FULL_URI=https://api.coreweave.com/v1/cwobject/temporary-credentials/oidc/[ORG-ID]export AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE=[PATH-TO-JWT-TOKEN-FILE]aws configure set s3.addressing_style virtualexport AWS_REGION="[AVAILABILITY-ZONE]"export AWS_ENDPOINT_URL_S3="https://cwobject.com"
With WIF configured, your S3-compatible tools and Boto3 code automatically obtain and refresh temporary credentials without passing access keys explicitly.Requires awscli >= 2.33.2 or boto3 >= 1.42.5. See Use Workload Identity Federation with OIDC for the full setup guide.
Use the following script to delete an object from a bucket. Replace [BUCKET-NAME] with the name of the bucket and [OBJECT-NAME] with the key of the object to delete.
Object my-important-file deleted from my-bucket-name
Use the CoreWeave fork of s5cmd with AI Object StorageThe upstream s5cmd uses path-style addressing with custom endpoint URLs. AI Object Storage doesn’t support path-style addressing and requires virtual-hosted style URLs. Use the CoreWeave fork of s5cmd, which is compatible with AI Object Storage and can safely replace any existing s5cmd installation. For setup and usage, see Migrate data to AI Object Storage.
To delete an object from a bucket, use the rm command. Replace [BUCKET-NAME] with the bucket name and [OBJECT-NAME] with the object key.
Rename an object to change its key without re-uploading the underlying data. This is useful when you need to correct a typo, restructure a key prefix, or align object names with a new naming convention.Make sure you have the following to rename an object:
s3:PutObject and s3:DeleteObject permissions.
A bucket that does not have versioning enabled, now or in the past.
A request scoped to renaming a single object within the same bucket. The source and destination keys must be in the same bucket.
s5cmd does not support the atomic RenameObject API. Use the AWS CLI or Boto3 for atomic renames.
AWS CLI
s3cmd
Boto3
Cloud Console
To rename an object, use the aws s3api rename-object command with the AWS CLI.Replace the following placeholders with the appropriate values for your request:
[BUCKET-NAME]: The name of the bucket containing the object.
[SOURCE-OBJECT-NAME]: The current name of the object.
[DESTINATION-OBJECT-NAME]: The new name for the object.
To rename an object, use the mv command with s3cmd.
Copy and delete operationThe s3cmd mv command performs a copy and delete operation instead of an atomic rename operation. For atomic rename operations, use the AWS CLI or Boto3 with the RenameObject API.
To prevent accidental overwrites when uploading objects, use the If-None-Match: * header with PutObject. For details, see Conditional writes.
Replace the following placeholders with the appropriate values for your request:
[BUCKET-NAME]: The name of the bucket containing the source object.
[SOURCE-OBJECT-NAME]: The name of the source object to rename.
[DESTINATION-OBJECT-NAME]: The new name for the object.
Alternatively, configure your CoreWeave credentials to work with the AWS CLI.We recommend using a separate profile for CoreWeave AI Object Storage to avoid conflicts with your other AWS profiles and S3-compatible services. If you don’t set up this configuration, you might encounter errors when using AI Object Storage. If you have no other AWS profiles, you can use the default profile instead of the cw profile created in the following steps. In that case, omit --profile cw from the commands.
For production workloads, CoreWeave recommends Workload Identity Federation for automated, short-lived credentials instead of static access keys.
Configure CoreWeave credentials with static access keys
Create a cw profile:
Create a new profile
aws configure --profile cw
When prompted, provide the following values:
AWS Access Key ID: The Access Key ID of your CoreWeave AI Object Storage Access Key.
AWS Secret Access Key: The Secret Key of your CoreWeave AI Object Storage Access Key.
Set the default endpoint URL to the appropriate endpoint for your use case:
The primary endpoint, https://cwobject.com, for use outside a CoreWeave cluster.
The LOTA endpoint, http://cwlota.com, for use inside a CoreWeave cluster. The LOTA endpoint routes to the LOTA path for best performance.
Set the primary endpoint for local development
aws configure set endpoint_url https://cwobject.com --profile cw
Set the S3 addressing_style to virtual:
Set virtual addressing style
aws configure set s3.addressing_style virtual --profile cw
To use this profile, pass --profile cw to your AWS CLI commands, or set AWS_PROFILE=cw in your environment.If you set endpoint_url and s3.addressing_style directly in your code (for example, in a Boto3 Config object), you can skip steps 3 and 4. The profile only needs the access key, secret key, and region.
Configure CoreWeave credentials with Workload Identity Federation (OIDC)
If you’ve configured Workload Identity Federation with OIDC, the AWS SDK automatically discovers temporary credentials. Set the following environment variables instead of configuring static access keys.Replace [ORG-ID] with your CoreWeave organization ID, [PATH-TO-JWT-TOKEN-FILE] with the path to your JWT token file, and [AVAILABILITY-ZONE] with your CoreWeave Availability Zone.
Configure WIF OIDC credentials
export AWS_CONTAINER_CREDENTIALS_FULL_URI=https://api.coreweave.com/v1/cwobject/temporary-credentials/oidc/[ORG-ID]export AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE=[PATH-TO-JWT-TOKEN-FILE]aws configure set s3.addressing_style virtualexport AWS_REGION="[AVAILABILITY-ZONE]"export AWS_ENDPOINT_URL_S3="https://cwobject.com"
With WIF configured, your S3-compatible tools and Boto3 code automatically obtain and refresh temporary credentials without passing access keys explicitly.Requires awscli >= 2.33.2 or boto3 >= 1.42.5. See Use Workload Identity Federation with OIDC for the full setup guide.
Use the rename_object method to rename an object. Replace [BUCKET-NAME], [SOURCE-OBJECT-NAME], and [DESTINATION-OBJECT-NAME] with the appropriate values for your request.