Create buckets - CoreWeave Docs

Buckets are the primary organizational unit used to store object data in CoreWeave AI Object Storage. They are similar to folders in a file system. This guide explains how to create a bucket with the CoreWeave Cloud Console or CLI clients such as S3cmd, AWS CLI, and Boto3. (You can also use the CoreWeave fork of s5cmd; upstream s5cmd uses path-style addressing, which AI Object Storage does not support.)

Prerequisites

You must generate an Access Key before creating a bucket. When using the Cloud Console, use Cloud Console tokens to generate the Access Key. Workload Identity Federation is supported when using CLI clients.
To create a bucket with the Cloud Console, you must have s3:CreateBucket and cwobject:CreateAccessKey permissions granted to your user or group by an organization access policy.
Make sure your configuration uses the correct endpoint URL and has virtual-hosted addressing enabled.

Create a bucket with Cloud Console

To create a bucket with the CoreWeave Cloud Console, follow these steps:

In the Cloud Console, navigate to the Buckets page.
Click the Create Bucket button at the top right corner of the page.
In the dialog, enter a Bucket Name.
Bucket naming rules
Bucket names must be globally unique, and adhere to the following rules:
- Length: 3 to 63 characters.
- Characters: Only lowercase letters (a-z), numbers (0-9), and hyphens (-). No dots, uppercase letters, underscores, spaces, or other special characters.
- Start and end: Must begin and end with a letter or number. Cannot start or end with a hyphen (-).
- Prohibited patterns: Cannot start with xn--.- Reserved: Must not begin with cw-, vip-, or log-stitcher-ch-. Must not be the exact name int. These are reserved for internal use.
Select an Availability Zone.
Click Create.
Click More options to view information in Grafana, or to delete the bucket.

Create a bucket with CLI clients

CoreWeave AI Object Storage is compatible with S3-compatible clients, including S3cmd, the AWS CLI, Boto3, and the CoreWeave fork of s5cmd.

Use the CoreWeave fork of s5cmd with AI Object StorageThe upstream s5cmd uses path-style addressing with custom endpoint URLs. AI Object Storage does not support path-style addressing and requires virtual-hosted style URLs. Use the CoreWeave fork of s5cmd, which is compatible with AI Object Storage and can safely replace any existing s5cmd installation. See Migrate data to AI Object Storage for setup and usage.

Before creating a bucket, configure your client. Note that the primary endpoint, https://cwobject.com, requires TLS v1.3. Ensure your S3-compatible tools and OpenSSL use TLS v1.3. In the sections that follow, replace these placeholders:

[BUCKET-NAME]: A globally unique name for your bucket. Follow the naming rules:
Bucket naming rules
Bucket names must be globally unique, and adhere to the following rules:
- Length: 3 to 63 characters.
- Characters: Only lowercase letters (a-z), numbers (0-9), and hyphens (-). No dots, uppercase letters, underscores, spaces, or other special characters.
- Start and end: Must begin and end with a letter or number. Cannot start or end with a hyphen (-).
- Prohibited patterns: Cannot start with xn--.- Reserved: Must not begin with cw-, vip-, or log-stitcher-ch-. Must not be the exact name int. These are reserved for internal use.
[AVAILABILITY-ZONE]: A CoreWeave Availability Zone that supports AI Object Storage (for example, US-EAST-04A). S3-compatible tools use the term “region” to define the location of the storage service, but at CoreWeave, Regions are larger constructs organized into Availability Zones. Use a CoreWeave Availability Zone that supports AI Object Storage:
Availability Zones that support AI Object Storage
- US-CENTRAL
- US-EAST
- US-WEST
- CA-EAST
- EU-SOUTH
US-CENTRAL-05A

US-CENTRAL-06A

US-CENTRAL-07A

US-CENTRAL-08A

US-CENTRAL-08B

US-EAST-01A

US-EAST-02A

US-EAST-03A

US-EAST-04A

US-EAST-04B

US-EAST-06A

US-EAST-08A

US-EAST-13A

US-EAST-14A

RNO2A

US-WEST-01A

US-WEST-04A

US-WEST-09B

US-WEST-10A

CA-EAST-01A

EU-SOUTH-03B

EU-SOUTH-04A
Learn more about Regions and Availability Zones.

Bucket creation delayWhen a bucket is created using an S3-compatible client, it takes approximately one minute before the bucket becomes available due to DNS caching. You may see the following error immediately after bucket creation: An error occurred (InvalidRegion) when calling the ListObjects operation: Region does not match. Wait one minute, then try again.This delay does not occur when creating buckets in the Cloud Console.

AWS CLI
s3cmd
Boto3

To create a bucket with AWS CLI, use the aws s3api command. The LocationConstraint parameter is required.

Create a bucket with AWS CLI

aws s3api create-bucket \
    --bucket [BUCKET-NAME] \
    --region [AVAILABILITY-ZONE] \
    --create-bucket-configuration LocationConstraint=[AVAILABILITY-ZONE]

If you need to install or configure AWS CLI for AI Object Storage, see Manage buckets with AWS CLI.

To create a bucket with S3cmd, use the mb (make bucket) subcommand. The --bucket-location parameter is required.

Make a bucket with S3cmd

s3cmd mb --bucket-location=[AVAILABILITY-ZONE] s3://[BUCKET-NAME]

If you need to configure S3cmd for AI Object storage, see Manage buckets with S3cmd.

To create a bucket with Boto3, use the create_bucket method. The LocationConstraint parameter is required.

Set environment variables for your CoreWeave credentials:
```
export ACCESS_KEY_ID="[ACCESS-KEY-ID]"
export SECRET_ACCESS_KEY="[SECRET-ACCESS-KEY]"
```
Alternatively, configure your CoreWeave credentials to work with the AWS CLI. Using a separate profile for CoreWeave AI Object Storage is recommended to avoid conflicts with your other AWS profiles and S3-compatible services; if you do not set up this configuration, you may encounter errors when using AI Object Storage.
Configure CoreWeave credentials
1. Create a new credentials file and profile in your CoreWeave configuration directory.
  Create a new credentials file and profile
  AWS_SHARED_CREDENTIALS_FILE=~/.coreweave/cw.credentials aws configure --profile cw
2. When prompted for information, provide the following values:
  - AWS Access Key ID: The Access Key ID of your CoreWeave AI Object Storage Access Key.
  - AWS Secret Access Key: The Secret Key of your CoreWeave AI Object Storage Access Key.
  - Default region name: Optional. To set a default region, refer to the CoreWeave Availability Zones.
  - Default output format: Use json for JSON output.
3. Set the default endpoint URL to the appropriate endpoint for your use case:
  - The primary endpoint, https://cwobject.com, for use when running outside of a CoreWeave cluster.
  - The LOTA endpoint, http://cwlota.com, for use when running inside a CoreWeave cluster. The LOTA endpoint routes to the LOTA path for best performance.
  Set the primary endpoint for local development
  AWS_CONFIG_FILE=~/.coreweave/cw.config aws configure set endpoint_url https://cwobject.com --profile cw
4. Set the S3 addressing_style to virtual:
  Set virtual addressing style
  AWS_CONFIG_FILE=~/.coreweave/cw.config aws configure set s3.addressing_style virtual --profile cw

Replace [BUCKET-NAME] with the name of the bucket and [AVAILABILITY-ZONE] with a CoreWeave Availability Zone.

Create a bucket with Boto3

import os
import boto3
from botocore.client import Config

boto_config = Config(
    region_name='US-EAST-04A',
    s3={'addressing_style': 'virtual'}
)

s3 = boto3.client(
    's3',
    endpoint_url='https://cwobject.com',
    aws_access_key_id=os.environ['ACCESS_KEY_ID'],
    aws_secret_access_key=os.environ['SECRET_ACCESS_KEY'],
    config=boto_config
)

response = s3.create_bucket(
    Bucket='[BUCKET-NAME]',
    CreateBucketConfiguration={
        'LocationConstraint': '[AVAILABILITY-ZONE]',
    },
)

print(response)

If you need to configure Boto3 for AI Object Storage, see Manage buckets with Boto3.

Versioned buckets

If you need a versioned bucket, versioning must be enabled at creation time. You cannot enable versioning on an existing bucket after it has been created. To create and configure a versioned bucket, see Versioned buckets.

Next steps

After creating a bucket, set an organization access policy to control access to the Object Storage S3-compatible API.

Storage

Documentation Index

​Prerequisites

​Create a bucket with Cloud Console

​Create a bucket with CLI clients

​Versioned buckets

​Next steps

Prerequisites

Create a bucket with Cloud Console

Create a bucket with CLI clients

Versioned buckets

Next steps