Create buckets - CoreWeave Docs

Buckets are the primary organizational unit that stores object data in CoreWeave AI Object Storage. They’re similar to folders in a file system. This guide explains how to create a bucket with the CoreWeave Cloud Console or CLI clients such as S3cmd, AWS CLI, and Boto3. You can also use the CoreWeave fork of s5cmd. Upstream s5cmd uses path-style addressing, which AI Object Storage doesn’t support.

Prerequisites

You must generate an Access Key before you create a bucket. When you use the Cloud Console, use Cloud Console tokens to generate the Access Key. Workload Identity Federation is supported with CLI clients.
To create a bucket with the Cloud Console, you must have s3:CreateBucket and cwobject:CreateAccessKey permissions granted to your user or group by an organization access policy.
Make sure your configuration uses the correct endpoint URL and has virtual-hosted addressing enabled.

Create a bucket with Cloud Console

To create a bucket with the CoreWeave Cloud Console, follow these steps:

In the Cloud Console, navigate to the Buckets page.
Click the Create Bucket button at the top right corner of the page.
In the dialog, enter a Bucket Name.
Bucket naming rules
Bucket names must be globally unique and adhere to the following rules:
- Length: 3 to 63 characters.
- Characters: Only lowercase letters (a-z), numbers (0-9), and hyphens (-). No dots, uppercase letters, underscores, spaces, or other special characters.
- Start and end: Must begin and end with a letter or number. Cannot start or end with a hyphen (-).
- Prohibited patterns: Cannot start with xn--.
- Reserved: Must not begin with cw-, vip-, or log-stitcher-ch-. Must not be the exact name int. CoreWeave reserves these for internal use.
Select an Availability Zone.
Click Create.
Click More options to view information in Grafana, or to delete the bucket.

Create a bucket with CLI clients

CoreWeave AI Object Storage is compatible with S3-compatible clients, including S3cmd, the AWS CLI, Boto3, and the CoreWeave fork of s5cmd.

Use the CoreWeave fork of s5cmd with AI Object StorageThe upstream s5cmd uses path-style addressing with custom endpoint URLs. AI Object Storage doesn’t support path-style addressing and requires virtual-hosted style URLs. Use the CoreWeave fork of s5cmd, which is compatible with AI Object Storage and can safely replace any existing s5cmd installation. For setup and usage, see Migrate data to AI Object Storage.

Before you create a bucket, configure your client. The primary endpoint, https://cwobject.com, requires TLS v1.3. Make sure your S3-compatible tools and OpenSSL use TLS v1.3. The following sections describe how to create a bucket with each supported CLI client. Replace these placeholders in the examples:

[BUCKET-NAME]: A globally unique name for your bucket. Follow the naming rules:
Bucket naming rules
Bucket names must be globally unique and adhere to the following rules:
- Length: 3 to 63 characters.
- Characters: Only lowercase letters (a-z), numbers (0-9), and hyphens (-). No dots, uppercase letters, underscores, spaces, or other special characters.
- Start and end: Must begin and end with a letter or number. Cannot start or end with a hyphen (-).
- Prohibited patterns: Cannot start with xn--.
- Reserved: Must not begin with cw-, vip-, or log-stitcher-ch-. Must not be the exact name int. CoreWeave reserves these for internal use.
[AVAILABILITY-ZONE]: A CoreWeave Availability Zone that supports AI Object Storage (for example, US-EAST-04A). S3-compatible tools use the term “region” to define the location of the storage service, but at CoreWeave, Regions are larger constructs organized into Availability Zones. Use a CoreWeave Availability Zone that supports AI Object Storage:
Availability Zones that support AI Object Storage
- US-CENTRAL
- US-EAST
- US-WEST
- CA-EAST
- EU-NORTH
- EU-SOUTH
US-CENTRAL-05A

US-CENTRAL-06A

US-CENTRAL-07A

US-CENTRAL-08A

US-CENTRAL-08B

US-EAST-01A

US-EAST-02A

US-EAST-03A

US-EAST-04A

US-EAST-04B

US-EAST-06A

US-EAST-08A

US-EAST-13A

US-EAST-14A

RNO2A

US-WEST-01A

US-WEST-04A

US-WEST-09B

US-WEST-10A

CA-EAST-01A

EU-NORTH-05A

EU-SOUTH-03B

EU-SOUTH-04A
Learn more about Regions and Availability Zones.

Bucket creation delayWhen you create a bucket with an S3-compatible client, it takes about one minute before the bucket becomes available due to DNS caching. You might see the following error immediately after bucket creation: An error occurred (InvalidRegion) when calling the ListObjects operation: Region does not match. Wait one minute, then try again.This delay doesn’t occur when you create buckets in the Cloud Console.

AWS CLI
s3cmd
Boto3

To create a bucket with AWS CLI, use the aws s3api command. The LocationConstraint parameter is required.

Create a bucket with AWS CLI

aws s3api create-bucket \
    --bucket [BUCKET-NAME] \
    --region [AVAILABILITY-ZONE] \
    --create-bucket-configuration LocationConstraint=[AVAILABILITY-ZONE]

If you need to install or configure the AWS CLI for AI Object Storage, see Manage buckets with AWS CLI.

To create a bucket with S3cmd, use the mb (make bucket) subcommand. The --bucket-location parameter is required.

Create a bucket with S3cmd

s3cmd mb --bucket-location=[AVAILABILITY-ZONE] s3://[BUCKET-NAME]

If you need to configure S3cmd for AI Object Storage, see Manage buckets with S3cmd.

To create a bucket with Boto3, use the create_bucket method. The LocationConstraint parameter is required.

Set environment variables for your CoreWeave credentials:
```
export ACCESS_KEY_ID="[ACCESS-KEY-ID]"
export SECRET_ACCESS_KEY="[SECRET-ACCESS-KEY]"
```
Alternatively, configure your CoreWeave credentials to work with the AWS CLI. We recommend using a separate profile for CoreWeave AI Object Storage to avoid conflicts with your other AWS profiles and S3-compatible services. If you don’t set up this configuration, you might encounter errors when using AI Object Storage.
Configure CoreWeave credentials
1. Create a new credentials file and profile in your CoreWeave configuration directory.
  Create a new credentials file and profile
  AWS_SHARED_CREDENTIALS_FILE=~/.coreweave/cw.credentials aws configure --profile cw
2. When prompted, provide the following values:
  - AWS Access Key ID: The Access Key ID of your CoreWeave AI Object Storage Access Key.
  - AWS Secret Access Key: The Secret Key of your CoreWeave AI Object Storage Access Key.
  - Default region name (Optional): To set a default region, see CoreWeave Availability Zones.
  - Default output format: Use json for JSON output.
3. Set the default endpoint URL to the appropriate endpoint for your use case:
  - The primary endpoint, https://cwobject.com, for use outside a CoreWeave cluster.
  - The LOTA endpoint, http://cwlota.com, for use inside a CoreWeave cluster. The LOTA endpoint routes to the LOTA path for best performance.
  Set the primary endpoint for local development
  AWS_CONFIG_FILE=~/.coreweave/cw.config aws configure set endpoint_url https://cwobject.com --profile cw
4. Set the S3 addressing_style to virtual:
  Set virtual addressing style
  AWS_CONFIG_FILE=~/.coreweave/cw.config aws configure set s3.addressing_style virtual --profile cw

Replace [BUCKET-NAME] with the name of the bucket and [AVAILABILITY-ZONE] with a CoreWeave Availability Zone.

Create a bucket with Boto3

import os
import boto3
from botocore.client import Config

boto_config = Config(
    region_name='US-EAST-04A',
    s3={'addressing_style': 'virtual'}
)

s3 = boto3.client(
    's3',
    endpoint_url='https://cwobject.com',
    aws_access_key_id=os.environ['ACCESS_KEY_ID'],
    aws_secret_access_key=os.environ['SECRET_ACCESS_KEY'],
    config=boto_config
)

response = s3.create_bucket(
    Bucket='[BUCKET-NAME]',
    CreateBucketConfiguration={
        'LocationConstraint': '[AVAILABILITY-ZONE]',
    },
)

print(response)

If you need to configure Boto3 for AI Object Storage, see Manage buckets with Boto3.

Versioned buckets

If you need a versioned bucket, you must enable versioning at creation time. You can’t enable versioning on an existing bucket after you create it, so decide whether you need versioning before you create the bucket. To create and configure a versioned bucket, see Versioned buckets.

Next steps

After you create a bucket, you must set an organization access policy to control access to the Object Storage S3-compatible API. Without an access policy, users and applications can’t interact with the bucket.

​Prerequisites

​Create a bucket with Cloud Console

​Create a bucket with CLI clients

​Versioned buckets

​Next steps

Prerequisites

Create a bucket with Cloud Console

Create a bucket with CLI clients

Versioned buckets

Next steps