Skip to main content

Get Started with AI Object Storage

This guide provides an high-level overview of the steps required to set up and use AI Object Storage.

Prerequisites​

  • Administrative access to your CoreWeave organization
  • A Cloud Console token with admin permissions

1. Configure Workload Identity Federation​

There are two methods to generate Access Keys for CoreWeave AI Object Storage:

  1. Using Cloud Console tokens
  2. Using SAML assertions with Workload Identity Federation

Workload Identity Federation provides a more secure, controlled way to generate time-limited Access Keys using SAML assertions, and is recommended for granting workloads access to AI Object Storage in production environments.

Before you can generate Access Keys for AI Object Storage using Workload Identity Federation, you must first configure Workload Identity Federation for your organization.

Get Started

How to: Manage Workload Identity Federation for AI Object Storage

2. Create Access Keys for workloads​

To grant a workload access to AI Object Storage, the workload must first have a valid Access Key.

Access Key generation using Workload Identity Federation is accomplished by submitting a CreateAccessKeysFromSAML request to the AI Object Storage API.

This request contains:

  1. A SAML assertion (usually generated by an IdP),
  2. a configId, generated from the Workload Identity Federation Console, and
  3. a duration parameter to specify the maximum lifespan of the Access Key.
Example
{

  "durationSeconds": 300,

  "orgId": "abc123",

  "configId": "<WORKLOAD_FEDERATION_CONFIG_ID>",

  "samlResponse": "<BASE64_ENCODED_SAML_RESPONSE>"

}
Info

For the most secure setup, using SAML assertions with Workload Identity Federation to generate workload Access Keys is strongly recommended. For less sensitive operations or for testing, the Cloud Console token method may be used.

Get Started

How To: Create Access Keys using Cloud Console Tokens
How To: Create Access Keys with Workload Identity Federation

3. Create a bucket​

After creating the Workload Identity Federation configuration, create a bucket to store data. Buckets are the primary organizational unit used by CoreWeave AI Object Storage to store object data.

You can create buckets using popular S3-compatible tools or the Cloud Console at these endpoints:

  • The primary endpoint is https://cwobject.com
  • The LOTA endpoint is http://cwlota.com

The LOTA endpoint uses HTTP protocol, rather than HTTPS.

4. Set policies​

You must create an access policy to be able to access the S3 API.

Also, before uploading data, you may want to set Bucket Policies for your organization and buckets. Policies are used to control access to data, control lifecycles, and configure auditing.

See these sections for more information:

5. Use objects and buckets​

CoreWeave AI Object Storage is compatible with the S3 API, so you can use any S3-compatible tool to manage your data. See How-To: Manage Objects to learn how to use CoreWeave AI Object Storage with s3cmd, Boto3, or the AWS CLI.

CoreWeave AI Object Storage is designed to work with LOTA (Local Object Transfer Accelerator), a first-of-its-kind intelligent proxy that provides a highly efficient, local gateway to CoreWeave AI Object Storage. See Concepts: LOTA for details about how LOTA works.

Info

CoreWeave AI Object Storage has default quota limits. See Manage quota limits for more information.

API reference and compatibility information​

Last updated on