Skip to main content

AI Object Storage S3 Compatibility

These are the supported API features and calls for CoreWeave AI Object Storage. They allow users to store, manage, retrieve, and manipulate objects stored in buckets.

Important

CoreWeave AI Object Storage compatibility with S3 features and the S3 API are subject to change.

S3 features​

CoreWeave AI Object Storage supports several important features for effective object management.

Authentication​

Every interaction with CoreWeave AI Object Storage is either authenticated or anonymous. When authenticated, AI Object Storage uses Signature Version 4 authentication.

See Using Signature Version 4 for details about using AWS CLI, Boto3, and s3cmd with Signature V4. The S3 Authenticating Requests guide also has more information.

Read After Write consistency​

CoreWeave AI Object Storage guarantees consistency. Objects can be read immediately after writing.

Bucket lifecycles​

A bucket lifecycle defines actions applied to a group of objects, such as expiring objects after a certain period of time. Use these API calls to manage lifecycles:

See Managing the lifecycle of objects for more information.

Bucket policy​

A bucket policy grants or denies permissions to the bucket and the objects within it. It defines what actions are allowed or denied on the bucket and its objects, and by whom. Bucket policies consist of one or more statements attached to a bucket in JSON format.

Use these API calls to manage bucket policies:

Versioning​

Versioning is a means of keeping multiple variants of an object in the same bucket. When enabled, all objects are assigned a unique version ID. This allows you to retrieve, restore, or delete specific versions of an object. Versioning is disabled by default. Use these API calls to manage versioning:

Other API calls are also version-aware, as noted in their descriptions below. See the AWS documentation for more information.

API calls​

Each API call requires permission to perform the related actions. The action names do not always match the API calls, and some perform multiple actions, which in turn require their related permission.

For example, copying an object requires permission to perform both s3:PutObject and s3:GetObject actions. When planning bucket policies, refer to the required permission listed in each API call description, and see Bucket Policy Reference for a complementary list of actions mapped to these API calls.

CoreWeave AI Object Storage supports standard S3 tools through the S3 API. See these resources for more information:

s3:AbortMultiPartUpLoad​

Requires permission to: s3:AbortMultipartUpload

Stop an in-progress multi-part upload (MPU).

No new parts may be uploaded to that MPU ID. Incomplete uploads are removed.

s3:CompleteMultiPartUpLoad​

Requires permission to: s3:PutObject

Complete a multi-part upload by listing all parts in the final object in ascending order.

Any parts uploaded but not specified in this call are discarded.

s3:CopyObject​

Requires permission to: s3:GetObject, s3:PutObject

Copy an object in object storage.

CoreWeave supports the x-amz-copy-source-if-match and x-amz-copy-source-if-none-match metadata headers for ETag matching.

s3:CreateBucket​

Requires permission to: s3:CreateBucket

Create a bucket with a valid CoreWeave AI Object Storage access key.

Anonymous requests are not allowed. The bucket creator becomes the owner. LocationConstraint must be supplied.

s3:CreateMultiPartUpLoad​

Requires permission to: s3:PutObject

Initiate a multi-part upload (MPU) and return an MPU ID. After starting a multi-part upload:

  • Upload parts with s3:UploadPart
  • Finalize the upload with s3:CompleteMultipartUpload
  • Stop the upload with s3:AbortMultipartUpload

s3:DeleteBucket​

Requires permission to: s3:DeleteBucket

Delete a bucket.

The bucket must be completely empty without any object versions or delete markers.

s3:DeleteBucketLifeCycle​

Requires permission to: s3:DeleteLifecycleConfiguration

Delete the lifecycle configuration from a bucket.

s3:DeleteBucketPolicy​

Requires permission to: s3:DeleteBucketPolicy

Remove a bucket policy from a bucket.

s3:DeleteBucketTagging​

Requires permission to: s3:DeleteBucketTagging

Delete tags from a bucket.

s3:DeleteObject​

Requires permission to: s3:DeleteObject

Remove an object, or specific version of an object, from a bucket.

s3:DeleteObjects​

Requires permission to: s3:DeleteObject, s3:DeleteObjectVersion

Delete up to 1000 objects per call.

Quiet mode only returns errors, while verbose mode returns a response for every object. Objects not found return success.

s3:DeleteObjectTagging​

Requires permission to: s3:DeleteObjectTagging

Remove all tags on an object.

s3:GetBucketACL​

Requires permission to: s3:ListBucket

Return the access control list of a bucket.

s3:GetBucketLifecycleConfiguration​

Requires permission to: s3:GetLifecycleConfiguration

Return the lifecycle configuration for a bucket.

s3:GetBucketLocation​

Requires permission to: s3:GetBucketLocation

Return the bucket's Region.

s3:GetBucketPolicy​

Requires permission to: s3:GetBucketPolicy

Return the policy for a bucket.

s3:GetBucketTagging​

Requires permission to: s3:GetBucketTagging

Get tags for a bucket.

s3:GetBucketVersioning​

Requires permission to: s3:GetBucketVersioning

Get the versioning state for the bucket, or empty if versioning is not enabled.

s3:GetObject​

Requires permission to: s3:GetObject

Retrieve an object.

Anonymous GET requests, Range reads, and ETag matching are supported.

When using LOTA, the GET request is cached by default, but can be modified with the Cache-Control HTTP header:

  • no-cache (Cache-Control: no-cache) queries LOTA for the object, but the GET request is not cached.
  • no-store (Cache-Control: no-store) does not query LOTA for the object, and the object is not cached.

s3:GetObjectAcl​

Requires permission to: s3:GetObject

Return the access control list of an object.

s3:GetObjectAttributes​

Requires permission to: s3:GetObject

Get the metadata of a specified object.

s3:GetObjectTagging​

Requires permission to: s3:GetObjectTagging

Return the set of tags associated with an object.

s3:HeadBucket​

Requires permission to: s3:ListBucket

Determine if a bucket exists and you have permission to access it.

Return codes​

  • 200 OK: The bucket exists and permission is allowed
  • 400 Bad Request: The bucket name is invalid
  • 403 Forbidden: Permission is denied
  • 404 Not Found: The bucket does not exist

s3:HeadObject​

Requires permission to: s3:GetObject

Get the metadata for the specified object.

s3:ListBuckets​

Requires permission to: s3:ListAllMyBuckets

Returns a list of all buckets owned by the authenticated sender of the request.

s3:ListObjectsV2​

Requires permission to: s3:ListBucket

List objects in a bucket.

Lists up to 1000 objects in a bucket per call, with ability to use ContinuationToken in the response to list more than 1000 objects.

s3:ListObjectVersions​

Requires permission to: s3:ListBucket

Return metadata on all versions or subsets of an object.

s3:ListParts​

Requires permission to: s3:ListMultipartUploadParts

Lists the uploaded parts for a specific multi-part upload.

s3:ListMultiPartUploads​

Requires permission to: s3:ListBucketMultipartUploads

List parts of a multi-part upload.

List up to 1000 in-progress multi-part uploads for a given bucket.

s3:PutBucketLifecycleConfiguration​

Requires permission to: s3:PutLifecycleConfiguration

Create or replace a new lifecycle configuration for a bucket.

This overwrites the previous configuration; it does not merge them.

s3:PutBucketPolicy​

Requires permission to: s3:PutBucketPolicy

Apply the provided policy to a bucket.

If no policy exists, access to create a new policy is granted if the user's Org ID matches the bucket's organization.

s3:PutBucketTagging​

Requires permission to: s3:PutBucketTagging

Set up to 10 tags for a bucket.

Keys can be up to 128 UTF-8 characters; values up to 256 characters.

s3:PutBucketVersioning​

Requires permission to: s3:PutBucketVersioning

Enable or suspend versioning for a bucket.

Important

If you use an object expiration lifecycle configuration on a non-versioned bucket and want to maintain the same permanent delete behavior after enabling versioning, you must add a separate lifecycle rule to delete noncurrent object versions.

s3:PutObject​

Requires permission to: s3:PutObject

Add an object to a bucket. The x-amz-meta and Expires headers are supported.

s3:PutObjectTagging​

Requires permission to: s3:PutObjectTagging

Set up to 10 tags on an object.

s3:UploadPart​

Requires permission to: s3:PutObject

Upload a part of a multi-part upload.

A multi-part upload can contain between 1 and 10,000 parts. A part number uniquely identifies each part and also defines its position within the object. If you reuse a part number, it overwrites the previously-uploaded part. Each part must be at least 5MB, except the last part.

s3:UploadPartCopy​

Requires permission to: s3:GetObject

Upload part of a multi-part upload, using an existing object as the source instead of providing new data.

You may also specify a range for the copied object.

Use Signature V4 with AWS CLI​

To use Signature V4 with the AWS CLI, set the configuration with aws:

AWS CLI
$
aws configure set default.s3.signature_version s3v4

Use Signature V4 with Boto3​

To configure Signature V4 with Boto3, set signature_version = s3v4 in the config file.

Use Signature V4 with S3Cmd​

Signature V4 is the default for S3cmd.

Important

s3cmd often simplifies its process by mapping commands to multiple actions. For this reason, a single S3cmd command may map to multiple different required permissions.

Last updated on