AI Object Storage S3 Compatibility

These are the supported API features and calls for CoreWeave AI Object Storage. They allow users to store, manage, retrieve, and manipulate objects stored in buckets.

Important

This list is subject to change as CoreWeave AI Object Storage develops.

s3cmd often simplifies its process by mapping commands to multiple actions. For this reason, a single S3cmd command may map to multiple different required permissions.

S3 features

The CoreWeave AI Object Storage supports several features that allow users to manage their objects effectively. These features include:

• Signature V4
• Bucket lifecycle configuration
• Bucket policy
• Read After Write consistency

For details about these features, usages, and available arguments, please see the AWS reference documentation, the S3 tools documentation, or the Boto3 Python SDK for AWS documentation.

These features are described in more detail below.

Signature V4

Every interaction with CoreWeave AI Object Storage is either authenticated or anonymous. This is version 4 of the authentication scheme used to sign requests. V4 is the default used by most setups. See the official Amazon documentation for more information.

Configure with AWS CLI

To use Signature V4 with the AWS CLI, set the configuration with aws:

AWS CLI

$
aws configure set default.s3.signature_version s3v4

Configure with Boto3

To configure Signature V4 with Boto3, set signature_version = s3v4 in the config file.

Configure with S3Cmd

Signature V4 is the default for S3cmd.

Bucket lifecycle configuration

In order to manage their objects so that they're stored effectively throughout their lifecycle, users create a Bucket Lifecycle configuration. A Lifecycle configuration is a set of rules that define actions that CoreWeave AI Object Storage applies to a group of objects. See the AWS docs for more information.

Bucket policy

A JSON object that defines access to operations and specific objects inside the bucket it has been assigned to.

Read After Write consistency

Objects can be read immediately after writing with guaranteed consistency. This is the default behavior for CoreWeave AI Object Storage.

API calls

Each API call requires permission to perform one or more related actions, and those action names do not always match the API calls. Some API calls perform multiple actions that also require permission. For example, copying an object requires permission to perform both s3:PutObject and s3:GetObject actions.

Use the permission mapping below when planning bucket policies. See the Bucket Policy Reference for a complementary list of actions mapped to their associated API calls.

API call	Required action permission
s3:AbortMultiPartUpLoad Stop an in-progress multi-part upload (MPU). No new parts may be uploaded to that MPU ID. Incomplete uploads are removed.	s3:AbortMultipartUpload
s3:CompleteMultiPartUpLoad Complete a multi-part upload by listing all parts in the final object in ascending order. Any parts uploaded but not specified in this call are discarded.	s3:PutObject
s3:CopyObject Copy an object in object storage. CoreWeave supports x-amz-copy-source-if-match and x-amz-copy-source-if-none-match headers for ETag matching.	s3:GetObject s3:PutObject
s3:CreateBucket Create a bucket with a valid CoreWeave AI Object Storage access key. Anonymous requests are not allowed. The bucket creator becomes the owner. LocationConstraint must be supplied.	s3:CreateBucket
s3:CreateMultiPartUpLoad Initiate a multi-part upload (MPU) and return an MPU ID, used to continue uploading parts via s3:UploadPart, finalize with s3:CompleteMultipartUpload, or stop with s3:AbortMultipartUpload.	s3:PutObject
s3:DeleteBucket Delete a bucket, which must be completely empty without any object versions or delete markers.	s3:DeleteBucket
s3:DeleteBucketLifeCycle Delete the lifecycle configuration from a bucket.	s3:DeleteLifecycleConfiguration
s3:DeleteBucketPolicy Remove a bucket policy from a bucket.	s3:DeleteBucketPolicy
s3:DeleteBucketTagging Delete tags from a bucket.	s3:DeleteBucketTagging
s3:DeleteObject Remove an object, or specific version of an object, from a bucket.	s3:DeleteObject
s3:DeleteObjects Delete up to 1000 objects per call. Quiet mode only returns errors, while verbose mode returns a response for every object. Objects not found return success.	s3:DeleteObject s3:DeleteObjectVersion
s3:DeleteObjectTagging Remove all tags on an object.	s3:DeleteObjectTagging
s3:GetBucketACL Return the access control list of a bucket.	s3:ListBucket
s3:GetBucketLifecycleConfiguration Return the lifecycle configuration for a bucket.	s3:GetLifecycleConfiguration
s3:GetBucketLocation Return the bucket's Region.	s3:GetBucketLocation
s3:GetBucketPolicy Return the policy for a bucket.	s3:GetBucketPolicy
s3:GetBucketTagging Get tags for a bucket.	s3:GetBucketTagging
s3:GetBucketVersioning Get the versioning state for the bucket, or empty if versioning is not enabled.	s3:GetBucketVersioning
s3:GetObject Retrieve an object. Anonymous GETS, Range reads, and ETag matching is supported. By default, the GET request will be cached when using the LOTA endpoint, however, the Cache-Control HTTP header may be used to alter the caching behavior of a GET request. Specifying no-cache (Cache-Control: no-cache) alters the behavior such that other LOTA instances are still queried for the object, but the GET request is not cached. Using no-store (Cache-Control: no-store) results in the object being retrieved directly, without other LOTA instances being queried, and the object is not cached.	s3:GetObject
s3:GetObjectAcl Return the access control list of an object.	s3:GetObject
s3:GetObjectAttributes Get the metadata of an specified object.	s3:GetObject
s3:GetObjectTagging Return the set of tags associated with a bucket.	s3:GetObjectTagging
s3:HeadBucket Determine if a bucket exists and you have permission to access it. The action returns a 200 OK if the bucket exists and you have permission to access it. Otherwise, the request returns 400 Bad Request, 403 Forbidden, or 404 Not Found.	s3:ListBucket
s3:HeadObject Get the metadata for the specified object.	s3:GetObject
s3:ListBuckets Returns a list of all buckets owned by the authenticated sender of the request.	s3:ListAllMyBuckets
s3:ListObjectsV2 Lists up to 1000 objects in a bucket per call, with ability to use ContinuationToken in the response to list more than 1000 objects.	s3:ListBucket
s3:ListObjectVersions Return metadata on all versions or subsets of an object.	s3:ListBucket
s3:ListParts Lists the uploaded parts for a specific multi-part upload.	s3:ListMultipartUploadParts
s3:ListMultiPartUploads List up to 1000 in-progress multi-part uploads for a given bucket.	s3:ListBucketMultipartUploads
s3:PutBucketLifecycleConfiguration Create or replace a new lifecycle configuration for a bucket. This overwrites the previous configuration; it does not merge them.	s3:PutLifecycleConfiguration
s3:PutBucketPolicy Apply the provided policy to a bucket. If no policy exists, access to create a new policy is granted if the user's Org ID matches the bucket's organization.	s3:PutBucketPolicy
s3:PutBucketTagging Set up to 10 tags for a bucket. Keys are limited to 128 UTF-8 characters, while values are limited to 256 characters.	s3:PutBucketTagging
s3:PutBucketVersioning Enable or suspend versioning for a bucket. Please note: if you have an object expiration lifecycle configuration in your non-versioned bucket and you want to maintain the same permanent delete behavior when you enable versioning, you must add another lifecycle rule to also remove noncurrent versions.	s3:PutBucketVersioning
s3:PutObject Add an object to a bucket. The x-amz-meta and Expires headers are supported.	s3:PutObject
s3:PutObjectTagging Set up to 10 tags on an object.	s3:PutObjectTagging
s3:UploadPart Upload part of a multi-part upload that contains between 1 and 10,000 parts. A part number uniquely identifies each part and also defines its position within the object. If you reuse a part number, it overwrites the previously-uploaded part. Each part must be at least 5MB, except the last part.	s3:PutObject
s3:UploadPartCopy Upload part of a multi-part upload, using an existing object as the source instead of providing new data. You may also specify a range for the copied object.	s3:GetObject