S3 features
CoreWeave AI Object Storage supports several features for object management.Authentication
Every interaction with CoreWeave AI Object Storage is either authenticated or anonymous. When authenticated, AI Object Storage uses Signature Version 4 authentication. For details about using AWS CLI, Boto3, and s3cmd with Signature V4, see Using Signature Version 4. For more information, see the S3 Authenticating Requests guide.Read after write consistency
CoreWeave AI Object Storage guarantees consistency. Objects can be read immediately after writing.Bucket lifecycles
A bucket lifecycle defines actions applied to a group of objects, such as expiring objects after a set period. Use these API calls to manage lifecycles: For more information, see Managing the lifecycle of objects.Bucket access policy
A bucket access policy grants or denies permissions to the bucket and the objects within it. It defines which actions are allowed or denied on the bucket and its objects, and by whom. Bucket access policies consist of one or more statements attached to a bucket in JSON format. Use these API calls to manage bucket access policies:Versioning
Versioning keeps multiple variants of an object in the same bucket. When enabled, all objects are assigned a unique version ID. This lets you retrieve, restore, or delete specific versions of an object. Versioning is disabled by default. Use these API calls to manage versioning: Other API calls are also version-aware, as noted in their descriptions in the following sections. For more information, see the AWS documentation.Presigned URLs
Presigned URLs let you grant temporary access to an object in a bucket without requiring the user to have access credentials. You can generate a presigned URL for any S3 API call that requires authentication, such ass3:GetObject or s3:PutObject. When a user accesses the presigned URL, they can perform the specified action on the object without authenticating with their own credentials.
For more information, see the S3 documentation.
Checksum algorithms
CoreWeave AI Object Storage supports the same checksum algorithms as AWS S3 for verifying object integrity on upload and download:| Algorithm | Value to use in requests |
|---|---|
| CRC-64/NVME | CRC64NVME |
| CRC-32 | CRC32 |
| CRC-32C | CRC32C |
| SHA-1 | SHA1 |
| SHA-256 | SHA256 |
Conditional writes
CoreWeave AI Object Storage supports conditional writes fors3:PutObject, s3:CompleteMultiPartUpLoad, s3:CopyObject, and s3:RenameObject. Conditional writes use HTTP precondition headers to make writes atomic and to prevent accidental overwrites.
| Header | Accepted value | Behavior |
|---|---|---|
If-None-Match | * | Write only if no object exists at the key. Returns 412 Precondition Failed if the object exists. |
If-Match | ETag string | Write only if the object’s current ETag matches. Returns 412 Precondition Failed if the ETag doesn’t match. |
If-None-Matchonly accepts the value*. Passing an ETag value returns400 InvalidRequest.- When two concurrent requests use
If-None-Match: *for the same key, one succeeds and the other returns409 ConditionalRequestConflict. Retry on409.
s3:RenameObject supports a richer set of conditional headers, including separate source and destination preconditions and time-based conditions, in addition to If-None-Match and If-Match. For details, see Preconditions and safeguards.
For the HTTP conditional request specification, see RFC 7232. For usage examples, including the read-then-write workflow for If-Match, see Use conditional requests.
API calls
The following sections list the supported S3 API calls, the permissions each one requires, and any CoreWeave-specific behavior. Each API call requires permission to perform the related actions. The action names don’t always match the API calls, and some perform multiple actions, which in turn require their related permission. For example, copying an object requires permission to perform boths3:PutObject and s3:GetObject actions. When you plan bucket access policies, refer to the required permission listed in each API call description. For a complementary list of actions mapped to these API calls, see Bucket Access Policies.
CoreWeave AI Object Storage supports standard S3 tools through the S3 API. For more information, see these resources:
s3:AbortMultiPartUpLoad
Requires permission to: s3:AbortMultipartUpload
Stop an in-progress multi-part upload (MPU).
No new parts can be uploaded to that MPU ID. Incomplete uploads are removed.
s3:CompleteMultiPartUpLoad
Requires permission to: s3:PutObject
Complete a multi-part upload by listing all parts in the final object in ascending order.
Any parts that are uploaded but not specified in this call are discarded.
Conditional write headers (If-None-Match, If-Match) are supported. See Conditional writes.
s3:CopyObject
Requires permission to: s3:GetObject, s3:PutObject
Copy an object in object storage.
The following conditional write headers are supported:
- Source precondition headers (
x-amz-copy-source-if-match,x-amz-copy-source-if-none-match,x-amz-copy-source-if-modified-since,x-amz-copy-source-if-unmodified-since) - Destination conditional write headers (
If-Match,If-None-Match)
s3:CreateBucket
Requires permission to: s3:CreateBucket
Create a bucket with a valid CoreWeave AI Object Storage access key.
Anonymous requests aren’t allowed. The bucket creator becomes the owner. LocationConstraint must be supplied.
Bucket naming rules
Bucket naming rules
Bucket names must be globally unique and adhere to the following rules:
- Length: 3 to 63 characters.
- Characters: Only lowercase letters (
a-z), numbers (0-9), and hyphens (-). No dots, uppercase letters, underscores, spaces, or other special characters. - Start and end: Must begin and end with a letter or number. Cannot start or end with a hyphen (
-). - Prohibited patterns: Cannot start with
xn--. - Reserved: Must not begin with
cw-,vip-, orlog-stitcher-ch-. Must not be the exact nameint. CoreWeave reserves these for internal use.
s3:CreateMultiPartUpLoad
Requires permission to: s3:PutObject
Initiate a multi-part upload (MPU) and return an MPU ID. After starting a multi-part upload:
- Upload parts with
s3:UploadPart - Finalize the upload with
s3:CompleteMultipartUpload - Stop the upload with
s3:AbortMultipartUpload
s3:DeleteBucket
Requires permission to: s3:DeleteBucket
Delete a bucket.
The bucket must be completely empty without any object versions or delete markers.
s3:DeleteBucketLifeCycle
Requires permission to: s3:DeleteLifecycleConfiguration
Delete the lifecycle configuration from a bucket.
s3:DeleteBucketPolicy
Requires permission to: s3:DeleteBucketPolicy
Remove a bucket access policy from a bucket.
s3:DeleteBucketTagging
Requires permission to: s3:DeleteBucketTagging
Delete tags from a bucket.
s3:DeleteObject
Requires permission to: s3:DeleteObject, and s3:DeleteObjectVersion when a versionId is specified
Remove an object, or specific version of an object, from a bucket.
s3:DeleteObjects
Requires permission to: s3:DeleteObject, s3:DeleteObjectVersion
Delete up to 1,000 objects per call.
Quiet mode returns only errors, while verbose mode returns a response for every object. Objects not found return success.
s3:DeleteObjectTagging
Requires permission to: s3:DeleteObjectTagging
Remove all tags on an object.
s3:GetBucketACL
Requires permission to: s3:ListBucket
Return the access control list of a bucket.
s3:GetBucketLifecycleConfiguration
Requires permission to: s3:GetLifecycleConfiguration
Return the lifecycle configuration for a bucket.
s3:GetBucketLocation
Requires permission to: s3:GetBucketLocation
Return the bucket’s Region.
s3:GetBucketPolicy
Requires permission to: s3:GetBucketPolicy
Return the policy for a bucket.
s3:GetBucketTagging
Requires permission to: s3:GetBucketTagging
Get tags for a bucket.
s3:GetBucketVersioning
Requires permission to: s3:GetBucketVersioning
Get the versioning state for the bucket, or empty if versioning is not enabled.
s3:GetObject
Requires permission to: s3:GetObject
Retrieve an object.
Anonymous GET requests, Range reads, and ETag matching are supported.
When using LOTA, the GET request is cached by default, but can be modified with the Cache-Control HTTP header:
no-cache(Cache-Control: no-cache) queries LOTA for the object, but theGETrequest is not cached.no-store(Cache-Control: no-store) does not query LOTA for the object, and the object is not cached.
s3:GetObjectAcl
Requires permission to: s3:GetObject
Return the access control list of an object.
s3:GetObjectAttributes
Requires permission to: s3:GetObject
Get the metadata of a specified object.
s3:GetObjectTagging
Requires permission to: s3:GetObjectTagging
Return the set of tags associated with an object.
s3:HeadBucket
Requires permission to: s3:ListBucket
Determine if a bucket exists and you have permission to access it.
Return codes
200 OK: The bucket exists and permission is allowed.400 Bad Request: The bucket name is invalid.403 Forbidden: Permission is denied.404 Not Found: The bucket doesn’t exist.
s3:HeadObject
Requires permission to: s3:GetObject
Get the metadata for the specified object.
s3:ListBuckets
Requires permission to: s3:ListAllMyBuckets
Return a list of all buckets owned by the authenticated sender of the request.
s3:ListObjectsV2
Requires permission to: s3:ListBucket
List objects in a bucket.
Lists up to 1,000 objects in a bucket per call. Use ContinuationToken in the response to list more than 1,000 objects.
s3:ListObjectVersions
Requires permission to: s3:ListBucket
Return metadata on all versions or subsets of an object.
s3:ListParts
Requires permission to: s3:ListMultipartUploadParts
List the uploaded parts for a specific multi-part upload.
s3:ListMultiPartUploads
Requires permission to: s3:ListBucketMultipartUploads
List parts of a multi-part upload.
List up to 1,000 in-progress multi-part uploads for a given bucket.
s3:PutBucketLifecycleConfiguration
Requires permission to: s3:PutLifecycleConfiguration
Create or replace a new lifecycle configuration for a bucket.
This overwrites the previous configuration. It doesn’t merge them.
s3:PutBucketPolicy
Requires permission to: s3:PutBucketPolicy
Apply the provided policy to a bucket.
If no policy exists, access to create a new policy is granted when the user’s Org ID matches the bucket’s organization.
s3:PutBucketTagging
Requires permission to: s3:PutBucketTagging
Set up to 10 tags for a bucket.
Keys can be up to 128 UTF-8 characters. Values can be up to 256 characters.
s3:PutBucketVersioning
Requires permission to: s3:PutBucketVersioning
Enable or suspend versioning for a bucket.
s3:PutObject
Requires permission to: s3:PutObject
Add an object to a bucket. The x-amz-meta and Expires headers are supported. An ETag is returned for every uploaded object.
Conditional write headers (If-None-Match, If-Match) are supported. See Conditional writes.
s3:PutObjectTagging
Requires permission to: s3:PutObjectTagging
Set up to 10 tags on an object.
s3:RenameObject
Requires permission to: s3:PutObject and s3:DeleteObject
Rename an object, subject to the following constraints:
- The source and destination keys must be in the same bucket.
- The bucket must not have versioning enabled, either currently or in the past.
- Each request supports one rename operation for one object key. Bulk or prefix-level rename operations aren’t supported.
| Parameter | Description |
|---|---|
| Bucket | The bucket contains both the source and destination keys. |
| Source key | The current object key (for example, checkpoints/model.ckpt.tmp). |
| Destination key | The new object key in the same bucket (for example, checkpoints/model.ckpt). |
| Idempotency token | An optional client-supplied token that makes the operation idempotent across retries. The same token and parameters don’t produce multiple renames. |
s3:UploadPart
Requires permission to: s3:PutObject
Upload a part of a multi-part upload.
A multi-part upload can contain between 1 and 10,000 parts. A part number uniquely identifies each part and defines its position within the object. If you reuse a part number, it overwrites the previously uploaded part. Each part must be at least 5 MB, except the last part.
s3:UploadPartCopy
Requires permission to: s3:GetObject, s3:PutObject
Upload part of a multi-part upload, using an existing object as the source instead of providing new data.
You can also specify a range for the copied object.
Use Signature V4 with AWS CLI
To use Signature V4 with the AWS CLI, set the configuration withaws:
AWS CLI
Use Signature V4 with Boto3
To configure Signature V4 with Boto3, setsignature_version = s3v4 in the config file.
Use Signature V4 with S3Cmd
Signature V4 is the default for S3cmd.Server-side encryption with customer-provided keys (SSE-C)
CoreWeave AI Object Storage supports server-side encryption with customer-provided keys (SSE-C). SSE-C lets you encrypt your data using keys that you provide and manage. The following sections describe the required request headers, example requests for upload, download, and copy, and the response headers and error codes you can expect.SSE-C headers
When using SSE-C, include the following headers in your requests:x-amz-server-side-encryption-customer-algorithm: Must be set toAES256.x-amz-server-side-encryption-customer-key: Your 256-bit (32-byte) encryption key in base64-encoded format.x-amz-server-side-encryption-customer-key-md5: MD5 hash of your encryption key in base64-encoded format.
Upload with SSE-C
Replace$BASE64_KEY with your base64-encoded encryption key. Replace [BUCKET-NAME] with the bucket name and [OBJECT-KEY] with the object key.
Download with SSE-C
Copy with SSE-C
Replace[SOURCE-BUCKET-NAME] and [SOURCE-OBJECT-KEY] with the source bucket name and object key. Replace [DEST-BUCKET-NAME] and [DEST-OBJECT-KEY] with the destination bucket name and object key.
SSE-C response headers
When using SSE-C, the following headers are returned in responses:x-amz-server-side-encryption-customer-algorithm: The encryption algorithm used.x-amz-server-side-encryption-customer-key-md5: MD5 hash of the encryption key used.
SSE-C error codes
Common SSE-C error codes:InvalidArgument: The encryption key is not 256 bits (32 bytes).InvalidRequest: The encryption key MD5 hash doesn’t match.AccessDenied: The provided encryption key doesn’t match the one used for upload.