Prerequisites
This guide assumes you’ve already met the following prerequisites:- You’ve already created a source bucket that you want to inventory.
- You have permissions to create a destination bucket for the output.
- You have permissions to set bucket access policies on the destination bucket.
- Set up an access key
- Set up an organization access policy, which is required before you can create a bucket.
- Create a bucket to inventory.
Choose a destination bucket
The destination bucket is where inventory reports are written. You can create a separate destination bucket for your inventory output, or use the same bucket as the source bucket. The rest of this guide refers to both options as the “destination bucket”. To use a separate destination bucket for your inventory reports, create a new bucket. When you create the bucket, choose a name that follows the bucket naming rules.Bucket naming rules
Bucket naming rules
Bucket names must be globally unique and adhere to the following rules:
- Length: 3 to 63 characters.
- Characters: Only lowercase letters (
a-z), numbers (0-9), and hyphens (-). No dots, uppercase letters, underscores, spaces, or other special characters. - Start and end: Must begin and end with a letter or number. Cannot start or end with a hyphen (
-). - Prohibited patterns: Cannot start with
xn--. - Reserved: Must not begin with
cw-,vip-, orlog-stitcher-ch-. Must not be the exact nameint. CoreWeave reserves these for internal use.
[DESTINATION-BUCKET] with the name for your destination bucket and [AVAILABILITY-ZONE] with the Availability Zone to create it in.
Set bucket access policies
To enable inventory reporting, you must grant the CoreWeave inventory service permission to write reports to your destination bucket. This requires setting a bucket access policy on the destination bucket. This policy is required whether you use a separate destination bucket or write reports to the same bucket as your source data. The following permissions are required:-
Allow the CoreWeave inventory service account to write to the destination bucket. This is a CoreWeave-managed service account that generates and writes inventory reports to your destination bucket. On a bucket access policy for the destination bucket, grant write permission (
s3:PutObject) to the service account with the ARNarn:aws:iam::static:role/static/inventory. -
Allow the user or entity that manages the bucket and accesses the reports to read and write to the destination bucket. The user configuring the inventory report needs the following permissions on the source bucket to apply the inventory configuration:
s3:PutInventoryConfigurations3:ListInventoryConfigurationss3:GetInventoryConfigurations3:DeleteInventoryConfiguration
Create source bucket policy
The following example bucket policy allows the entity requesting the inventory configuration to apply and manage the inventory configuration on the source bucket.-
Create a source bucket policy file.
Replace
[SOURCE-BUCKET]with the name of the bucket you want to inventory,[ORG-ID]with your organization ID, and[USER-UID]with the UID of the user managing the inventory configuration.source-bucket-policy.json -
Apply the source bucket policy:
Create destination bucket policy
The following example bucket policy allows the CoreWeave inventory service account to write inventory reports to the destination bucket, and the user or entity that issues the inventory list request to access the destination bucket. Create a destination bucket policy file. Replace[DESTINATION-BUCKET] with the name of your destination bucket, [ORG-ID] with your organization ID, and [USER-UID] with the UID of the user managing the reports.
destination-bucket-policy.json
- The first statement,
AllowServiceAccountWriteReportsToDestination, allows the CoreWeave inventory service account to write to the destination bucket. ThePrincipalfield is set toarn:aws:iam::static:role/static/inventory, which is a CoreWeave-managed service account that generates and writes inventory reports to your destination bucket. - The second statement,
AllowOwnerAccessToDestination, allows the entity to access the destination bucket. ThePrincipalfield is set to the ARN of the user who reads and manages the inventory reports.
Apply source and destination bucket policies
With both policy files prepared, apply them to their respective buckets so the inventory service and the user managing the configuration have the access they need:Use the same bucket for source and destination
Alternatively, to write inventory reports to the same bucket as the source bucket, grant access to the service account to write to the source bucket. You still need the inventory configuration permissions. If you created the bucket, you likely already have these permissions.-
Create the following bucket policy file:
Replacesame-bucket-policy.json
[SOURCE-BUCKET]with the name of your source bucket. -
Apply this policy to the source bucket:
Configure the inventory report
With the destination bucket and access policies in place, you can now configure the inventory report itself. To configure inventory reporting, you specify:- A source bucket to inventory. You can inventory all the objects in the bucket or only prefixes that you specify. If you don’t specify a prefix, all objects are inventoried.
- A destination bucket where reports are written. You can use a separate destination bucket or the same bucket as the source bucket.
- An optional prefix (subdirectory) within the destination bucket for organizing reports. If you don’t specify a prefix, the reports are written to the root of the destination bucket. Use a prefix (like
inventory-reports/) in your inventory configuration to separate reports from your source data.
aws s3api. The configuration is a JSON object that defines the inventory configuration for the bucket.
The following fields are available:
| Field | Description | Note |
|---|---|---|
Destination.S3BucketDestination | Configures destination bucket for inventory report | |
Destination.S3BucketDestination.Bucket | Destination bucket name, can be same as source bucket | Example: [DESTINATION-BUCKET] |
Destination.S3BucketDestination.Format | Report object format | Accepts: CSV, TSV, JSON, ORC, Parquet |
Destination.S3BucketDestination.Prefix | Prefix for report objects | Example: inventory/ Inventory report objects are created with inventory/ prefix in the object key. |
IsEnabled | Controls enablement of the inventory configuration | Accepts: true or false |
Filter | Filters source objects to include in inventory report | |
Filter.Prefix | Source objects’ prefix | Example: documents/ Only objects with prefix documents/ are included in the report. |
Id | Inventory configuration ID | Example: my-inventory-config |
IncludedObjectVersions | Controls which object versions are included in the inventory report | Accepts: All or Latest - All: Include all versions- Latest: Only latest versions |
OptionalFields | Controls additional information to include in the report | Example: ["Size", "LastAccessedDate"] Inventory report then has fields: - BucketName- ObjectKey- Size- LastAccessedDate |
Schedule | Controls the schedule of inventory reporting | |
Schedule.Frequency | Sets schedule frequency | Accepts: Daily or Weekly |
-
Create a bucket inventory configuration file. The following is an example bucket inventory configuration:
Replace
[DESTINATION-BUCKET]with the name of the bucket where inventory reports are written.bucket-inventory-config.json -
Apply the bucket inventory configuration:
Replace
[SOURCE-BUCKET]with the name of the bucket you want to inventory.
Manage inventory report configurations
After an inventory configuration is in place, you can update, inspect, or remove it as your needs change. You can manage your inventory report configurations using the S3 API with standard S3 tools. The following is an example of how to create, update, get, and delete inventory report configurations using the AWS CLI: Replace[SOURCE-BUCKET] with the name of the bucket you’re inventorying.
get command, you might see output like this:
bucket-inventory-config.json
Use Terraform
If you manage your infrastructure as code, you can use the AWS Terraform provider to manage inventory report configurations alongside your other resources. The following example declares the destination bucket policy, source and destination buckets, and the inventory configuration as Terraform resources.Check your inventory report
After you apply your inventory configuration, allow some time for the inventory report to be generated and written to the destination bucket, then check the path where you configured the inventory report to be written. No notification is sent when the inventory report completes, so you must check the destination bucket to confirm the report is ready. When the inventory report completes, you should see amanifest.json file written to the root of the subdirectory specified in the inventory configuration, and one or more inventory data files. You can use the manifest file to understand the structure of the inventory report. See Manifest file.
Disable inventory reporting
If you no longer need inventory reports for a bucket, you can stop generating new reports without losing the configuration’s history. To disable inventory reporting, either modify the inventory configuration to setIsEnabled to false, or delete the inventory configuration.
The following example shows a modified inventory configuration file that disables inventory reporting:
modified-inventory-config.json