Configure Inventory Reporting for CoreWeave AI Object Storage

How to configure inventory reports for CoreWeave AI Object Storage buckets and objects

This guide explains how to configure and enable inventory reports for CoreWeave AI Object Storage buckets and objects.

Prerequisites

This guide assumes you have already met the following prerequisites:

• You have already created a source bucket that you want to inventory.
• You have permissions to create a destination bucket for the output.
• You have permissions to set bucket access policies on the destination bucket.

If you need to revisit any of these prerequisites, you can do so by following the links to the relevant guides:

• Set up an access key
• Set up an organization access policy, which is required before you can create a bucket.
• Create a bucket to inventory.

Choose a destination bucket

You can choose to create a separate destination bucket for your inventory output, or use the same bucket as the source bucket. The rest of this guide refers to both options as the "destination bucket".

If you want to use a separate destination bucket for your inventory reports, create a new bucket:

Example

# Example variables for creating inventory destination bucket
export DESTINATION_BUCKET_NAME="my-destination-bucket"
export AVAILABILITY_ZONE="us-east-03a"

# Create inventory destination bucket
$
aws s3api create-bucket \
--bucket ${DESTINATION_BUCKET_NAME} \
--create-bucket-configuration LocationConstraint=${AVAILABILITY_ZONE}

Inventory reporting is available in all AI Object Storage regions.

Set bucket access policies

To enable inventory reporting, you must grant the CoreWeave inventory service permission to write reports to your destination bucket. This requires setting a bucket access policy on the destination bucket. This policy is required whether you use a separate destination bucket or write reports to the same bucket as your source data.

The following permissions are required:

1. Allow the CoreWeave inventory service account to write to the destination bucket. This is a CoreWeave-managed service account that generates and writes inventory reports to your destination bucket. On a bucket access policy for the destination bucket, grant write permission (s3:PutObject) to the service account with the ARN arn:aws:iam::service:role/static/inventory.

2. Allow the user or entity that will manage the bucket and access the reports to read and write to the destination bucket. The user configuring the inventory report needs the following permissions on the source bucket in order to apply the inventory configuration:

   • s3:PutInventoryConfiguration
   • s3:ListInventoryConfigurations
   • s3:GetInventoryConfiguration
   • s3:DeleteInventoryConfiguration

Depending on your use case, you may need to grant additional permissions to the user or entity.

Create source bucket policy

The following is an example bucket policy that allows the entity requesting the inventory configuration to apply and manage the inventory configuration on the source bucket.

1. Create a source bucket policy file.

   source-bucket-policy.json

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "AllowInventoryConfigurationAccess",
         "Effect": "Allow",
         "Action": [
           "s3:GetObject",
           "s3:PutObject",
           "s3:PutInventoryConfiguration",
           "s3:ListInventoryConfigurations",
           "s3:GetInventoryConfiguration",
           "s3:DeleteInventoryConfiguration"
         ],
         "Resource": [
           "arn:aws:s3:::my-source-bucket/*"
         ],
         "Principal": {
           "CW": ["arn:aws:iam::123456789012:coreweave/UserUID"]
         }
       }
     ]
   }

   Make sure to replace 123456789012 with your organization's cloud ID and UserUID with the user's UID.

2. Apply the source bucket policy:

   Example

   # Example variables for setting source bucket policy
   export SOURCE_BUCKET_NAME="my-source-bucket"
   export SOURCE_BUCKET_POLICY_FILE_PATH="./source-bucket-policy.json"  # assuming source bucket policy is created with this file name in a same directory as working directory.
   
   # Set source bucket policy
   $
   aws s3api put-bucket-policy \
   --bucket ${SOURCE_BUCKET_NAME} \
   --policy file://${SOURCE_BUCKET_POLICY_FILE_PATH}

Create destination bucket policy

The following is an example bucket policy that allows the CoreWeave inventory service account to write inventory reports to the destination bucket, and the user or entity that will issue the inventory list request to access the destination bucket.

Create a destination bucket policy file:

destination-bucket-policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowServiceAccountWriteReportsToDestination",
      "Effect": "Allow",
      "Principal": {
        "CW": "arn:aws:iam::service:role/static/inventory"
      },
      "Action": "s3:PutObject",
      "Resource": [
        "arn:aws:s3:::my-destination-bucket/*"
      ]
    },
    {
      "Sid": "AllowOwnerAccessToDestination",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListObjects"
      ],
      "Resource": [
        "arn:aws:s3:::my-destination-bucket",
        "arn:aws:s3:::my-destination-bucket/*"
      ],
      "Principal": {
        "CW": ["arn:aws:iam::123456789012:coreweave/UserUID"]
      }
    }
  ]
}

Make sure to replace 123456789012 with your organization's cloud ID and UserUID with the user's UID.

• The first statement, AllowServiceAccountWriteReportsToDestination, allows the CoreWeave inventory service account to write to the destination bucket. The Principal field is set to arn:aws:iam::service:role/static/inventory, which is a CoreWeave-managed service account that generates and writes inventory reports to your destination bucket.
• The second statement, AllowOwnerAccessToDestination, allows the entity to access the destination bucket. The Principal field is set to the ARN of the user who will read and manage the inventory reports. In this example, the entity is a user with the ARN arn:aws:iam::123456789012:coreweave/UserUID. Make sure to replace 123456789012 with your organization's cloud ID and UserUID with the user's UID.

Apply source and destination bucket policies

Apply both source and destination bucket policies:

Example

# Example variables for setting source and destination bucket policies
export SOURCE_BUCKET_NAME="my-source-bucket"
export SOURCE_BUCKET_POLICY_FILE_PATH="./source-bucket-policy.json"
export DESTINATION_BUCKET_NAME="my-destination-bucket"
export DESTINATION_BUCKET_POLICY_FILE_PATH="./destination-bucket-policy.json"

# Set source and destination bucket policies
$
aws s3api put-bucket-policy \
--bucket ${SOURCE_BUCKET_NAME} \
--policy file://${SOURCE_BUCKET_POLICY_FILE_PATH}

# Set destination bucket policy
$
aws s3api put-bucket-policy \
--bucket ${DESTINATION_BUCKET_NAME} \
--policy file://${DESTINATION_BUCKET_POLICY_FILE_PATH}

Use the same bucket for source and destination

Alternatively, if you want to write inventory reports to the same bucket as the source bucket, grant access to the service account to write to the source bucket. You will still need the inventory configuration permissions listed above; if you created the bucket, you likely already have these permissions.

1. Create the following bucket policy file:

   same-bucket-policy.json

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "AllowPutObject",
         "Effect": "Allow",
         "Principal": {
           "CW": "arn:aws:iam::service:role/static/inventory"
         },
         "Action": "s3:PutObject",
         "Resource": [
           "arn:aws:s3:::my-source-bucket/*"
         ]
       }
     ]
   }

2. Apply this policy to the source bucket:

   Example

   # Example variables for setting source bucket policy
   export SOURCE_BUCKET_NAME="my-source-bucket"
   export SAME_BUCKET_POLICY_FILE_PATH="./same-bucket-policy.json"  # assuming source bucket policy is created with this file name in a same directory as working directory.
   
   # Set source bucket policy
   $
   aws s3api put-bucket-policy \
   --bucket ${SOURCE_BUCKET_NAME} \
   --policy file://${SAME_BUCKET_POLICY_FILE_PATH}

Configure inventory report

To configure inventory reporting, you specify:

• A source bucket to inventory. You can inventory all the objects in the bucket or only prefixes that you specify. If you don't specify a prefix, all objects will be inventoried.
• A destination bucket where reports are written. You can choose to use a separate destination bucket or the same bucket as the source bucket.
• An optional prefix (subdirectory) within the destination bucket for organizing reports. If you don't specify a prefix, the reports will be written to the root of the destination bucket. Use a prefix (like inventory-reports/) in your inventory configuration to separate reports from your source data.

You can configure your inventory report using the S3 API with standard S3 tools like aws s3api. The configuration is a JSON object that defines the inventory configuration for the bucket.

The following fields are available:

Field	Description	Note
Destination.S3BucketDestination	Configures destination bucket for inventory report
Destination.S3BucketDestination.Bucket	Destination bucket name; can be same as source bucket	Example: my-destination-bucket
Destination.S3BucketDestination.Format	Report object format	Accepts: CSV, TSV, JSON, ORC, Parquet
Destination.S3BucketDestination.Prefix	Prefix for report objects	Example: inventory/ Inventory report objects will be created with inventory/ prefix in the object key.
IsEnabled	Controls enablement of the inventory configuration	Accepts: true or false
Filter	Filters source objects to include in inventory report
Filter.Prefix	Source objects' prefix	Example: documents/ Only objects with prefix documents/ will be included in the report.
Id	Inventory configuration ID	Example: my-inventory-config
IncludedObjectVersions	Controls which object versions are included in the inventory report	Accepts: All or Latest - All: Include all versions - Latest: Only latest versions
OptionalFields	Controls additional information to include in the report	Example: ["Size", "LastAccessedDate"] Inventory report will then have fields: - BucketName - ObjectKey - Size - LastAccessedDate
Schedule	Controls the schedule of inventory reporting
Schedule.Frequency	Sets schedule frequency	Accepts: Daily or Weekly

1. Create a bucket inventory configuration file. The following is an example bucket inventory configuration:

   bucket-inventory-config.json

       {
       "Destination": {
           "S3BucketDestination": {
               "Bucket": "arn:aws:s3:::my-destination-bucket",
               "Format": "ORC",
               "Prefix": "inventory/"
           }
       },
       "IsEnabled": true,
       "Filter": {
           "Prefix": "documents/"
       },
       "Id": "my-inventory-config",
       "IncludedObjectVersions": "All",
       "OptionalFields": [
           "LastAccessedDate"
       ],
       "Schedule": {
           "Frequency": "Daily"
       }
   }

2. Apply the bucket inventory configuration:

   Example

   # Example variables for configuring inventory on source bucket
   export SOURCE_BUCKET_NAME="my-bucket"
   export INVENTORY_CONFIG_ID="my-inventory-config"
   export INVENTORY_CONFIG_FILE_PATH="./bucket-inventory-config.json"  # assuming inventory configuration is created with this file name in a same directory as working directory.
   # Set bucket inventory configuration
   $
   aws s3api put-bucket-inventory-configuration \
   --bucket ${SOURCE_BUCKET_NAME} \
   --id ${INVENTORY_CONFIG_ID} \
   --inventory-configuration file://${INVENTORY_CONFIG_FILE_PATH}

Manage inventory report configurations

You can manage your inventory report configurations using the S3 API with standard S3 tools.

The following is an example of how to create, update, get, and delete inventory report configurations using the AWS CLI:

Example

# Example variables for configuring inventory on source bucket
export SOURCE_BUCKET_NAME="my-bucket"
export INVENTORY_CONFIG_ID="my-inventory-config"
export INVENTORY_CONFIG_FILE_PATH="./bucket-inventory-config.json"  # assuming inventory configuration is created with this file name in a same directory as working directory.

# Create/update bucket inventory configuration by ID
$
aws s3api put-bucket-inventory-configuration \
--bucket ${SOURCE_BUCKET_NAME} \
--id ${INVENTORY_CONFIG_ID} \
--inventory-configuration file://${INVENTORY_CONFIG_FILE_PATH}

# List bucket inventory configurations
$
aws s3api list-bucket-inventory-configurations --bucket ${SOURCE_BUCKET_NAME}

# Get bucket inventory configuration by ID
$
aws s3api get-bucket-inventory-configuration \
--bucket ${SOURCE_BUCKET_NAME} \
--id ${INVENTORY_CONFIG_ID}

# Delete bucket inventory configuration by ID
$
aws s3api delete-bucket-inventory-configuration \   
--bucket ${SOURCE_BUCKET_NAME} \
--id ${INVENTORY_CONFIG_ID}

If you check your inventory configuration with the get command, you may see output like this:

bucket-inventory-config.json

{
    "InventoryConfiguration": {
        "Destination": {
            "S3BucketDestination": {
                "Bucket": "arn:aws:s3:::my-destination-bucket",
                "Format": "ORC",
                "Prefix": "inventory/"
            }
        },
        "IsEnabled": true,
        "Filter": {
            "Prefix": "documents/"
        },
        "Id": "my-inventory-config",
        "IncludedObjectVersions": "All",
        "OptionalFields": [
            "LastAccessedDate"
        ],
        "Schedule": {
            "Frequency": "Daily"
        }
    }
}

Using Terraform

If you manage your infrastructure as code, you can use the AWS Terraform provider to manage inventory report configurations alongside your other resources.

Example

resource "aws_s3_bucket_policy" "test_destination_bucket_policy" {
  bucket = aws_s3_bucket.inventory_dest.id
  policy = data.aws_iam_policy_document.allow_access.json
}

data "aws_iam_policy_document" "allow_access" {
  statement {
    sid = "AllowPutObject"
    effect = "Allow"

    principals {
      type = "CW"
      identifiers = ["arn:aws:iam::service:role/static/inventory"]
    }
    actions = [
      "s3:PutObject",
    ]

    resources = [
      "${aws_s3_bucket.inventory_dest.arn}/*",
    ]
  }

  statement {
    sid = "AllowOwnerAccess"
    effect = "Allow"

    principals {
      type = "CW"
      identifiers = ["arn:aws:iam::123456789012:coreweave/UserUID"]
    }
    actions = [
      "s3:*",
    ]

    resources = [
      aws_s3_bucket.inventory_dest.arn,
      "${aws_s3_bucket.inventory_dest.arn}/*",
    ]
  }
}

resource "aws_s3_bucket" "inventory_test" {
  bucket = "my-bucket"
}

resource "aws_s3_bucket" "inventory_dest" {
  bucket = "my-destination-bucket"
}

resource "aws_s3_bucket_inventory" "test-prefix" {
  bucket = aws_s3_bucket.inventory_test.id
  name   = "my-inventory-config"

  enabled = true
  included_object_versions = "All"

  schedule {
    frequency = "Daily"
  }

  filter {
    prefix = "documents/"
  }

  destination {
    bucket {
      format     = "ORC"
      bucket_arn = aws_s3_bucket.inventory_dest.arn
      prefix     = "inventory/"
    }
  }
}

Checking your inventory report

After you have applied your inventory configuration, allow some time for the inventory report to be generated and written to the destination bucket, then check the path where you configured the inventory report to be written. There is no notification when the inventory report has completed.

When the inventory report has completed, you should see a manifest.json file written to the root of the subdirectory specified in the inventory configuration, and one or more inventory data files. You can use the manifest file to understand the structure of the inventory report. See more details about the manifest files.

Disabling inventory reporting

To disable inventory reporting, you can either modify the inventory configuration to set IsEnabled to false, or delete the inventory configuration.

The following is an example of a modified inventory configuration file that disables inventory reporting:

modified-inventory-config.json

{
  "Destination": {
    "S3BucketDestination": {
      "Bucket": "arn:aws:s3:::my-destination-bucket",
      "Format": "ORC",
      "Prefix": "inventory/"
    }
  },
  "IsEnabled": false,
  "Filter": {
    "Prefix": "documents/"
  },
  "Id": "my-inventory-config",
  "IncludedObjectVersions": "All",
  "OptionalFields": [
    "LastAccessedDate"
  ],
  "Schedule": {
    "Frequency": "Daily"
  }
}

These commands can be used to apply the modified inventory configuration or delete the inventory configuration:

Example

# Example variables for configuring inventory on source bucket
export SOURCE_BUCKET_NAME="my-bucket"
export INVENTORY_CONFIG_ID="my-inventory-config"
export INVENTORY_CONFIG_FILE_PATH="./modified-inventory-config.json"  # assuming inventory configuration is created with this file name in a same directory as working directory.

# Modify bucket inventory configuration
$
aws s3api put-bucket-inventory-configuration \
--bucket ${SOURCE_BUCKET_NAME} \
--id ${INVENTORY_CONFIG_ID} \
--inventory-configuration file://${INVENTORY_CONFIG_FILE_PATH}

# Delete bucket inventory configuration
$
aws s3api delete-bucket-inventory-configuration \
--bucket ${SOURCE_BUCKET_NAME} \
--id ${INVENTORY_CONFIG_ID}