Skip to main content

Best Practices for CoreWeave AI Object Storage

Use LOTA​

To achieve the highest performance for CoreWeave AI Object Storage, always use LOTA when making requests within a CoreWeave cluster. Clients only need to point their requests to the LOTA endpoint instead of the primary endpoint to take advantage of LOTA's caching capabilities.

Use multipart uploads​

Use multipart uploads for large objects. When uploading an object with s3:CreateMultiPartUpLoad, LOTA distributes the object's parts across the GPU Nodes in the cache. By placing the parts of the object across multiple GPU Nodes, read performance is significantly improved. LOTA automatically invalidates the cache when parts of the object are updated, so that no stale data is served.

Multipart uploads have several other inherent advantages, too:

  • Improved throughput: Upload the parts in parallel to make the best use of available bandwidth.
  • Fast recovery: Smaller parts minimize the impact of a network error.
  • Ability to pause and resume: Multipart uploads do not expire. It's possible to pause and resume at any time, because the upload must be explicitly completed or deliberately stopped.
  • Upload as objects are created: Multipart uploads can begin before the final object size is known.

Conversely, objects uploaded via s3:PutObject are stored on a single GPU Node, which can lead to performance bottlenecks with large objects.

Optimum part size​

The part size in a multipart upload has a significant impact on LOTA's cache performance.

Breaking small objects into too many parts can degrade read performance due to the overhead of excessive HTTP requests. A recommended minimum part size is 50 MB, which is large enough to reduce the number of HTTP requests while still small enough for LOTA to distribute parts across the GPU Nodes efficiently.

Also, note that LOTA only caches objects larger than 4 MB; smaller objects are fetched directly from the backend, bypassing the cache.

Avoid sequential object keys​

Do not use sequential object keys. When creating an object in a bucket, the name given to that object is the object key. Use a random prefix for any sequential keys to maintain high performance.

Manage quota limits​

CoreWeave AI Object Storage has the following quota limits by default:

  • Capacity: 10 TiB per Availability Zone
  • Max bucket count: 1,000
  • LOTA cache capacity per GPU Node: 1 TiB

To request a quota increase, contact CoreWeave support.

Last updated on