Use s3cmd with CoreWeave Object Storage

S3cmd is a popular command-line tool used to manage and interact with CoreWeave Object Storage. This guide explains how to configure it and run basic commands. Before proceeding, you'll need a Linux or macOS machine with terminal access and a CoreWeave Object Storage Token.

Installation

To install s3cmd on macOS, use Homebrew.

Example

$
brew install s3cmd

For Linux, use the system package manager. For example, on Ubuntu:

Example

$
sudo apt-get install s3cmd

Alternatively, download the latest release from the sources listed on the s3cmd website.

Configuration

The default config file for s3cmd is located in the user's home directory at ~/.s3cfg. The configuration file can be automatically downloaded from CoreWeave, or manually created with the configuration wizard.

Tip

If desired, different configuration files can be used with the --config flag on the command-line.

Configure automatically

To automatically download the configuration, check the download box when creating the Object Storage token, or downloaded it later from the Web UI as described in the Object Storage documentation.

The downloaded file is similar to this:

title="cw-object-storage-config"

[default]
access_key = REDACTED
secret_key = REDACTED
# The region for the host_bucket and host_base must be the same.
host_base = object.ord1.coreweave.com
host_bucket = %(bucket)s.object.ord1.coreweave.com
check_ssl_certificate = True
check_ssl_hostname = True

After downloading the file, move it to ~/.s3cfg:

Example

$
mv ~/Downloads/cw-object-storage-config ~/.s3cfg

S3cmd is now ready to use.

Configure manually

To manually configure s3cmd with a CoreWeave token, run the configuration wizard:

Example

$
s3cmd --configure

When prompted by the wizard, enter the Access key and Secret key.

Leave the default value when prompted for the region. CoreWeave Object Storage does not use this value.

Enter an S3 endpoint when prompted.

Region	Endpoint
New York - LGA1	object.lga1.coreweave.com
Chicago - ORD1	object.ord1.coreweave.com
Las Vegas - LAS1	object.las1.coreweave.com

Enter the DNS-style template, which must be in the same region as the S3 endpoint.

Region	DNS-style template
New York - LGA1	%(bucket)s.object.lga1.coreweave.com
Chicago - ORD1	%(bucket)s.object.ord1.coreweave.com
Las Vegas - LAS1	%(bucket)s.object.las1.coreweave.com

Leave the Encryption password blank.

Encryption is optional, please refer to the s3cmd documentation for more information if you require encryption.

Take the default values for the rest of the prompts.

When complete, the session should look similar to this example.

Click to expand: s3cmd configuration session

Example

$
s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: REDACTED
Secret Key: REDACTED
Default Region [US]:

Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]: object.ord1.coreweave.com

Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: %(bucket)s.object.ord1.coreweave.com

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password:
Path to GPG program [/usr/bin/gpg]:

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]:

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name:

New settings:
  Access Key: REDACTED
  Secret Key: REDACTED
  Default Region: US
  S3 Endpoint: object.ord1.coreweave.com
  DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.object.ord1.coreweave.com
  Encryption password:
  Path to GPG program: /usr/bin/gpg
  Use HTTPS protocol: True
  HTTP Proxy server name:
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Not configured. Never mind.

Save settings? [y/N] y
Configuration saved to '/home/username/.s3cfg'

Usage

The most common s3cmd commands for interacting with Object Storage are:

Create a new bucket

Example

$
s3cmd mb s3://bucket-name

Remove a bucket

Example

$
s3cmd rb s3://bucket-name

List buckets

Example

$
s3cmd ls

List objects in a bucket

Example

$
s3cmd ls s3://bucket-name

Delete all objects in a bucket

Example

$
s3cmd del --recursive --force s3://bucket-name

Upload a file for private access

Example

$
s3cmd put local-file.txt s3://bucket-name/remote-filename.txt

Upload a file for public access

Example

$
s3cmd put -P local-file.txt s3://bucket-name/remote-filename.txt

Download a file

Example

$
s3cmd get s3://bucket-name/remote-filename.txt local-file.txt

Delete a file

Example

$
s3cmd rm s3://bucket-name/remote-filename.txt

Change file permission to public access

Example

$
s3cmd setacl s3://bucket-name/remote-filename.txt --acl-public

Change file permission to private access

Example

$
s3cmd setacl s3://bucket-name/remote-filename.txt --acl-private

Enable public directory listing for a bucket

Example

$
s3cmd setacl s3://bucket-name/ --acl-public

Disable public directory listing for a bucket

Example

$
s3cmd setacl s3://bucket-name/ --acl-private

Additional Resources

To learn more about s3cmd, please see the official documentation.