Skip to main content

Nydus

Get faster pull times with Nydus on CoreWeave

Important

Please note that Nydus is still in an experimental phase on CoreWeave Cloud.

Embedding machine learning models directly into images has become a popular ease-of-use technique, but it has made image pull times slower due to the increased size of container images. As a result, pulling images is often the most time-consuming aspect of spinning up new containers, and for those who rely on fast autoscaling to respond to changes in demand, the time it takes to create new containers can pose as a major hurdle.

It's for this reason that CoreWeave Cloud supports Nydus, for dramatically shorter container image pull times.

What is Nydus?​

Nydus is an external plugin for containerd leveraging the Nydus container image service, which implements a content-addressable filesystem on top of a RAFS format for container images. This formatting allows for major improvements to the current OCI image specification in terms of container launching speed, image space, network bandwidth efficiency, and data integrity.

Some of Nydus' key features include:

  • a method called "lazy pulling," container images are downloaded on-demand in "chunks," which significantly boosts container startup times,
  • content-addressable data deduplication, which minimizes storage, transmission, and memory footprints,
  • a merged filesystem tree, which allows all intermediate layers to be removed,
  • compatibility with the OCI artifacts and distribution specs, so Nydus images may be stored in any regular container registry

For clients who would like to bake machine learning models directly into their images, or who otherwise have very large container images, Nydus provides a solution that significantly shortens pull times of hefty container images.

Additional Resources

For a list of additional features and official benchmarks, refer to the Nydus project's GitHub.

Setup​

Nydus configures the container runtime (containerd), to inspect the given container image at runtime. If Nydus recognizes the image as a Nydus-formatted image, then it will pull the image using the Nydus container image service.

Nydus is also backwards-compatible - if Nydus recognizes that the image being pulled is a standard OCI image, then it will pull it normally; just as a normal configuration of the containerd runtime would.

To leverage Nydus on CoreWeave Cloud, base images must be converted to the Nydus format, but no aspect of the Deployment manifest needs to change.

Convert images to Nydus format​

To convert the images to Nydus format, use the Nydusify conversion tool. Converting base images is as easy as either targeting an OCI image in a repository:

Example
$
nydusify convert \

  --source myregistry/repo:tag \

  --target myregistry/repo:tag-nydus

Or, a local dictionary may be packed as a Nydus-formatted image:

Example
$
nydusify pack \

  --bootstrap target.bootstrap \

  --target-dir /path/to/target \

  --output-dir /path/to/output
Important

At this time, using Nydus images on CoreWeave requires that:

  • base images are converted manually using Nydusify
  • images must be stored in a public repository
Last updated on