July 2023
New on CoreWeave Cloud this month:
📈 New Tensorizer benchmarks and benchmark tutorial
CoreWeave Tensorizer is a tool for fast PyTorch module, model, and tensor serialization and deserialization, making it possible to load models extremely quickly from HTTP/HTTPS and S3 endpoints. It also speeds up loading from network and local disk volumes.
With faster model loading times for LLMs and reduces GPU memory utilization, Tensorizer helps accelerate model instance spin up times while reducing overall costs to serve inference.
Tensorizer is S3/HTTP-compatible, enabling model streams directly from S3 into the container without having to download the model to the container's local filesystem.
The average latency per request was >5x faster for Tensorizer compared to Hugging Face when scaling from zero, and required fewer pod spin ups and less RAM.
In addition to a brand new blog post about Tensorizer's performance benchmarks, a new tutorial for running a real-world benchmark test is now available to try yourself!
✨ Updates to the CoreWeave Cloud UI
.4a42152.1200.png)
The CoreWeave Cloud UI is now even easier and more intuitive to use! Manage all your resources and account information right from your browser. Additionally, a new guide exploring all of the features of the updated Cloud UI has been added to better introduce you to this feature-rich GUI.
🔏 New namespace access controls for user tokens
.7a8d87f.1200.png)
With new namespace access controls, organization administrators can create access tokens with specific namespace permissions, allowing for a greater level of security for organization members. A token with no specified namespace permissions can also be created, granting the organization administrator the ability to create Kubernetes custom RBAC policies.
👋 Support dropped for Ubuntu 18.04
As per the out of support EOL notice for Ubuntu 18.04 begun at the end of May, CoreWeave no longer supports Ubuntu 18.04. Existing images will not yet be deleted, but no new 18.04 images will be built.