Introduction to third-party frameworks

Any framework that runs on Kubernetes or in Linux containers works on CoreWeave Kubernetes Service (CKS) and SUNK. The following sections highlight popular AI and ML frameworks, grouped by category, with links to CoreWeave guides where available.

Interactive development

The following table lists notebook environments and interactive UIs.

Framework	Description	CKS guide	SUNK guide
marimo	Reactive notebook environment for interactive Python development with GPU support.	marimo notebooks on CKS	Notebooks on SUNK
Jupyter	Interactive computing platform for notebooks, code, and data visualization.	Compatible on CKS	Notebooks on SUNK
Open WebUI	Web-based chat interface for interacting with locally deployed language models.	Deploy a model on CKS	Not applicable

Training frameworks

The following table lists frameworks used for training models and distributed training. CoreWeave also publishes optimized PyTorch base images you can use as a starting point for distributed training on CKS and SUNK. For details, see ML container images.

Framework	Description	CKS guide	SUNK guide
PyTorch	Open source deep-learning framework with GPU acceleration and distributed training through `torchrun`.	Compatible on CKS	Submit a training job
TensorFlow	Open source platform for building and deploying machine-learning models across a range of tasks.	Compatible on CKS	Submit a training job
Keras	High-level neural network API that runs on top of TensorFlow, JAX, or PyTorch.	Compatible on CKS	Compatible on SUNK
JAX	High-performance numerical computing library with automatic differentiation and XLA compilation for GPUs and TPUs.	JAX on marimo notebooks	Compatible on SUNK
Hugging Face Transformers	Library for pretrained models for natural language processing, computer vision, and audio tasks.	Compatible on CKS	Compatible on SUNK
DeepSpeed	Deep-learning optimization library for distributed training and inference with memory-efficient techniques.	Compatible on CKS	Compatible on SUNK
Megatron-LM	NVIDIA framework for training large transformer models using model and data parallelism.	Compatible on CKS	Compatible on SUNK
Horovod	Distributed deep-learning training framework supporting TensorFlow, Keras, PyTorch, and Apache MXNet.	Compatible on CKS	Compatible on SUNK
PaddlePaddle	Deep-learning framework with support for distributed training and model deployment.	Compatible on CKS	Compatible on SUNK
PAX	JAX-based framework for configuring and running ML experiments at scale.	Compatible on CKS	Compatible on SUNK
BigDL	Distributed deep-learning library for Apache Spark and Intel hardware.	Compatible on CKS	Compatible on SUNK
MegEngine	Deep-learning framework with dynamic and static graph support.	Compatible on CKS	Compatible on SUNK
MindSpore	AI computing framework for training and inference across cloud, edge, and device.	Compatible on CKS	Compatible on SUNK
torchforge	Training framework for reinforcement learning with group relative policy optimization (GRPO).	Compatible on CKS	torchforge on SUNK
veRL	Framework for reinforcement learning training of large language models with Slurm and Ray integration.	Compatible on CKS	veRL on SUNK

Inference frameworks

The following table lists frameworks used for model serving and inference.

Framework	Description	CKS guide	SUNK guide
vLLM	High-throughput and memory-efficient LLM inference and serving engine.	Deploy vLLM inference	Compatible on SUNK
SGLang	Serving framework for large language models and vision language models.	SGLang on CKS	Compatible on SUNK
NVIDIA TensorRT	GPU inference optimizer that delivers low-latency and high-throughput model serving.	Compatible on CKS	Compatible on SUNK
NVIDIA TensorRT-LLM	Library for optimizing and deploying large language model inference on GPUs.	TensorRT-LLM on marimo notebooks	Compatible on SUNK
NVIDIA NIM	NVIDIA inference microservices for deploying optimized AI models as containerized services.	Deploy NIMs on CKS	Compatible on SUNK
NVIDIA Dynamo	Cluster-wide inference orchestration with the Kai scheduler and Grove.	Dynamo inference on CKS	Compatible on SUNK
Red Hat AI Inference Stack	GPU-based LLM inference using llm-d, KServe, Istio, and the Gateway API.	Red Hat AI on CKS	Compatible on SUNK
OpenVINO	Intel toolkit for optimizing and deploying inference across CPU, GPU, and accelerator hardware.	Compatible on CKS	Compatible on SUNK
TensorFlow Lite	Lightweight runtime for running TensorFlow models on edge and mobile devices.	Compatible on CKS	Compatible on SUNK

Orchestration and management

The following table lists frameworks for scheduling workloads and managing ML operations on clusters.

Framework	Description	CKS guide	SUNK guide
Ray	Distributed computing framework for scaling AI and Python applications across clusters.	Ray with Kueue	Ray on SUNK
Anyscale	Managed platform for deploying and scaling Ray-based distributed AI and ML workloads.	Anyscale on CKS	Compatible on SUNK
Kubeflow	Open source machine-learning platform for portable, scalable ML workflows on Kubernetes.	Kubeflow on CKS	Compatible on SUNK
SkyPilot	Framework for running AI and ML workloads across cloud infrastructure.	SkyPilot on CKS	SkyPilot on SUNK
Union.ai	Managed platform for Flyte, providing workflow orchestration for data and ML pipelines with a hosted control plane.	Union.ai on CKS	Compatible on SUNK
Kueue	Kubernetes-native job queueing system for prioritized workload scheduling and resource management.	Ray with Kueue	Compatible on SUNK

ML libraries

The following table lists general-purpose machine-learning and AutoML libraries.

Framework	Description	CKS guide	SUNK guide
XGBoost	Optimized gradient boosting library for classification, regression, and ranking tasks.	Compatible on CKS	Compatible on SUNK
scikit-learn	General-purpose machine-learning library for classification, regression, clustering, and preprocessing.	Compatible on CKS	Compatible on SUNK
MLlib	Apache Spark’s distributed machine-learning library for classification, regression, clustering, and pipelines.	Compatible on CKS	Compatible on SUNK
ML.NET	Cross-platform machine-learning framework for .NET applications.	Compatible on CKS	Compatible on SUNK
TPOT	Automated machine-learning tool that optimizes ML pipelines using genetic programming.	Compatible on CKS	Compatible on SUNK
AutoKeras	AutoML library built on Keras for automated neural architecture search.	Compatible on CKS	Compatible on SUNK
Auto-sklearn	AutoML toolkit that automatically selects and tunes scikit-learn models.	Compatible on CKS	Compatible on SUNK
PlaidML	Portable deep-learning backend that runs on a variety of hardware including CPUs and GPUs.	Compatible on CKS	Compatible on SUNK

GPU acceleration libraries

The following table lists GPU-accelerated computing libraries.

Framework	Description	CKS guide	SUNK guide
cuDNN	NVIDIA GPU-accelerated library of primitives for deep neural networks.	Compatible on CKS	Compatible on SUNK
cuBLAS	NVIDIA GPU-accelerated implementation of the BLAS (Basic Linear Algebra Subprograms) standard.	Compatible on CKS	Compatible on SUNK

Deprecated frameworks

The following frameworks are deprecated or no longer actively maintained by their upstream projects. They can still run on CKS and SUNK in custom container images.

Framework	Description	Status
Apache MXNet	Distributed deep-learning framework.	Retired by the Apache Software Foundation in 2023.
Caffe and Caffe2	Deep-learning frameworks for image classification and convolutional networks.	Caffe2 merged into PyTorch. Caffe is no longer actively maintained.
Chainer	Deep-learning framework with a define-by-run approach.	Development ended in 2019. Successor: PyTorch.
Microsoft Cognitive Toolkit (CNTK)	Deep-learning framework for distributed training.	Archived by Microsoft in 2019.
Theano	Numerical computation library with GPU support for defining and optimizing mathematical expressions.	Development ended in 2017.
Torch	Scientific computing framework with GPU support based on Lua.	Superseded by PyTorch.

This list isn’t exhaustive. If a framework isn’t listed, you can build custom container images and deploy them on CKS or SUNK.

​Interactive development

​Training frameworks

​Inference frameworks

​Orchestration and management

​ML libraries

​GPU acceleration libraries

​Deprecated frameworks

Interactive development

Training frameworks

Inference frameworks

Orchestration and management

ML libraries

GPU acceleration libraries

Deprecated frameworks