Skip to main content

Deploy Milvus on CKS

Deploy Milvus Vector Database on CoreWeave Kubernetes Service (CKS)

These instructions explain how to deploy Milvus, an open-source vector database built for GenAI applications, on CoreWeave Kubernetes Service (CKS).

Prerequisites​

Before you start, you need:

  • A working CKS cluster with at least one CPU Node. GPU Nodes are also supported, but CPU Nodes are preferred unless you plan to use GPU-based indexing.
  • Access Key and Secret Key with permissions to use a CoreWeave AI Object Storage bucket.

You'll need the following tools on your local machine:

  • Kubectl installed and configured for your cluster.
  • Helm version 3.8+
  • Git

It's also helpful to have an Object Storage command-line tool such as the AWS CLI to access your bucket when testing.

Step 1. Verify your system configuration​

  1. Verify that you can access your cluster with kubectl. For example:

    $
    kubectl cluster-info

    You should see something similar to:

    Kubernetes control plane is running at...

    CoreDNS is running at...

    node-local-dns is running at...

  2. Verify your cluster has at least one CPU Node. GPU Nodes are also supported, but CPU Nodes are preferred unless you plan to use GPU-based indexing. For example:

    $
    kubectl get nodes -o=custom-columns="NAME:metadata.name,CLASS:metadata.labels['node\.coreweave\.cloud\/class']"

    You should see something similar to the following:

    NAME      CLASS

    g137a10   gpu

    g5424e0   cpu

    g77575e   cpu

    gd926d4   gpu

Step 2. Deploy Milvus​

  1. Install the Milvus Operator. See the Operator installation guide for more details.

    $
    helm install milvus-operator \

      -n milvus-operator --create-namespace \

      --wait --wait-for-jobs \

      https://github.com/zilliztech/milvus-operator/releases/download/v1.3.0/milvus-operator-1.3.0.tgz

    This installs the Milvus Custom Resource Definition (CRD), which is used to define Milvus clusters, along with the Operator that manages them. It creates a new namespace called milvus-operator for the Operator itself.

    It's possible to add milvus-operator as a dependency to the CoreWeave chart (which you'll download in the next step), but it's preferred to install it in a separate namespace from the database. The Operator can manage multiple Milvus clusters in different namespaces.

  2. Clone the CoreWeave Milvus chart repository at:

    https://github.com/coreweave/reference-architecture/tree/main/tooling/vector_dbs/cw-milvus

  3. Edit the chart's values.yaml with your details:

    caiosBucketName: YOUR_BUCKET_NAME

    caiosAccessKey: YOUR_ACCESS_KEY

    caiosSecretKey: YOUR_SECRET_KEY
    Check the deleteDependencies policy

    This chart automatically deletes dependencies like etcd and kafka by default. This makes it easier to clean up after an experiment is complete.

    You should set deleteDependencies to false if you want to retain these dependencies after deleting the Milvus custom resource.

  4. If you plan to use GPU-based indexing, set the gpuIndexing value to true in values.yaml. This will schedule the relevant components on GPU Nodes as shown.

    dataNode:

      affinity:

        nodeAffinity:

          # override global component affinity when using GPU indexing

          requiredDuringSchedulingIgnoredDuringExecution:

            nodeSelectorTerms:

            - matchExpressions:

              - key: node.coreweave.cloud/class

                operator: In

                values:

                - gpu

    queryNode:

      affinity:

        nodeAffinity:

          # override global component affinity when using GPU indexing

          requiredDuringSchedulingIgnoredDuringExecution:

            nodeSelectorTerms:

            - matchExpressions:

              - key: node.coreweave.cloud/class

                operator: In

                values:

                - gpu

Chart defaults explained​

The CoreWeave chart handles the following items:

  • It provisions a secret with the Object Storage credentials you specified in values.yaml.

  • It configures the Object Storage bucket you specified, with aliyun as the provider. This ensures that Milvus uses virtual-host style addressing.

    config:

      minio:

        # Use "aliyun" as cloud provider, as that enables virtual host style bucket requests

        cloudProvider: aliyun

        useVirtualHost: true

        bucketName: {{ .Values.caiosBucketName }}

        # Optional, config the prefix of the bucket milvus will use

        rootPath: {{ include "cw-milvus.name" . }}

        useSSL: true

  • It sets the storage type to S3 and uses the CoreWeave AI Object Storage endpoint (cwobject.com:443).

    dependencies:

      storage:

      # enable external object storage

      external: true

      type: S3 # MinIO | S3

      # the endpoint of AWS S3

      endpoint: cwobject.com:443

      # the secret storing the access key and secret key

      secretRef: '{{ include "cw-milvus.name" . }}-caios-secret'

  • It configures Kafka as the message storage service and overrides the default image (which is not yet available).

    kafka:

      inCluster:

        deletionPolicy: Delete

        pvcDeletion: true

        values:

        image:

          repository: bitnami/kafka

          tag: 3.7.0-debian-12-r6

  • It sets Node affinities for the Milvus components to CPU Nodes. However, scheduling onto GPU Nodes is also possible when using GPU-based indexing as described in Step 2.

    affinity:

      nodeAffinity:

        # prefer running on CPU nodes, if available

        # see also queryNode and dataNode affinities for GPU-based indexing

        preferredDuringSchedulingIgnoredDuringExecution:

          - weight: 100

            preference:

              matchExpressions:

              - key: node.coreweave.cloud/class

                operator: In

                values:

                - cpu

  • It configures a CronJob for backup using milvus-backup. You can control the cron expression for scheduling the job, as well as settings for the backup bucket via values in values.yaml. If you do not specify a bucket name and credentials for backups, the ones specified for the database are used.

    backupCaiosBucketName: YOUR_BACKUP_BUCKET_NAME

    backupCaiosAccessKey: YOUR_BACKUP_BUCKET_ACCESS_KEY

    backupCaiosSecretKey: YOUR_BACKUP_BUCKET_SECRET_KEY

    # defaults to "backup"

    backupCaiosBucketRootPath:

    backupCronExpression: "30 7 * * *"
    Backups will accumulate

    Backup names are based on timestamps, so are unique. This means backups will accumulate in your bucket unless you clean out unneeded ones periodically.

Step 3. Install the chart​

  1. Change to the chart directory:

    $
    cd reference-architecture/tooling/vector_dbs/cw-milvus

  2. Install the chart in a new namespace, e.g. milvus:

    $
    helm install -n milvus cw-milvus .

  3. Check the status of the custom resource. This may take several minutes to complete, as the Operator sets up the database and its dependencies.

    kubectl -n milvus describe milvus cw-milvus

    The Conditions block will show the status of individual components. Once everything is set up, that block should look like this:

    Conditions:

        Last Transition Time:  2025-09-22T21:47:24Z

        Message:               All Milvus components are healthy

        Reason:                ReasonMilvusHealthy

        Status:                True

        Type:                  MilvusReady

        Last Transition Time:  2025-09-18T23:14:29Z

        Message:               Milvus components are all updated

        Reason:                MilvusComponentsUpdated

        Status:                True

        Type:                  MilvusUpdated

        Last Transition Time:  2025-09-18T23:12:32Z

        Message:               Etcd endpoints is healthy

        Reason:                EtcdReady

        Status:                True

        Type:                  EtcdReady

        Last Transition Time:  2025-09-18T23:12:32Z

        Reason:                StorageReady

        Status:                True

        Type:                  StorageReady

        Last Transition Time:  2025-09-18T23:13:29Z

        Reason:                MsgStreamReady

        Status:                True

        Type:                  MsgStreamReady

Step 4. Access the database​

After the database is ready, the web UI service is available on port 9091, and the database service is available on port 19530. To access these, forward local ports from your machine to the service.

  1. Forward a local port to the database service.

    $
    kubectl port-forward --address 0.0.0.0 service/cw-milvus-milvus 27017:19530

    Forwarding from 0.0.0.0:27017 -> 19530

  2. Forward a local port to the web UI service.

    kubectl port-forward --address 0.0.0.0 service/cw-milvus-milvus 27018:9091

    Forwarding from 0.0.0.0:27018 -> 9091

You can now access the web UI at http://localhost:27018 and connect to the database at localhost:27017.

Additional resources​

See the Milvus documentation to learn more.

Last updated on