Set up infrastructure dependencies

Install Traefik ingress controller and cert-manager for your vLLM inference deployment

Overview

Before deploying vLLM, you need to set up the infrastructure components that will handle external traffic routing and TLS certificate management. This step installs Traefik (for ingress) and cert-manager (for automatic SSL certificates).

Step 1: Add CoreWeave Helm repository

First, add CoreWeave's official Helm repository to access the required charts:

Example

$
helm repo add coreweave https://charts.core-services.ingress.coreweave.com
$
helm repo update

After running the previous commands, you should see output like the following:

Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "kedacore" chart repository
...Successfully got an update from the "coreweave" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "prometheus-community" chart repository
Update Complete. ⎈Happy Helming!⎈

Step 2: Install Traefik ingress controller

Deploy Traefik to handle external traffic routing and TLS termination:

Example

$
helm install traefik coreweave/traefik \
  --namespace traefik \
  --create-namespace

You should see something like the following:

NAME: traefik
LAST DEPLOYED: Mon Aug 16:17:22
NAMESPACE: traefik
STATUS: deployed
REVISION: 1
TEST SUITE: None

Verify the installation by checking that all Traefik pods are running:

Example

$
kubectl get pods -n traefik

You should see output similar to the following:

NAME                       READY   STATUS    RESTARTS   AGE
traefik-6b7c8d9f4b-abc12   1/1     Running   0          2m

Step 3: Install and enable cert-manager

Run the following command to install cert-manager for automatic TLS certificate management:

Example

helm install cert-manager coreweave/cert-manager \
  --namespace cert-manager \
  --create-namespace

You should see output similar to the following:

NAME: cert-manager
LAST DEPLOYED: Mon Aug 16:21:38
NAMESPACE: cert-manager
STATUS: deployed
REVISION: 1
TEST SUITE: None

Run the following command to enable cert-manager:

Example

helm upgrade cert-manager coreweave/cert-manager \
  --namespace cert-manager \
  --set cert-issuers.enabled=true

You should see output similar to the following:

Release "cert-manager" has been upgraded. Happy Helming!
NAME: cert-manager
LAST DEPLOYED: Mon 16:25:47
NAMESPACE: cert-manager
STATUS: deployed
REVISION: 2
TEST SUITE: None

Verify cert-manager is running by using the following command:

Example

kubectl get pods -n cert-manager

You should see output similar to the following:

NAME                                      READY   STATUS    RESTARTS   AGE
cert-manager-6b7c8d9f4b-def34              1/1     Running   0          1m
cert-manager-cainjector-6b7c8d9f4b-ghi56   1/1     Running   0          1m
cert-manager-webhook-6b7c8d9f4b-jkl78      1/1     Running   0          1m

Step 4: Verify infrastructure setup

Check that all components are properly installed and running:

Example

kubectl get svc -n traefik

You should see output similar to the following:

NAME          TYPE           CLUSTER-IP    EXTERNAL-IP   PORT(S)                                    AGE
traefik       LoadBalancer   10.16.3.150   166.00.0.00   80:37187/TCP,443:33016/TCP,443:33016/UDP   12m
traefik-k8s   LoadBalancer   10.16.2.181   10.16.4.0     443:36556/TCP                              12m

Check cert-manager services by running the following command:

Example

kubectl get svc -n cert-manager

You should see output similar to the following:

NAME                      TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)            AGE
cert-manager              ClusterIP   10.16.1.116   <none>        9402/TCP           10m
cert-manager-cainjector   ClusterIP   10.16.3.76    <none>        9402/TCP           10m
cert-manager-webhook      ClusterIP   10.16.2.145   <none>        443/TCP,9402/TCP   10m

Verify certificate issuers are available by running the following command:

Example

$
kubectl get clusterissuer

You should see output similar to the following:

NAME                        READY   AGE
letsencrypt-prod            True    7h4m
letsencrypt-staging         True    7h4m
selfsigned-cluster-issuer   True    7h4m

What's next

Your infrastructure dependencies are now set up! In the next step, you'll configure monitoring and observability for your vLLM deployment.

Tip

If you encounter any issues with the installation, check the pod logs for troubleshooting information:

Example

$
kubectl logs -n traefik -l app.kubernetes.io/name=traefik
$
kubectl logs -n cert-manager -l app.kubernetes.io/name=cert-manager

⬅ Previous Deploy vLLM for Inference

Next ➡ Configure monitoring and observability