Slurm parameter reference - CoreWeave Docs

Parameters

Key & Description Type Default

accounting.config
slurmdbd.conf overrides (optional).

object

null

accounting.database
Database connection configuration for slurmdbd. database.backend selects the database backend: “moco”, “cwdb”, or “external”.

object

{
  "backend": "moco",
  "existingSecret": null,
  "storageHost": null,
  "storageLoc": null,
  "storagePort": null,
  "storageUser": null
}

accounting.database.backend
Select the database backend for Slurm Accounting (slurmdbd). “moco”: operator creates a MOCO MySQLCluster (requires database.* config fields). “cwdb”: operator creates a CWDBCluster (requires database.* config fields). “external”: user-managed database (requires storageHost, storageUser, existingSecret).

string

moco

accounting.database.existingSecret
Specify the name of the Kubernetes Secret that contains the password used by slurmdbd to access the Slurm accounting database. For moco/cwdb: auto-derived if not set (key: WRITABLE_PASSWORD). For external: required (key: db-password). Important: The password value stored in the Secret cannot contain the hash (#) character.

string

null

accounting.database.storageHost
The hostname of the server where the database resides. For moco/cwdb: auto-derived if not set. For external: required.

string

null

accounting.database.storageLoc
The name of the database used to store Slurm accounting records.

string

slurm_acct_db

accounting.database.storagePort
The port of the database server.

int

accounting.database.storageUser
The username slurmdbd uses for authentication and storing job accounting data. For moco/cwdb: defaults to “moco-writable” if not set. For external: required.

string

null

accounting.enabled
Enable the accounting.

bool

true

accounting.image
The image to use for slurmdbd deployment.

object

repository:
tag:

accounting.livenessProbe
The liveness probe for the slurmdbd container.

object

sacctmgr ping; initialDelaySeconds 15; periodSeconds 10; failureThreshold 5; successThreshold 1; timeoutSeconds 60

accounting.readinessProbe
The readiness probe for the slurmdbd container.

object

sacctmgr ping; initialDelaySeconds 15; periodSeconds 10; failureThreshold 5; successThreshold 1; timeoutSeconds 60

accounting.replicas
The number of replicas of the accounting instance to run.

int

accounting.resources
Resources for the accounting container. object requests: cpu 4 memory 16Gi; limits: memory 16Gi

accounting.securityContext
Accounting container security context.

object

runAsUser/runAsGroup 401

accounting.startupProbe
Startup probe override for the slurmdbd container.

object

sacctmgr ping; periodSeconds 10; failureThreshold 360; successThreshold 1; timeoutSeconds 60

accounting.terminationGracePeriodSeconds
The termination grace period for the accounting pod.

int

cleanupCompleting.deleteInvalidNodes
Delete nodes with invalid states.

bool

false

cleanupCompleting.dryRun
Run in dry-run mode without making changes.

bool

false

cleanupCompleting.enabled
Enable cleanup of nodes with jobs stuck in COMPLETING state.

bool

true

cleanupCompleting.image
The image to use for cleanup-completing Job.

object

repository:
tag:

cleanupCompleting.intervalSeconds
Interval in seconds between cleanup checks.

int

cleanupCompleting.resources
Resources for the cleanup-completing Job container. object requests: cpu 100m memory 64Mi; limits: memory 256Mi

cleanupCompleting.timeoutSeconds
Timeout in seconds for jobs in COMPLETING state before downing nodes.

int

cleanupCompleting.verbose
Enable verbose logging for cleanup.

bool

false

compute.annotations
Additional annotations for compute services only. Use compute.nodes.custom-definition.annotations to add annotations to specific node definitions instead.

object

{}

compute.cacheDropper
Cache-dropper sidecar across all slurmd pods.

object

{
  "enabled": null,
  "resources": null
}

compute.cacheDropper.enabled
An option to enable or disable the cache-dropper sidecar container across all slurmd pods.

bool

true

compute.cacheDropper.resources
Resources for the cache-dropper sidecar container.

object

limits.memory 32Mi; requests.cpu 500m; requests.memory 32Mi

compute.cudaVersion
CUDA toolkit version for GPU compute nodes (e.g. “13.0”). When set, GPU NodeSets use the CUDA-specific extras image (slurmd-cw-cu<version>-extras) and the CUDA feature label (e.g. “cu130”) is injected into each GPU NodeSet’s static features.

string

"13.0"

compute.epilogConfigMap
The name or list of ConfigMap names containing epilog scripts. These user-managed ConfigMaps are combined with the operator-managed epilog ConfigMap into a single projected directory at /etc/slurm/epilog.d. Editing the contents of an attached ConfigMap hot-reloads without rolling compute pods; adding or removing a ConfigMap name is a pod-spec change and rolls the pods. To supply inline epilog content managed by the operator, use compute.managedEpilogConfig.

string | list

[]

compute.externalClusterName
The name of an external cluster to join when the control plane is deployed separately.

string, optional

null

compute.generateTopology
Enable topology generation for compute nodes. spec.options.generateTopology is always rendered; when this key is omitted from values, the chart uses default true for that field.

bool

true

compute.gpusd
Configuration for GPUSD (GPU Straggler Detection) metrics collection.

object

{
  "enabled": null,
  "version": null
}

compute.gpusd.enabled
Enable GPUSD package installation, metrics collection, and VMPodScrape resource.

bool

false

compute.gpusd.version
GPUSD version to install.

string

"1.0.0"

compute.initialState
Initial state when nodes join the cluster (idle, drain, etc.); may be set per node definition.

string

idle

compute.initialStateReason
Reason when initial state is down, drained, or fail; may be set per node definition.

string

Node added to the cluster for the first time

compute.labels
Additional labels for compute services only. Use compute.nodes.custom-definition.labels to add labels to specific node definitions instead.

object

{}

compute.livenessProbe
The liveness probe for the compute slurmd container. No default is applied by the operator; set this field to add a liveness probe.

object

none

compute.lowProfileRequests
Overlay low CPU/memory requests (10 CPU / 10Gi) on the slurmd container so the Kubernetes scheduler can place other workloads on the same nodes. This overlay is also applied whenever scheduler.enabled is true, regardless of this setting.

bool

true

compute.managedEpilogConfig
Map of epilog scripts rendered into /etc/slurm/epilog.d on compute pods. Keys are file names and values are file contents. The backing ConfigMap is always created and mounted as a directory, so editing values hot-reloads without rolling compute pods.

Example: click to expand

yaml managedEpilogConfig: 50-site-epilog.sh: | #!/usr/bin/env sh echo "site epilog"

object

{}

compute.managedPrologConfig
Map of prolog scripts rendered into /etc/slurm/prolog.d on compute pods, in addition to CoreWeave-provided defaults. Keys are file names and values are file contents. The backing ConfigMap is always created and mounted as a directory, so editing values hot-reloads without rolling compute pods. Supplying a key that matches a CoreWeave-provided default overrides that default. Only executable *.sh files are run.

Example: click to expand

yaml managedPrologConfig: 50-site-prolog.sh: | #!/usr/bin/env sh echo "site prolog"

object

{}

compute.maxUnavailable
Maximum unavailability of compute nodes during a rolling update (integer or percentage).

string

10%

compute.nodes
Multiple node definitions can be declared, but only one may be enabled: true. Node definitions can reference other definitions to include or overlay values. See the example below or the Compute Node Definitions documentation for more details. Each node definition also supports the following pod-level fields, applied to the rendered NodeSet pod template: - serviceAccountName (string, default “default”): the ServiceAccount the compute pods run as. The chart does not create the ServiceAccount, so it must already exist in the release namespace. - automountServiceAccountToken (bool, default false): whether to auto-mount the ServiceAccount token. Set to true (or attach a projected serviceAccountToken volume via volumes/volumeMounts) to expose credentials to in-pod tooling. - realMemory (quantity, default derived from resources.limits.memory): overrides the slurmd RealMemory value advertised to the Slurm controller for this node definition (e.g. 150Gi). Set this when the node’s true usable memory differs from the pod’s memory limit. The value is passed through to slurmd verbatim (no compute.reservedMemory headroom is subtracted from an explicit override). When unset, it defaults to resources.limits.memory converted to MB.

Example: click to expand

compute:
  nodes:
    # A custom definition to be referenced by other nodes
    custom-dns:
      dnsPolicy: "None"
      dnsConfig:
        nameservers:
          - 127.0.0.1
    # A simple CPU-only Node that uses the custom-dns definition above
    simple-cpu:
      enabled: true
      # ServiceAccount the pods run as; must already exist in the namespace.
      serviceAccountName: my-compute-sa
      # Mount the ServiceAccount token (or add a projected serviceAccountToken volume
      # under volumes/volumeMounts for a custom audience or mount path).
      automountServiceAccountToken: true
      # Partition creation for the nodeset. The nodeset will still be assigned to the `all`
      # partition if creation is disabled
      createPartition: true
      # The following Partition configurations are for the customer to update.
      # These values will be applied to the autogenerated partition and will override the defaults.
      partitionConfig:
        OverSubscribe: "YES"
        MaxTime: "12:00:00"
        QoS: "NORMAL"
      replicas: 1
      definitions:
        # Use the custom-dns definition
        - custom-dns
      staticFeatures:
        - cpu
      dynamicFeatures:
        node.coreweave.cloud/class: {}
      image:
        name: controller-extras
      gresGpu: null
      config:
        weight: 1
      # Create a small node with 1cpu and 1g memory
      resources:
        limits:
          memory: 1Gi
          cpu: 1
        requests:
          memory: 1Gi
          cpu: 1
      # Advertise less RealMemory to Slurm than the pod's memory limit, e.g. to
      # leave headroom for node-level overhead not captured by the pod limit.
      realMemory: 900Mi
      tolerations:
        - key: is_cpu_compute
          operator: Exists
      volumeMounts:
        - name: ramtmp
          mountPath: /tmp
      volumes:
        - name: ramtmp
          emptyDir:
            medium: Memory
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: kubernetes.io/os
                    operator: In
                    values:
                      - linux

Compute node definitions are composed from the presets in charts/slurm/compute-defs/ and supplied per-deployment (each entry sets enabled/replicas/definitions). The base chart ships no preset catalog here; CW and other environments define their nodes in their own values overlays.

object

See Compute Node Definitions.

compute.plugstackConfig
Additional plug-in stack configuration items for plugstack.conf file config. Config Options: https://slurm.schedmd.com/spank.html#SECTION_CONFIGURATION

list

[]

compute.ports
Additional ports to expose on the compute nodes.

Example: NCCL Plugin ports

ports:
  - containerPort: 10400
    protocol: TCP
  - containerPort: 10401
    protocol: TCP
  - containerPort: 10402
    protocol: TCP
  - containerPort: 10403
    protocol: TCP
  - containerPort: 10404
    protocol: TCP
  - containerPort: 10405
    protocol: TCP
  - containerPort: 10406
    protocol: TCP
  - containerPort: 10407
    protocol: TCP

list

[]

compute.prologConfigMap
The name or list of ConfigMap names containing prolog scripts. These user-managed ConfigMaps are combined with the operator-managed prolog ConfigMap into a single projected directory at /etc/slurm/prolog.d. Editing the contents of an attached ConfigMap hot-reloads without rolling compute pods; adding or removing a ConfigMap name is a pod-spec change and rolls the pods. To supply inline prolog content managed by the operator (and to override CoreWeave defaults), use compute.managedPrologConfig.

string | list

[]

compute.pyxis
Pyxis / enroot integration for compute nodes.

object

{
  "appArmorProfile": null,
  "enabled": null,
  "enrootConfig": null,
  "plugstackOptions": null,
  "podSecurityContext": null
}

compute.pyxis.appArmorProfile
AppArmor profile for pyxis nodeset containers. When omitted, defaults to localhost/enroot.

string, optional

null

compute.pyxis.enabled
Enable the pyxis container.

bool

true

compute.pyxis.enrootConfig
enroot.conf overrides.

object, optional

null

compute.pyxis.plugstackOptions
Additional arguments for the pyxis plugin in plugstack.conf file config. Config Options: https://github.com/NVIDIA/pyxis/wiki/Setup#slurm-plugstack-configuration

list, optional

null

compute.pyxis.podSecurityContext
Security context for pyxis nodeset pods. When omitted, defaults to a Localhost seccomp profile targeting profiles/enroot.

object, optional

null

compute.readinessProbe
The readiness probe for the compute slurmd container. When omitted the NodeSet controller injects the default scontrol show slurmd probe. object exec: [scontrol, show, slurmd]; failureThreshold: 3; periodSeconds: 10; successThreshold: 1; timeoutSeconds: 5

compute.reservedMemory
Reserved memory when calculating DefMemPerCPU config for slurm.conf

string

4Gi

compute.s6
oneshot and longrun jobs are supported. See Running Scripts with S6 for more information.

Example: click to expand

s6:
  packages:
    type: oneshot
    timeoutUp: 360000 # 360 seconds
    timeoutDown: 0
    script: |
      #!/usr/bin/env bash
      apt -y update
      apt -y install nginx
  nginx:
    type: longrun
    timeoutUp: 360000 # 360 seconds
    timeoutDown: 0
    script: |
      #!/usr/bin/env bash
      nginx -g "daemon off;"

object

[]

compute.securityContext
Security context for the slurmd container (NodeSet pod template and SlurmCluster compute spec). Omit for operator defaults; set to add capabilities (e.g. SYS_ADMIN for pyxis).

object

operator-applied defaults

compute.ssh
SSH / sshd settings for compute nodes.

object

{
  "enabled": null
}

compute.ssh.enabled
Enable ssh to the compute nodes.

bool

false

compute.startupProbe
The startup probe for the compute slurmd container. When RDMA/InfiniBand resources are present the NodeSet controller injects an IB-specific startup probe automatically; set this field only to override that behaviour.

object

operator-injected IB probe when rdma/ib limits are present; omitted otherwise

compute.umask
Default umask applied to user shells on both compute and login pods. When set, the chart injects SUNK_UMASK into the relevant pod environments. Leave unset to use the image default (0022). Values are passed through verbatim, so any form accepted by the umask builtin works (for example “0027” or “u=rwx,g=rwx,o=“).

string

""

compute.volumeMounts
Additional volume mounts for all compute pods. Appended to the mounts derived from the top-level storage block; on a mountPath collision the entry here wins.

list

[]

compute.volumes
Additional volumes for all compute pods. Appended to the volumes derived from the top-level storage block; on a name collision the entry here wins. (Login pods take the same storage-derived volumes plus any under login.{common,userPods,groupPods}.pod.volumes.)

list

[]

controller.etcConfigMap
The ConfigMap(s) with keys mapping to files in /etc/slurm on the controller and compute slurmd pods.
This ConfigMap must not contain:

slurm.conf
plugstack.conf
gres.conf
cgroup.conf
topology.conf

string | list

null

controller.image
The image to use for the controller.

object

repository:
tag:

controller.replicas
The number of replicas of the controller to run, currently should be left at 1.

int

controller.resources
Resources for the controller container. object requests: cpu 4 memory 16Gi; limits: memory 16Gi

controller.securityContext
Security context for the controller container.

object

runAsUser/runAsGroup 401

controller.stateVolume
Controller’s persistent volume claim.

object

{
  "size": null,
  "storageClassName": null
}

controller.stateVolume.size
The size of the persistent volume claim.

string

32Gi

controller.stateVolume.storageClassName
The storage class name to use for the volume.

string

omit; Kubernetes uses the cluster default storage class when unset.

controller.terminationGracePeriodSeconds
The termination grace period for the controller.

int

controller.volumeMounts
Additional volume mounts to apply to the controller pod.

list

[]

controller.volumes
Additional volumes to mount to the controller pod.

list

[]

controller.watch
Configure the watch sidecar. Omit subkeys to use CRD / operator defaults.

object

{
  "enabled": null,
  "interval": null
}

controller.watch.enabled
Enable the controller watch sidecar.

bool

true

controller.watch.interval
Interval in seconds between watch sidecar checks.

int

database.mysqlCluster
MySQL (MOCO) backend configuration, used when accounting.database.backend is moco. Each sub-key below is passed through to the SlurmCluster CR’s database spec when set; omit a key (leave it null) to keep the operator default.

object

See the per-field defaults below.

database.mysqlCluster.config
Additional MySQL configuration for my.cnf as key-value pairs. The operator automatically calculates the following values based on resources.limits.memory: - innodb_buffer_pool_size: 50% of memory limit - innodb_redo_log_capacity: 25% of innodb_buffer_pool_size - innodb_lock_wait_timeout: 900 (default) User-provided values in this config will override the calculated defaults. Note: If you override innodb_buffer_pool_size, innodb_redo_log_capacity will be recalculated based on your override (unless you also explicitly override it). Validation: innodb_redo_log_capacity cannot exceed persistence.size. MySQL option file documentation

Example: click to expand

config:
  max_connections: "1000"
  wait_timeout: "28800"
  innodb_buffer_pool_size: "17179869184"  # Override to 16Gi

map[string]string

Operator calculates innodb values from resources.limits.memory

database.mysqlCluster.image
The image to use for mysql. Passed through verbatim when set.

object

ghcr.io/cybozu-go/moco/mysql:8.4.6

database.mysqlCluster.inodeLockFixer
Init container that copies/moves/replaces the MySQL data directory to prevent known inode locking issues in certain storage environments. Only used when accounting.database.backend is moco; no effect when backend is cwdb. Sub-keys (passed through verbatim when set): enabled (bool) and image (object, repository/tag). object enabled: false, image: alpine:3.20.0

database.mysqlCluster.persistence
The volume settings to use for mysql. Passed through verbatim when set. object size: 512Gi, accessModes: [ReadWriteOnce] (operator defaults); storageClassName unset uses the cluster default StorageClass

database.mysqlCluster.replicas
Number of MySQL replicas.

int

database.mysqlCluster.resources
Resources for the database container. Passed through verbatim when set. object requests: {memory: 64Gi, cpu: “16”}, limits: {memory: 64Gi}

jwt.existingSecret
The name of an existing secret containing the JWT private key; otherwise the chart generates one.

string

(omit; chart generates a Secret)

login.common
Config used for both userPods and groupPods. Any value here can be overridden in the respective userPods or groupPods blocks.

object

{
  "pod": {
    "affinity": {},
    "automountServiceAccountToken": null,
    "containers": [],
    "env": [],
    "hostAliases": [],
    "image": {
      "repository": null,
      "tag": null
    },
    "podSecurityContext": {},
    "resources": {},
    "securityContext": {},
    "serviceAccountName": null,
    "tolerations": [],
    "volumeMounts": [],
    "volumes": []
  },
  "service": {
    "additionalPorts": [],
    "annotations": {},
    "enabled": null,
    "externalTrafficPolicy": null,
    "individual": {},
    "labels": {},
    "loadBalancerClass": null,
    "type": null
  }
}

login.common.pod.affinity
Affinity constraining which nodes login pods schedule onto. Login pods do not inherit slurmCluster.spec.affinity (which targets control-plane components), so this is the only affinity applied to login pods. When unset, login pods receive no affinity.

object

map[]

login.common.pod.automountServiceAccountToken
Automatically mount the service account token into the login pod.

bool

false

login.common.pod.containers
Additional sidecar containers to add to the login pod.

list

[]

login.common.pod.env
Additional environment variables passed to the sshd container.

list

[]

login.common.pod.hostAliases
Pod-level override of hostname resolution when DNS and other options are not applicable. See Adding entries to Pod /etc/hosts with HostAliases.

list

[]

login.common.pod.image
Image used by the login pod sshd container. When repository is omitted, the chart builds <slurm.registry>/<name> with name defaulting to controller-extras.

object

repository:
tag:

login.common.pod.podSecurityContext
Pod-level security context applied to the login pod.

object

map[]

login.common.pod.resources
sshd container resources. object requests: cpu 4 memory 8Gi; limits: memory 8Gi

login.common.pod.securityContext
Container-level security context applied to the login sshd container. Exposed to support workloads such as enroot that require capabilities (e.g. SYS_ADMIN) or a seccomp profile.

object

map[]

login.common.pod.serviceAccountName
Service account name for the login pod.

string

"default"

login.common.pod.tolerations
Tolerations allowing login pods to schedule onto tainted nodes. When set, these replace the cluster-wide slurmCluster.spec.tolerations for login pods; when unset, the cluster-wide tolerations apply.

list

[]

login.common.pod.volumeMounts
Additional volume mounts for the login pods. Appended to the mounts derived from the top-level storage block; on a mountPath collision the entry here wins. (Compute pods take the same storage-derived mounts plus any under compute.volumeMounts.)

list

[]

login.common.pod.volumes
Additional volumes for the login pods. Appended to the volumes derived from the top-level storage block; on a name collision the entry here wins. (Compute pods take the same storage-derived volumes plus any under compute.volumes.)

Example: click to expand

volumes:
  - name: cache-vol
    emptyDir:
      medium: Memory

list

[]

login.common.service.additionalPorts
Additional port definitions to expose on each login Service.

Example: click to expand

additionalPorts:
  - name: eternal-shell
    port: 2022
    targetPort: 20222 # optional
    protocol: TCP # optional

list

[]

login.common.service.annotations
Annotations applied to every per-entity login Service. %s is replaced with the entity name (user or group) and %h with a 7-character hash of the entity name.

object

{}

login.common.service.enabled
Enable the creation of service(s) for login pods.

bool

true

login.common.service.externalTrafficPolicy
External traffic policy.

string

Local

login.common.service.individual
Per-entity Service metadata overrides keyed by user or group name. Each value is an object with optional annotations and labels maps; overrides are applied on top of the kind’s annotations/labels and win on conflict.

Example: click to expand

individual:
  jdoe:
    annotations:
      dns.example.com/hostname: jdoe.login.example.com

object

{}

login.common.service.labels
Labels applied to every per-entity login Service. %s/%h substitution is supported.

object

{}

login.common.service.loadBalancerClass
Load balancer class to use for login Services.

string

null

login.common.service.type
The type of service to create. This defaults to LoadBalancer for cloud deployments. For development and test systems without an external load balancer to handle the service routing, such as when deploying on kind (Kubernetes IN Docker), this may be set to ClusterIP.

string

LoadBalancer

login.groupPods
Per-group login pod overrides (one shared StatefulSet per group). Same shape as login.common; values set here override the corresponding fields from common.

Example: click to expand

groupPods:
  pod:
    resources:
      requests:
        cpu: 4
        memory: 8Gi
      limits:
        memory: 8Gi
  service:
    annotations:
      service.beta.kubernetes.io/external-hostname: "groups.login.example.com"

object

map[]

login.groups
Groups to enumerate for per-user and per-group login pods. Users are resolved from nsscache secrets. Each entry has required name and optional userPods (default true) and groupPods (default true) to control whether per-user pods and the shared group pod are created for that group.

Example: click to expand

groups:
  - name: slurm-users
    userPods: true
    groupPods: true
  - name: sudo-users
    userPods: true
    groupPods: false

list

[]

login.s6
Additional s6 services for login pods, shared across all login pods in the cluster. oneshot and longrun jobs are supported. See Running Scripts with S6 for more information.

Example: click to expand

s6:
  packages:
    type: oneshot
    timeoutUp: 360000 # 360 seconds
    timeoutDown: 0
    script: |
      #!/usr/bin/env bash
      apt -y update
      apt -y install nginx
  nginx:
    type: longrun
    timeoutUp: 360000 # 360 seconds
    timeoutDown: 0
    script: |
      #!/usr/bin/env bash
      nginx -g "daemon off;"

object

{}

login.sshKeyVolume
The shared SSH host keys persistent volume claim. One PVC is provisioned per cluster and mounted by every login pod (both userPods and groupPods).

object

{
  "accessModes": null,
  "enabled": null,
  "size": null,
  "storageClassName": null
}

login.sshKeyVolume.accessModes
PVC access modes. If scaling login beyond a single replica per kind this must be ReadWriteMany. In a development setting with a volume provider that doesn’t support ReadWriteMany, such as kind, set to [ReadWriteOnce].

list

[ReadWriteMany]

login.sshKeyVolume.enabled
Enable provisioning of the shared SSH host keys PVC.

bool

true

login.sshKeyVolume.size
PVC size.

string

1Gi

login.sshKeyVolume.storageClassName
Storage class name to use for the PVC.

string

omit; Kubernetes uses the cluster default storage class when unset.

login.sshdConfig
Additional sshd configuration entries. Rendered once per cluster into a shared ConfigMap mounted by every login pod.

Example: click to expand

sshdConfig:
  PasswordAuthentication: "no"

object

PasswordAuthentication: "no"

login.userPods
Per-user login pod overrides. Same shape as login.common; values set here override the corresponding fields from common. Anything not set here inherits from common.

Example: click to expand

userPods:
  pod:
    resources:
      requests:
        cpu: 4
        memory: 8Gi
      limits:
        memory: 8Gi
  service:
    annotations:
      service.beta.kubernetes.io/external-hostname: "%s.login.example.com"

object

map[]

munge.args
The additional arguments to pass to the munge container. The defaults run Munge with 10 threads instead of 2.

list

[
  "--num-threads",
  "10"
]

munge.resources
Resources for the munge container. object requests: cpu 1 memory 2Gi; limits: memory 2Gi

nsscache.defaultShell
The default shell for all users (used for both LDAP and SCIM).

string

/bin/bash

nsscache.enabled
Enable nsscache. Set explicitly to false to disable nsscache-related volumes,

bool

true

nsscache.existingSecret
Name of an existing k8s secret containing the authentication credentials for this domain. For SCIM, this secret should contain a key named nsscache-scim-auth-token which contains the token to use for the SCIM server. For LDAP, this secret should contain a key named nsscache-ldap-password which contains the password to use for the LDAP bind DN.

string

null

nsscache.fetchInterval
The interval in seconds between nsscache updates and Slurm user provisioning sync runs.

int

nsscache.groups
List of groups for both POSIX users and Slurm user creation. Example: groups: - slurm-users - sudo-users

list

[]

nsscache.image
The image to use for nsscache containers.

object

repository:
tag:

nsscache.nsscacheConfig
Options for defining nsscache.conf. Click to expand examples. Per-section nsscache.conf overrides. Leave unset to use the chart defaults. Keys live in a default section plus optional per-map sections (passwd, group, shadow, sshkey). SCIM is the CoreWeave default source; LDAP is supported. default section

Key	Type	Description	Default
`maps`	list	List of nsscache maps to fetch. The chart joins these into a comma-separated string when rendering `nsscache.conf`. The defaults are:	`passwd, shadow, group, sshkey`
`files_dir`	string	Directory location to store the plain text files in.	`/etc/nsscache`
`source`	string	Specify the data source to use. Supported options are `scim` and `ldap`.	`scim`
`scim_base_url`	string	The base URL for the SCIM server. Example: `https://api.coreweave.com/scim/<org>`	—
`scim_users_parameters`	string	Option to use URL parameters for the users endpoint. Special characters (spaces, quotes, etc.) are automatically URL encoded. There is a custom parameter for filtering by groups that is a comma-separated list. A filter for inactive users is included by default. Including the bare `cursor` key opts into nsscache’s cursor-based SCIM pagination (one page per `nextCursor` token) instead of the legacy startIndex/totalResults loop; omit it to keep the legacy behavior. Example: `filter=active eq “true”&groups=slurm-users,sudo-users&cursor`	`filter=active eq “true”&cursor`
`scim_groups_parameters`	string	Option to use URL parameters for the groups endpoint. Special characters (spaces, quotes, etc.) are automatically URL encoded. There is a custom parameter for creating virtual user groups that is a comma-separated list: it creates an entry in the groups map for the user’s gid for the members of the selected group(s). This parameter typically should match any group filtering in `scim_users_parameters`. A filter for inactive users is included by default. Including the bare `cursor` key opts into cursor-based pagination. Example: `excludeInactiveUsers=true&includeVirtualUserGroups=slurm-users,sudo-users&cursor`	`excludeInactiveUsers=true&cursor`
`ldap_uri`	string	The LDAP URI to connect to. Example: `ldap://authentik-outpost-ldap-outpost`	—
`ldap_base`	string	The base to perform LDAP searches under. Example: `dc=coreweave,dc=cloud`	—
`ldap_bind_dn`	string	The bind DN to use when connecting to LDAP. Empty string is an anonymous bind. Example: `cn=ldapsvc,dc=coreweave,dc=cloud`	—
`ldap_bind_password`	string	The password to use for the LDAP bind DN. We strongly recommend storing this in a Kubernetes secret and referencing it via `nsscache.existingSecret` rather than setting it here.	—
`ldap_rfc2307bis`	int	Set to `1` if groups are stored as a list of DNs in the `member` attribute (rfc2307bis schema); the default uses rfc2307. Example: `1`	—
`ldap_scope`	string	The search scope to use: `sub`, `one`, or `base`.	`one`
`ldap_uidattr`	string	The uid-like attribute in your directory.	`uid`
`ldap_default_shell`	string	Default login shell for users that do not specify one.	—

The SCIM auth token is supplied via nsscache.existingSecret (key nsscache-scim-auth-token), not in this config.

Key	Type	Description	Default
`ldap_filter`	string	The search filter to use when querying. Example: `(objectClass=user)`	—
`ldap_override_home_dir`	string	Override every user’s home directory. `%%u` substitutes the username. Example: `/mnt/home/%%u`	—

Key	Type	Description	Default
`ldap_filter`	string	The search filter to use when querying. Example: `(objectClass=user)`	—

Key	Type	Description	Default
`ldap_filter`	string	The search filter to use when querying. Example: `(objectClass=group)`	—

Key	Type	Description	Default
`ldap_filter`	string	The search filter to use when querying. Example: `(objectClass=user)`	—

SCIM example (default)

nsscacheConfig:
  default:
    source: scim
    scim_base_url: https://api.coreweave.com/scim/abc123
    scim_users_parameters: filter=active eq "true"&groups=slurm-users,sudo-users&cursor
    scim_groups_parameters: excludeInactiveUsers=true&includeVirtualUserGroups=slurm-users,sudo-users&cursor

LDAP example

nsscacheConfig:
  default:
    source: ldap
    ldap_uri: ldap://authentik-outpost-ldap-outpost
    ldap_base: dc=coreweave,dc=cloud
    ldap_bind_dn: cn=ldapsvc,dc=coreweave,dc=cloud
    ldap_bind_password:
    ldap_rfc2307bis: 1
    ldap_default_shell: /bin/bash
    ldap_scope: sub
    ldap_uidattr: cn
  passwd:
    ldap_filter: (objectClass=user)
    ldap_override_home_dir: /mnt/home/%%u
  group:
    ldap_filter: (objectClass=group)
  shadow:
    ldap_filter: (objectClass=user)
  sshkey:
    ldap_filter: (objectClass=user)

SCIM

nsscacheConfig:
  default:
    source: scim
    scim_base_url: https://api.coreweave.com/scim/abc123
    scim_users_parameters: filter=active eq "true"&groups=slurm-users,sudo-users
    scim_groups_parameters: excludeInactiveUsers=true&includeVirtualUserGroups=slurm-users,sudo-users

object

See the nsscache.conf documentation.

nsscache.overrideHomeDirectory
The home directory pattern to override for all users. %%u will be replaced with the username (used for both LDAP and SCIM).

string

/mnt/home/%%u

nsscache.slurmUserProvisioning
Configuration for slurm user provisioning.

object

{
  "defaultSlurmAccount": null,
  "enabled": null
}

nsscache.slurmUserProvisioning.defaultSlurmAccount
The default Slurm account for automated provisioning of users.

string

cw-sup

nsscache.slurmUserProvisioning.enabled
Enable slurmUserProvisioning.

bool

true

nsscache.sudoGroups
List of Unix groups with sudo privileges.

list

[]

partitions
Partitions configuration for the slurm cluster The key is the partition name and the value is the partition configuration.The following partition keys are special keys that defines partition configuration:

auto: If enabled, it reads nodeset information and automatically creates partition. Supports enabled and config; config entries are merged case-insensitively into every auto-generated partition (PartitionName/Nodeset/Nodes/Default are rejected).
all: Creates a partition for all nodes, and set to default if no default partition is set. Overriding it keeps Nodes=ALL unless the override selects nodes itself.

Partitions may omit node selection (feature / a Nodes config entry); Slurm treats them as partitions with no nodes. Disabling all requires at least one other enabled partition; an enabled auto counts, but only yields partitions once NodeSets are discovered.

Partition Configuration

partitions:
  hpc-mid:
    enabled: true
    feature: ALL
    config:
      PriorityTier: "16384"
      Priority: "16384"
      DefCpuPerGPU: "16"
      DefMemPerCPU: "15500"
      MaxTime: "INFINITE"
      State: "UP"

object

{
  "auto": {
    "config": null,
    "enabled": null
  }
}

partitions.auto
auto is a reserved key that defines if nodeset detection is automated added to partitions. Supports enabled and config; other partition fields are rejected at admission.

object

{
  "config": null,
  "enabled": null
}

partitions.auto.enabled
Enable automatic partition generation from discovered NodeSets.

bool

true

rest.args
Arguments passed to slurmrestd. The listen address is appended automatically by the operator. Use -d to optionally pin a data_parser plugin (e.g. “-dv0.0.44”).

list

["-vv", "-sslurmdbd,slurmctld"]

rest.containers
Additional sidecar containers to add to the restd pod.

list

[]

rest.enabled
When true, the chart emits restd.enabled: true on the SlurmCluster and the operator creates slurmrestd. When omitted, the chart does not emit spec.restd from this toggle.

bool

false

rest.image
Image override for the slurmrestd container. Defaults to cluster slurmImage.

object

repository:
tag:

rest.livenessProbe
Liveness probe override for the slurmrestd container.

object

TCP socket on the slurmrestd port; periodSeconds 10; failureThreshold 2

rest.readinessProbe
Readiness probe override for the slurmrestd container.

object

TCP socket on the slurmrestd port; periodSeconds 5; failureThreshold 1

rest.replicas
The number of replicas of the rest pod to run.

int

rest.resources
Resources for the slurmrestd container.
These defaults are appropriate for small and medium-sized clusters. object requests: cpu 2 memory 8Gi; limits: memory 64Gi

rest.service.additionalPorts
Additional port definitions to expose on the Service.

Example: click to expand

additionalPorts:
  - name: proxy
    port: 8080
    targetPort: 8080 # optional
    protocol: TCP # optional

list

[]

rest.service.annotations
Additional annotations to apply to the slurmrestd Service.

object

{}

rest.service.clusterIP
The clusterIP for the Service. Use None for a headless Service.

string

None

rest.service.enabled
Enable the creation of the Service for the slurmrestd pod.

bool

true

rest.service.externalName
The external DNS name the Service resolves to. Only valid when type is ExternalName; the operator then drops the clusterIP, selector, and ports.

string

null

rest.service.externalTrafficPolicy
The external traffic policy.

string

null

rest.service.labels
Additional labels to apply to the slurmrestd Service.

object

{}

rest.service.loadBalancerClass
The load balancer class to use when type is LoadBalancer.

string

null

rest.service.type
The type of Service to create. One of ClusterIP, NodePort, LoadBalancer, or ExternalName.

string

ClusterIP

rest.startupProbe
Startup probe override for the slurmrestd container.

object

TCP socket on the slurmrestd port; periodSeconds 2; failureThreshold 20

rest.volumeMounts
Additional volume mounts to apply to the rest container (passed through to CR when set).

list

[]

rest.volumes
Additional volumes to add to the restd pod (passed through to CR when set).

list

[]

scheduler.config
Scheduler and Slurm client configuration.

object

{
  "scheduler": {},
  "slurm": {}
}

scheduler.config.scheduler
Scheduler-specific configuration.

Operator defaults

Key	Type	Description	Default
`gpuTypes`	map	Mapping of k8s GPU types (node affinity key `gpu.nvidia.com/class`) to Slurm gres GPU types; added to a job’s description.	`{A100_NVLINK_80GB: a100, H100_NVLINK_80GB: h100}`
`pollInterval`	string	Polling interval for the Slurm API.	`10s`
`terminationOffset`	string	Offset to the termination grace period to account for communication delays.	`5s`

scheduler:
  gpuTypes:
    H100_NVLINK_80GB: h100
    A100_NVLINK_80GB: a100
  pollInterval: 10s
  terminationOffset: 5s

object

See the operator defaults below.

scheduler.config.slurm
Slurm client configuration.

Operator defaults

Key	Type	Description	Default
`poolSize`	int	Number of connections to maintain in the connection pool.	`10`
`usePersistentConnection`	bool	Use Slurm’s persistent connections for connection reuse.	`true`

slurm:
  poolSize: 10
  usePersistentConnection: true

object

See the operator defaults below.

scheduler.enabled
Enable the scheduler.
To schedule k8s pods on the Slurm cluster nodes, this must be enabled.

bool

false

scheduler.image
The image to use for the scheduler.

object

repository:
tag:

scheduler.resources
Resources for the scheduler container. object requests: cpu 200m memory 1Gi; limits: memory 1Gi

scheduler.scope
Scheduler scope configuration. The type sub-key (string) selects the scope: cluster or namespace. When type is namespace, set namespaces below. scope is passed through verbatim, so set type only when overriding the default.

object

type: cluster

scheduler.scope.namespaces
The list of the namespaces to scope the scheduler to.
Only used when scope.type is set to namespace. Namespaces other than the release namespace will need role bindings created.

list

[]

slurmCluster.spec
Specification for the SlurmCluster custom resource

object

{
  "affinity": {
    "nodeAffinity": {
      "requiredDuringSchedulingIgnoredDuringExecution": {
        "nodeSelectorTerms": [
          {
            "matchExpressions": [
              {
                "key": "node.coreweave.cloud/class",
                "operator": "In",
                "values": [
                  "cpu"
                ]
              }
            ]
          }
        ]
      }
    }
  },
  "annotations": {},
  "dnsConfig": {
    "additionalSearches": []
  },
  "imagePullSecrets": [],
  "labels": {},
  "priorityClassName": null,
  "protocolVersion": null,
  "slurmImage": {
    "repository": null,
    "tag": null
  },
  "sunkImage": {
    "repository": null,
    "tag": null
  },
  "tolerations": null,
  "version": null
}

slurmCluster.spec.affinity
Affinity for control plane component deployments. The default schedules control-plane components onto CoreWeave CPU nodes; set to {} or override to change scheduling.

object

nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
            - matchExpressions:
                - key: node.coreweave.cloud/class
                  operator: In
                  values:
                    - cpu

slurmCluster.spec.annotations
Additional custom annotations to add to control plane component resources (deployment, service, etc.)

object

{}

slurmCluster.spec.dnsConfig.additionalSearches
A list of namespaces to add to the list of DNS searches. These additional searches extend hostname lookup in the control-plane, compute, and login pods. Default dns searches: - name-compute.namespace.svc.cluster.local - slurm_cluster_name-controller.namespace.svc.cluster.local

list

[]

slurmCluster.spec.imagePullSecrets
Image pull secrets for all cluster components, including compute (NodeSet) pods.

list

[]

slurmCluster.spec.labels
Additional custom labels to add to control plane component resources (deployment, service, etc.)

object

{}

slurmCluster.spec.priorityClassName
Priority class name for all control plane components.

string

sunk-control-plane

slurmCluster.spec.protocolVersion
The protocol version to use for communication with the Slurm controller.

string

current wire protocol

slurmCluster.spec.slurmImage
Default image configuration for Slurm components (slurmctld, login nodes, nsscache, cleanup-completing) when component-specific images are not specified

object

{
  "repository": null,
  "tag": null
}

slurmCluster.spec.slurmImage.repository
Container image repository

string

ghcr.io/coreweave/slurm-containers/controller

slurmCluster.spec.slurmImage.tag
Container image tag

string

v{Chart.AppVersion}-ubuntu24.04

slurmCluster.spec.sunkImage
Default image configuration for operator components (syncer, scheduler) when component-specific images are not specified

object

{
  "repository": null,
  "tag": null
}

slurmCluster.spec.sunkImage.repository
Container image repository

string

images.coreweave.com/cluster-images/sunk/operator

slurmCluster.spec.sunkImage.tag
Container image tag

string

v{Chart.Version}

slurmCluster.spec.tolerations
Additional custom tolerations to add to control plane component resources (deployment, service, etc.)

list

null

slurmCluster.spec.version
Slurm version selector passed through to the SlurmCluster spec when set.

string

null

slurmConfig
Slurm cluster configuration overrides (slurm.conf and cgroup.conf). Most values are managed by the operator defaults and do not need to be set here. Keys you add under slurmConfig are passed through to the SlurmCluster CRD as user overrides and take precedence over the operator defaults. cgroup.conf overrides go under the nested cgroupConfig. The operator-set defaults are listed below for reference.

Operator defaults

slurmConfig:
  AccountingStorageEnforce: qos,limits
  AccountingStorageTRES: gres/gpu
  AccountingStorageType: accounting_storage/slurmdbd
  AuthAltParameters: jwt_key=/etc/jwt/jwt.key
  AuthAltTypes: auth/jwt
  BatchStartTimeout: "120"
  CommunicationParameters: KeepAliveTime=60,keepaliveinterval=10,keepaliveprobes=3
  DebugFlags: NO_CONF_HASH
  DefMemPerCPU: "4096"
  Epilog: /usr/share/sunk/bin/epilog.sh
  GresTypes: gpu
  InactiveLimit: "0"
  JobAcctGatherFrequency: "30"
  JobAcctGatherType: jobacct_gather/cgroup
  JobCompType: jobcomp/none
  KillWait: "30"
  MaxNodeCount: "3072"
  MessageTimeout: "100"
  MinJobAge: "300"
  MpiDefault: pmix
  ProctrackType: proctrack/cgroup
  Prolog: /usr/share/sunk/bin/prolog.sh
  PrologFlags: Alloc,Serial
  RebootProgram: /usr/share/sunk/bin/reboot.sh
  ReturnToService: "2"
  SchedulerParameters: nohold_on_prolog_fail,max_rpc_cnt=256
  SchedulerType: sched/backfill
  SelectType: select/cons_tres
  SelectTypeParameters: CR_CPU_MEMORY
  SlurmSchedLogFile: /dev/null
  SlurmSchedLogLevel: "1"
  SlurmUser: slurm
  SlurmctldDebug: verbose
  SlurmctldLogFile: /dev/null
  SlurmctldParameters: conmgr_max_connections=512,conmgr_threads=32,idle_on_node_suspend,node_reg_mem_percent=95,rl_enable,rl_log_freq=30,rl_bucket_size=50,rl_refill_period=1,rl_refill_rate=10
  SlurmctldPidFile: /var/run/slurmctld.pid
  SlurmctldPort: "6817"
  SlurmctldTimeout: "60"
  SlurmdDebug: verbose
  SlurmdLogFile: /proc/1/fd/1
  SlurmdPidFile: /var/run/slurmd.pid
  SlurmdPort: "6818"
  SlurmdSpoolDir: /var/spool/slurmd
  SlurmdTimeout: "60"
  StateSaveLocation: /var/spool/slurmctld/save
  SUNKJobDashboardURL: https://cks-grafana.coreweave.com/d/slurm-job-metrics/slurm-job-metrics
  SUNKNodeDashboardURL: https://cks-grafana.coreweave.com/d/ddbdicm9sw7c5x/node-details
  SuspendTime: INFINITE
  SwitchType: switch/none
  TCPTimeout: "15"
  TaskPlugin: task/cgroup,task/affinity
  TaskPluginParam: SlurmdSpecOverride
  TopologyParam: TopoOptional
  TopologyPlugin: topology/tree
  TreeWidth: "65533"
  UnkillableStepProgram: /usr/share/sunk/bin/unkillable-step.sh
  UnkillableStepTimeout: "900"
  WaitTime: "0"
  cgroupConfig:
    CgroupPlugin: cgroup/v2
    ConstrainCores: "yes"
    ConstrainDevices: "yes"
    ConstrainRAMSpace: "yes"
    IgnoreSystemd: "yes"

object

See the operator defaults below.

slurmConfig.SlurmJobSkipIds
The controller skips processing this list of Slurm JobIds. Comma separated list without spaces of JobIds to skip. Example: “2,4,6,8”

string

null

storage
Shared cluster storage volumes. Adopts the entry shape of the external slurm-supporting chart (name / storageClassName / size), so an existing storage list can be carried over on upgrade with no edits. Empty by default (opt-in). For each entry that sets mountPath, the chart derives a PVC-backed volume + mount and wires it into both the login pods (common, userPods, groupPods) and the compute nodesets. Any volumes/mounts declared directly under login.{common,userPods,groupPods}.pod or compute are appended to the storage-derived set; on a name/mountPath collision the directly-declared entry wins. Whether a PVC is created is controlled solely by create:

name (string, required) — PVC name; also the pod volume name and the claim the mount references.
create (bool, optional, default false) — when true the chart creates the PVC; when false it assumes the claim already exists and only references it (bring-your-own / the orphan-then-adopt upgrade path). size/storageClassName/ accessModes are used only when create is true.
mountPath (string, optional) — when set, the claim is mounted at this path on login and compute pods; when omitted the entry only manages the PVC (create or reference) and is not auto-mounted.
size (string, required when create) — requested storage.
storageClassName (string, optional) — omitted when empty so Kubernetes uses the cluster default storage class.
accessModes (list, optional) — defaults to [ReadWriteMany], required to share a volume across pods.

Created PVCs are annotated helm.sh/resource-policy: keep and argocd.argoproj.io/sync-options: Prune=false,Delete=false so an upgrade or prune never deletes the data. When adopting an existing PVC (create: false, or a create=true whose spec ArgoCD applies onto a live volume) the rendered spec must match the existing PVC’s immutable fields. Overriding storage replaces the whole list (Helm does not merge lists), so an override must re-list every volume it wants, including its mountPath.

list

[]

syncer.config
Configuration for the syncer component.

object

{
  "hooksAPI": {},
  "slurm": {},
  "syncer": {}
}

syncer.config.hooksAPI
Webhooks API configuration

Operator defaults

hooksAPI:
  nodeRebootCondition: PhaseState
  nodeRebootReason: production-powerreset
  safeNodeRebootCondition: PendingPhaseState
  safeNodeRebootReason: production-powerreset
  waitForPodDeletionInterval: 1s
  waitForNodeLockedInterval: 1s
  waitForNodeLockedTimeout: 120s

object

See the operator defaults below.

syncer.config.slurm
Slurm client configuration

Operator defaults

Key	Type	Description	Default
`poolSize`	int	Number of connections to maintain in the connection pool.	`10`
`usePersistentConnection`	bool	Use Slurm’s persistent connections for connection reuse.	`true`

slurm:
  poolSize: 10
  usePersistentConnection: true

object

See the operator defaults below.

syncer.config.syncer
Syncer-specific configuration. Copy a key from the reference below only when overriding it.

Operator defaults

syncer:
  qosInterruptable: qos.coreweave.cloud/interruptable
  orphanedPodDelay: 120s
  pollInterval: 10s
  slurmNodeCleanUp: true
  reconfigureRateLimit: 3600s
  logLevel: info
  maxConcurrentReconciles: 50

nodesetUpdateJobPreemption — preempt jobs during NodeSet rolling upgrades

Key	Type	Description	Default
`enabled`	bool	Enable job preemption during rolling upgrades. Must be a real boolean when present — a null value is rejected by the syncer controller’s validation.	`false`
`method`	string	Preemption strategy; required when `enabled` is true. One of `partition` (preempt jobs in partitions matching `partitions`), `qos` (preempt jobs whose QoS matches `qos`), or `time` (preempt jobs whose rolling-delete condition is older than `timeLimit`).	—
`partitions`	string	Comma-separated regexes matching partition names. Required (and only valid) when `method` is `partition`. Must be a string — the operator splits it on commas; a YAML list is rejected by the syncer controller’s validation.	—
`qos`	string	Comma-separated regexes matching QoS names. Required (and only valid) when `method` is `qos`. Must be a string — the operator splits it on commas; a YAML list is rejected by the syncer controller’s validation.	—
`timeLimit`	string	Duration (e.g. `30m`). Required (and only valid) when `method` is `time`.	—

syncer:
  nodesetUpdateJobPreemption:
    enabled: true
    method: partition
    partitions: "^b200$,^priority-.*$"

object

See the operator defaults below.

syncer.image
The image to use for the syncer.

object

repository:
tag:

syncer.resources
Resources for the syncer container. object requests: cpu 200m memory 1Gi; limits: memory 1Gi

​Parameters

Parameters