> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
> Use this file to discover all available pages before exploring further.
# July 23, 2026 - SUNK v8.0.0 release
> SUNK v8.0.0 moves cluster configuration to the SUNK operator through a single SlurmCluster resource
## Overview
SUNK v8.0.0 is a re-architecture, not a routine version bump. This page describes the configuration and behavior changes in 8.0 so that you can update your Helm values before you upgrade from a 7.x release.
In 7.x, the Helm chart was the cluster. The chart shipped the defaults and rendered `slurmctld`, `slurmdbd`, the login pods, and everything else directly. In 8.0, the `slurm` chart renders a single `SlurmCluster` resource, and the SUNK operator builds the cluster from it.
You still configure the cluster through Helm values, and most of your keys survive with the same names and the same override semantics. What changed is where a value comes from when you don't set it. The operator supplies it now, instead of the chart's `values.yaml`.
This release also removes SSSD directory integration, consolidates four database mechanisms into one, replaces shared login replicas with per-user and per-group login pods, and deletes the `slurm-login` chart.
#### Action required
Upgrading from any 7.x release isn't a standard `helm upgrade`. The chart no longer templates the stateful objects that hold your cluster state and accounting data, so Helm treats them as removed and deletes them, taking the accounting database and cluster state with them.
Do not run `helm upgrade --reuse-values`. It carries your 7.x values forward without the new chart defaults, which can silently select the wrong database backend.
Your 7.x cluster must already be on the MOCO database backend before you upgrade. SUNK 8.0 doesn't include the Bitnami MySQL to MOCO migration, so run that migration while you're still on 7.x.
This page covers the configuration changes in 8.0. It isn't a complete upgrade guide, and your cluster may have upgrade considerations beyond the ones listed here. Contact [CoreWeave Support](/support) to plan your upgrade before you begin.
The following sections describe the Slurm version upgrade, the defaults that change if you do nothing, the identity, accounting, and login re-architectures, and the full list of new, renamed, and removed values.
## Upgrade to Slurm 25.11
SUNK v8.0.0 upgrades Slurm from 25.05 to 25.11, a major version bump that brings upstream bug fixes and performance improvements.
The Slurm protocol version default moves from 25.05 to 25.11 alongside it. If you need to stage the protocol change separately from the version upgrade, pin the protocol version explicitly with `slurmCluster.spec.protocolVersion`, which accepts `25_05` or `25_11`.
## Defaults that change if you do nothing
SUNK 8.0 drops a set of chart defaults and replaces them with operator defaults that differ. None of these produce an error. You get different behavior, and which ones reach you depends on your setup.
| If this describes you | Set this |
| ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Every cluster | `accounting.database.backend`. Both the chart template and the CRD fall back to `cwdb` when the value is empty, which may not be the backend you want and may not have its operator installed. |
| You pull images from a private registry | `imagePullSecrets` in both places. `slurmCluster.spec.imagePullSecrets` covers the operator-managed control plane, and the NodeSet template still reads the top-level `imagePullSecrets`, which no longer appears in the default values. If you miss the top-level one, compute pods hit `ImagePullBackOff`. |
| You run `slurmrestd` | `rest.enabled: true`. The chart emits the block only when the value is explicitly `true`, and nothing errors if you leave it out. |
| You care which storage class holds `slurmctld` state | `controller.stateVolume.storageClassName`. SUNK 8.0 sets no default, so Kubernetes uses the cluster default storage class. SUNK 7.x shipped `shared-vast`. |
| Your control plane nodes aren't labeled `node.coreweave.cloud/class=cpu` | `slurmCluster.spec.affinity`. The 8.0 default requires that label, and the field takes a full affinity object rather than a node selector. |
| You rely on namespace-scoped scheduling | `scheduler.scope.type: namespace`. The CRD defaults to `cluster`, and the chart doesn't set it. |
| Any compute image runs cgroup v1 | `slurmConfig.cgroupConfig.CgroupPlugin`. SUNK 8.0 defaults to `cgroup/v2` instead of `autodetect`, so nothing detects it for you. |
| You don't want SSH or Pyxis on compute nodes | `compute.ssh.enabled` and `compute.pyxis.enabled`. Both default to `true` in 8.0, and both were `false` in 7.x. |
| You rely on cleanup deleting invalid nodes, or on its verbose logging | `cleanupCompleting.deleteInvalidNodes` and `cleanupCompleting.verbose`. Both shipped as `true` in 7.x, and both default to `false` in 8.0. |
The cgroup v1 plugin is deprecated and won't be supported in future Slurm versions, so SUNK now defaults to `cgroup/v2`. The `cgroup/v1` plugin is still supported, subject to upstream Slurm support and support in the broader ecosystem, including Linux distributions, the kernel, and systemd.
## Identity moves from SSSD to nsscache
SUNK 7.x supported SSSD, configured across `directoryService`, an `sssdContainer` sidecar, and a `userLookupContainer`. All of it is removed, along with `directoryCache` from the deleted `slurm-login` chart.
Identity comes from `nsscache` in 8.0. The operator runs it, refreshes it on an interval, and publishes the results into per-map Secrets that login and compute pods consume. Two sources are available:
* SCIM, the default. Set `nsscache.existingSecret` to a Secret holding the `nsscache-scim-auth-token` key, and `nsscache.groups` to select which groups SCIM fetches.
* LDAP. Set `source: ldap` under `nsscache.nsscacheConfig` along with your directory settings.
`nsscache.defaultShell` and `nsscache.overrideHomeDirectory` set the login shell and home path for SCIM-sourced accounts. The home path defaults to `/mnt/home/%u`, so check that against where your home directories live.
Two smaller changes come with it:
* The operator generates `nsswitch.conf`, and `nsscache.nsswitchConfig` is removed.
* `nsscache.cronJobSchedule` becomes `nsscache.fetchInterval`, an integer number of seconds defaulting to 60, with an operator-managed Deployment in place of the CronJob.
`nsscache.enabled`, `nsscache.existingSecret`, `nsscache.sudoGroups`, and `nsscache.slurmUserProvisioning` all carry over unchanged. `nsscache.nsscacheConfig` now requires every inner value to be a string, so quote numbers and booleans, and convert 7.x lists into the comma-separated strings that `nsscache.conf` expects.
## Accounting consolidates on one database backend
SUNK 7.x gave you four overlapping ways to attach a database: the Bitnami MySQL subchart, MOCO, `accounting.externalDB`, and `accounting.external`. SUNK 8.0 replaces all four with one selector, `accounting.database.backend`, and one set of connection fields.
| Backend | What the operator does |
| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `moco` | Creates a MOCO `MySQLCluster` named after your cluster. This is the same kind of resource the 7.x chart created itself, and it's the chart's shipped default. |
| `cwdb` | Creates a `CWDBCluster` for the `cwdb-operator` to fulfill. The `cwdb-operator` must be installed on the cluster. |
| `external` | Creates nothing. Your database must be reachable at the connection fields, and `storageHost`, `storageUser`, and `existingSecret` are all required. Admission enforces that requirement. |
Write `backend` out explicitly even when you want the default, because both the chart template and the CRD fall back to `cwdb` when the value is empty.
Everything else about `slurmdbd` keeps its meaning. `accounting.config` still configures `slurmdbd.conf`, and replicas, image, resources, security context, and probes all remain overridable. Only their defaults moved into the operator. Your MOCO connection and authentication settings move to `accounting.database`, and MOCO infrastructure tuning moves to `database.mysqlCluster`.
Accounting also changes in the following ways:
* The Bitnami to MOCO migration Job (`moco.migration`) exists only in 7.x. Run it before you upgrade. SUNK 8.0 ships only a manual `moco-migrator.sh` script in the custom scripts ConfigMap.
* External `slurmdbd` (`accounting.external`) has no 8.0 equivalent. Only an external database is supported.
## Login capacity is provisioned per user and per group
SUNK 7.x gave you two kinds of login pods, under two names, built by two different charts. Shared login pods came from the `slurm` chart as one StatefulSet of `login.replicas` interchangeable pods behind `-login`. Individual login pods came from the `slurm-login` subchart, off by default, one StatefulSet per user, sized by `login.individualResources`.
SUNK 8.0 keeps both ideas, renames them, and moves both into the operator. Shared becomes group pods, and individual becomes user pods. They're now configured identically: `login.common` sets the base, `login.userPods` and `login.groupPods` override per kind, and each entry in `login.groups` selects which kinds that group gets, with membership resolved from `nsscache`.
Resource names change. Per-user StatefulSets and Services are `-login--`, and per-group ones are `-login-`. The headless `-login-all-users` Service keeps its 7.x name and shape. The old `-login` StatefulSet, its Service, and the per-replica `-login-` Services have no 8.0 counterpart.
Check the login defaults before you list groups. A group entry that doesn't say otherwise gets both per-user and per-group pods, because `userPods` and `groupPods` both default to `true`. Every login entity gets its own Service defaulting to `LoadBalancer`, and each pod requests 4 CPU and 8Gi.
That last default applies to per-user pods too. In 7.x those ran on the much smaller `individualResources` of 500m and 300Mi, so the per-user CPU request grows roughly eight times and the memory request roughly 27 times unless you override `login.userPods.pod.resources`. A group of N users on defaults means N+1 StatefulSets and N+1 external load balancers.
Service annotations and labels support `%s` and `%h` substitution for the entity name, and `login.common.service.individual` overrides them per user or group.
`login.s6` keeps its name but gains validation. Script-based oneshots need `timeoutUp`, longruns need a `timeout`, and each service must set exactly one of `script` or `packages`. The same rules apply to `compute.s6`.
## Slurm configuration defaults
Slurm configuration changes little in 8.0. The following table lists the keys whose defaults differ:
| File | Key | 7.x default | 8.0 default |
| ------------- | ---------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `slurm.conf` | `SlurmctldParameters` | `conmgr_max_connections=512`, `conmgr_threads=32`, `idle_on_node_suspend`, `node_reg_mem_percent=95` | Adds `rl_enable`, `rl_log_freq=30`, `rl_bucket_size=50`, `rl_refill_period=1`, `rl_refill_rate=10`, which turns on RPC rate limiting |
| `cgroup.conf` | `CgroupPlugin` | `autodetect` | `cgroup/v2` |
| `slurm.conf` | `SUNKNodeDashboardURL` | unset | CoreWeave Grafana node details dashboard |
| `slurm.conf` | `SUNKJobDashboardURL` | unset | CoreWeave Grafana job metrics dashboard |
### RPC rate limiting is on by default
`slurmctld` now throttles RPCs, so review any tooling that issues them at volume. For more information about tuning the limits, see the [Slurm configuration documentation](https://slurm.schedmd.com/slurm.conf.html).
## Other behavior changes
The following changes affect areas not covered in the preceding sections.
* **Hooks API configuration moved to the syncer.** Both `syncer.hooksAPI` and `scheduler.hooksAPI.waitForPodDeletionInterval` become `syncer.config.hooksAPI`. If your 7.x syncer and scheduler settings disagreed, they now converge on a single value, so pick the one you want.
* **Low-profile compute requests default on.** The new `compute.lowProfileRequests` overlays low CPU and memory requests on `slurmd` containers so Kubernetes can co-schedule alongside them, and it's forced on when the scheduler is enabled. This changes how compute nodes pack.
* **Cleanup runs every 300 seconds instead of every 60.** `cleanupCompleting.timeoutSeconds` is now a flat 2400 rather than twice `KillWait`, so the timeout no longer tracks your `KillWait`.
* **The `slurmrestd` workload and Service are renamed** from `-rest` to `-restd`. Repoint every client, dashboard, and script that talks to the old name.
* **Scheduler scrape configuration can no longer be turned off.** The operator creates it whenever the `VMPodScrape` CRD is installed, and the `sunk` chart's `scheduler` section is gone. Operator-created `VMPodScrape` objects replace the chart-rendered monitoring templates.
* **The SUNK admission webhooks and their cert-manager serving certificate are gone.** CRD validation rules and declarative defaults replace them, at parity with what the 7.x webhook covered for NodeSets.
* **The operator provisions the JWT and cluster secrets.** `secretJob` is removed. `jwt.existingSecret` still works unchanged.
Workload sizing barely moved. `slurmctld`, `slurmdbd`, the syncer, `slurmrestd`, `munge`, group login pods, cleanup-completing, and the database all keep their 7.x requests and limits, including the 32Gi `slurmctld` state volume. `nsscache` rises to 500m and 200Mi requested with a 1Gi limit, up from 200m and 100Mi with a 500Mi limit. Per-user login pods are the exception, covered in [Login capacity is provisioned per user and per group](#login-capacity-is-provisioned-per-user-and-per-group).
## New in 8.0
These values have no 7.x equivalent.
| Key | What it does |
| ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `compute.cudaVersion` | Selects the CUDA variant, defaulting to 13.0, and injects a `cu` feature onto GPU NodeSets. Replaces the per-CUDA-version compute image definitions you had to choose between in 7.x. |
| `compute.managedPrologConfig`, `compute.managedEpilogConfig` | Operator-managed prolog and epilog scripts in `/etc/slurm/prolog.d` and `/etc/slurm/epilog.d`. They hot-reload, and they can override the CoreWeave defaults. |
| `compute.lowProfileRequests` | Overlays low CPU and memory requests on `slurmd` containers so Kubernetes can co-schedule alongside them. Defaults to `true`. |
| `slurmCluster.spec.slurmImage`, `slurmCluster.spec.sunkImage` | One Slurm image and one operator image, applied across all components. |
| `slurmCluster.spec.protocolVersion` | Pins the Slurm protocol version. |
| `slurmCluster.spec.tolerations`, `slurmCluster.spec.priorityClassName` | Cluster-wide, replacing the per-component keys. |
| `nsscache.groups` | Selects which groups SCIM fetches. |
| `nsscache.defaultShell`, `nsscache.overrideHomeDirectory` | Login shell and home directory for SCIM-sourced users. Home defaults to `/mnt/home/%u`. |
| `nsscache.image`, `cleanupCompleting.image` | Per-workload image overrides. |
| `rest.service.externalName` | New Service field on `slurmrestd`. |
| `partitions..default`, `partitions..feature` | Per-partition default flag and node feature selector, part of the new partition object shape. |
## Renamed and moved values
Set the new key and delete the old one.
| 7.x | 8.0 |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `global.labels`, `global.annotations` | `slurmCluster.spec.labels`, `slurmCluster.spec.annotations`. Control plane resources only. Use `compute.labels` and `compute.annotations` for NodeSets. |
| `global.nodeSelector.affinity` | `slurmCluster.spec.affinity`, now a full affinity object |
| `global.dnsConfig.additionalSearches` | `slurmCluster.spec.dnsConfig.additionalSearches` |
| `global.volumes`, `global.volumeMounts` | Per-component `volumes` and `volumeMounts` on `controller`, `rest`, `login.common.pod`, and `compute` |
| `nsscache.cronJobSchedule` | `nsscache.fetchInterval`, seconds rather than a cron expression, default 60 |
| `cleanupCompleting.cronJobSchedule` | `cleanupCompleting.intervalSeconds`, seconds rather than a cron expression, default 300 |
| `syncer.hooksAPI` | `syncer.config.hooksAPI` |
| `syncer.logLevel`, `syncer.maxConcurrentReconciles` | `syncer.config.syncer.logLevel`, `syncer.config.syncer.maxConcurrentReconciles` |
| `scheduler.hooksAPI.waitForPodDeletionInterval` | `syncer.config.hooksAPI.waitForPodDeletionInterval` |
| `scheduler.name`, `scheduler.logLevel`, `scheduler.maxConcurrentReconciles` | `scheduler.config.scheduler.name`, `scheduler.config.scheduler.logLevel`, `scheduler.config.scheduler.maxConcurrentReconciles` |
| `moco.enabled: true` | `accounting.database.backend: moco` |
| `moco.mysqlCluster.auth.storageHost`, `storageUser`, `storageLoc`, `existingSecret` | `accounting.database`, same field names |
| `moco.mysqlCluster.replicas`, `image`, `persistence`, `resources`, `inodeLockFixer`, `config` | `database.mysqlCluster`, same field names |
| `accounting.externalDB.enabled: true` | `accounting.database.backend: external` |
| `accounting.externalDB.storageHost`, `storageUser`, `storageLoc`, `existingSecret` | `accounting.database`, same field names |
| `accounting.useExistingSecret: true` | `accounting.database.existingSecret: `. Name the Secret. The boolean toggle is gone. |
| `login.image.repository`, `login.image.tag` | `login.common.pod.image` |
| `login.resources`, `login.individualResources` | `login.common.pod.resources`, overridable per kind at `login.userPods.pod.resources` and `login.groupPods.pod.resources` |
| `login.nodeSelector.affinity` | `login.common.pod.affinity` |
| `login.env`, `hostAliases`, `containers`, `volumes`, `volumeMounts`, `podSecurityContext`, `securityContext`, `serviceAccountName`, `automountServiceAccountToken` | `login.common.pod.` |
| `login.service.enabled`, `type`, `externalTrafficPolicy`, `loadBalancerClass`, `additionalPorts` | `login.common.service.` |
| `login.service.metadata.global`, `login.service.metadata.common` | `login.common.service.annotations` and `login.common.service.labels`. The global and common split is gone. |
| `login.service.metadata[]`, per service | `login.common.service.individual`, keyed per user or group |
| `compute.partitions.` | `partitions.`, now a top-level map |
| `compute.autoPartition.enabled`, `compute.autoPartition.config` | `partitions.auto.enabled`, `partitions.auto.config`. `auto` is a reserved, validated entry. |
### Values that changed shape
Same key, different type. Check these manually, because a wrong type here doesn't always produce a visible error.
| Key | 7.x | 8.0 |
| --------------------------------------------------------- | -------------------------- | ---------------------------------------------------------------- |
| `controller.etcConfigMap` | Bare string or list | String array only, so wrap scalars in a list |
| `compute.prologConfigMap`, `compute.epilogConfigMap` | Bare string | String arrays |
| `compute.s6` | Map | List, and `dependencies:` becomes `depends:` |
| `nsscache.nsscacheConfig` | Mixed types, lists allowed | Every inner value must be a string |
| `partitions.`, formerly `compute.partitions.` | A config string | An object exposing `enabled`, `config`, `default`, and `feature` |
## Removed values
Delete these. Where something replaces the behavior, it's named.
| Removed | Replacement |
| --------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `global.cks` | None |
| `global.imagePullPolicy` | The operator controls pull policy. |
| `controlPlane.enabled`, `controller.enabled`, `syncer.enabled` | None. The chart always renders the control plane and the syncer. |
| `syncer.controllerAddress`, `watchNodeSets`, `watchAllNodeSets`, `nodePermissions.enabled`, `scheduler.controllerAddress` | None. The operator wires the syncer itself. |
| `secretJob` | The operator provisions the JWT and cluster secrets, including a JWT Job it runs when the syncer Secret is missing. `jwt.existingSecret` still works unchanged. |
| `image.name` on every component | Set `repository` and `tag`. |
| `annotations`, `labels`, and `priorityClassName` on individual components | `slurmCluster.spec.labels`, `slurmCluster.spec.annotations`, and `slurmCluster.spec.priorityClassName` for the control plane. `compute.labels` and `compute.annotations` for NodeSets. Per-component granularity is gone. |
| Probes on `controller` (including `watch` probes), `syncer`, `munge`, and `login.sshdStartupProbe`, `sshdReadinessProbe`, `sshdLivenessProbe` | None. The operator manages them. Probes on `rest`, `accounting`, and `compute` are still overridable, so check before deleting any of those. |
| `rest.env`, `rest.securityContext`, `rest.terminationGracePeriodSeconds` | Use `rest.containers` for environment and container tweaks. |
| `nsscache.nsswitchConfig` | The operator generates `nsswitch.conf`. |
| `nsscache.resources`, `munge.securityContext` | Not configurable through Helm values in 8.0. The operator defaults apply. |
| `nodeSelector.affinity` and `tolerations` on `nsscache` and `cleanupCompleting` | Both workloads are operator-managed, and the cluster-wide `slurmCluster.spec` settings apply. |
| Scheduler pod metadata and probes | None. Remove them rather than assume they're still read. |
| `login.enabled` | Login is on when `login.groups` is non-empty. |
| `login.replicas`, `login.updateStrategy`, `login.terminationGracePeriodSeconds` | Capacity is provisioned per user and per group. |
| The `slurm-login` chart: `loginController`, `directoryCache`, `slurm-login.enabled` | Chart deleted. The operator reconciles login pods, and `nsscache` replaces the directory cache. |
| `directoryService`, `sssdContainer`, `userLookupContainer` | `nsscache`. See [Identity moves from SSSD to nsscache](#identity-moves-from-sssd-to-nsscache). |
| `mysql`, `moco.migration`, `moco.mysqlCluster.affinity`, `moco.priorityClassName` | See [Accounting consolidates on one database backend](#accounting-consolidates-on-one-database-backend). Database pod placement is backend-managed. |
| `accounting.enabled`, `accounting.external` | None |
| `accounting.volumes`, `accounting.volumeMounts` | None |
| `compute.dra.enabled` | None |
| `compute.partitionBaseConfig` | None. Set `partitions..config` on each partition. |
| Per-CUDA-version compute image definitions | `compute.cudaVersion` |
| `namespace-manager` in the `slurm` chart | The `sunk` chart manages namespaces. |
| The SUNK admission webhooks and their cert-manager serving certificate | CRD validation rules and declarative defaults, at parity with what the 7.x webhook covered for NodeSets |
| `scheduler.podMonitor.enabled` and `scheduler.vmPodScrape.enabled` in the `sunk` chart | The operator creates scrape configuration whenever the `VMPodScrape` CRD is installed. No toggle exists. |
| Chart-rendered monitoring templates | Operator-created `VMPodScrape` objects |