> ## Documentation Index > Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt > Use this file to discover all available pages before exploring further. # September 23, 2026 - SUNK v8.1 release > SUNK v8.1 isolates jobs in per-job PID namespaces and moves to Slurm 25.11.8 ## Overview SUNK v8.1.0 was released on September 23, 2026, and the v8.1.1 patch release followed on September 24, 2026. Both ship with Slurm `25.11.8-coreweave.1` by default, up from `25.11.6-coreweave.2` in SUNK 8.0. See [SUNK to Slurm version mapping](/products/sunk/reference/sunk-slurm-versions.md). The main change in 8.1 is per-job isolation: each job now gets its own PID namespace, `/tmp`, and `/dev/shm` by default. The release also tightens validation of the `SlurmCluster` resource, runs `slurmctld` as a non-root user, and adds two SUNK Pod Scheduler annotations. New SUNK Self-Service clusters default to SUNK 8.1.1. Patch releases for 8.1 are listed on this page under [Patch releases](#patch-releases). ## Upgrade to 8.1 **SUNK Self-Service** Set `sunkVersion` on the `SunkCluster` resource. To also take the Slurm build that ships with 8.1, set `slurmVersion` in the same change. The following command upgrades a cluster in the `tenant-slurm` namespace, the namespace the Cloud Console uses, to SUNK 8.1.1 and its default Slurm build: ```bash theme={"system"} kubectl patch sunkcluster [CLUSTER-NAME] -n tenant-slurm --type=merge \ -p '{"spec":{"sunkVersion":"v8.1.1","slurmVersion":"v25.11.8-coreweave.1"}}' ``` `slurmVersion` needs the leading `v`. `sunkVersion` works with or without it. For how the rollout treats running jobs, see [Upgrade a SUNK Self-Service cluster](/products/sunk/deploy_sunk/upgrade-sunk-cluster.md). SUNK Self-Service copies `spec.slurmConfig` from the `SunkCluster` onto the `SlurmCluster` resource, so before you upgrade, check your `spec.slurmConfig` for the `StateSaveLocation` and `PrologFlags=NoHold` settings in [Changes to check before you upgrade](#changes-to-check-before-you-upgrade). **SUNK Standard** Upgrade the `sunk` and `slurm` charts to 8.1.1. Before you upgrade, check your `slurm` chart values for the settings in [Changes to check before you upgrade](#changes-to-check-before-you-upgrade). ## Changes to check before you upgrade | Change | What to check | | - | - | | Per-job namespaces are on by default | The operator enables Slurm's `namespace/linux` plugin, which gives each job its own PID namespace and a private `/proc`, `/tmp`, and `/dev/shm`. Jobs that share a node can no longer see or signal each other's processes. Slurm clears each job's `/tmp` and `/dev/shm` when the job ends, so a workflow that stages data in `/tmp` for a later job, or shares `/dev/shm` between jobs on the same node, stops working. To opt out on SUNK Standard, set `slurmConfig.namespaceConfig.enabled: false`. Changes take effect only after `slurmd` restarts. SUNK Self-Service doesn't expose `namespaceConfig` on the `SunkCluster` resource, so you can't turn the plugin off there. See [Memory accounting](/products/sunk/optimize_workloads/memory-accounting.md). | | `PrologFlags=NoHold` disables the namespace plugin | Slurm rejects `NoHold` alongside `Contain`, which the plugin requires. If `PrologFlags` includes `NoHold`, the operator doesn't enable the plugin and emits a `NamespacePluginSkipped` Warning event. On SUNK Self-Service, the event is on the `SlurmCluster` resource, not the `SunkCluster`. | | `StateSaveLocation` can't be set in `slurmConfig` | The `SlurmCluster` resource rejects a `StateSaveLocation` key in `spec.slurmConfig`. SUNK manages the `slurmctld` state path and pins it to `/var/spool/slurmctld/save`. Remove the key before you upgrade. On SUNK Self-Service, the `SunkCluster` status doesn't report the rejection, so later changes to `spec.slurmConfig` stop applying without a visible error. | | Login hostnames in Service annotations are sanitized | In a login Service annotation whose key contains `hostname`, such as `external-dns.alpha.kubernetes.io/hostname` or `tailscale.com/hostname`, `%s` is now the user or group name in lowercase, with each character other than a letter, digit, or hyphen replaced by a hyphen. For example, a user named `first.last` gets `first-last` in their login hostname. Users who connect by the old hostname need to use the new one. | | `nsscache.overrideHomeDirectory` applies to LDAP users | Before 8.1, `nsscache.overrideHomeDirectory` applied only to users provisioned through SCIM. It now applies to LDAP users too, so on an LDAP-backed cluster, users' home directories follow its pattern, which defaults to `/mnt/home/%%u`. Check that the pattern matches where your users' home directories are. | | `accounting.enabled` is removed from the `slurm` chart | Accounting is required. In 8.0, setting `accounting.enabled: false` omitted the accounting configuration, so the resource fell back to the `cwdb` database backend instead of the chart's `moco` default. Remove the key from your values. | | `slurmctld` runs as a non-root user | The `slurmctld` container now runs as UID and GID `401`, the same as its sidecars. Check any customization that assumed `slurmctld` runs as root. | ## New in 8.1 * **SUNK Pod Scheduler annotations.** `sunk.coreweave.com/wckey` sets the Slurm workload characterization key, and `sunk.coreweave.com/segment` sets the topology segment size. See the [annotations reference](/products/sunk/run_workloads/schedule-kubernetes-pods.md#annotations-reference). * **Block topology labels.** The operator now resolves each node's block-topology label per node. It uses the `ds.coreweave.com/nvlink.domain` label, and falls back to the `nvidia.com/gpu.clique` label when the domain label isn't set. * **Database cleanup on deletion.** Deleting a `SlurmCluster` that uses the `moco` or `cwdb` backend now waits for its managed MySQL cluster to finish cleaning up, through a `sunk.coreweave.com/database` finalizer. Before 8.1, compute nodes could be torn down first, which could leave the MySQL cluster stuck in `Terminating`. ## Fixes * Changes to `nsscache.sudoGroups` now reach running login and compute Pods without a Pod restart. Before 8.1, removing a group didn't revoke its passwordless `sudo` grant on running Pods. * The node health check epilog skips its exclusive tests when the node is already allocated to another job, so a stale epilog no longer drains a node in use. * A custom `topology.conf` is preserved when `compute.generateTopology` is `false`. * Disabling `slurmrestd`, for example by removing `rest.enabled: true` from your `slurm` chart values, now deletes it. Before 8.1, it kept running. The `SlurmCluster` status now reports the status of `slurmrestd`. * `SlurmCluster` conditions report errors from child resources, and the `DatabaseAvailable` condition includes failure details from the `CWDBCluster`. * A login Pod's default memory limit no longer caps a larger request. Before 8.1, a request above the default limit failed validation and blocked the login Pod from being created. * SUNK scripts match `slurm.conf` keys case-insensitively. * Compute Pods now carry the labels that the `[CLUSTER-NAME]-compute` headless Service selects on. Before 8.1, `NodeSets` that SUNK Self-Service created, and hand-written `NodeSet` resources, lacked them, so the Service had no endpoints and per-Pod DNS names didn't resolve. This broke distributed setups that rely on node hostnames. On upgrade, the affected `NodeSets` roll once. ## Patch releases ### 8.1.1 Released September 24, 2026. Ships with Slurm `25.11.8-coreweave.1` by default, the same as 8.1.0. * [GPU Straggler Detection](/products/sunk/discover_sunk/straggler-detection.md) installs after all other one-shot s6 services finish, so it no longer contends with packages you install for the `apt` lock. * `nsscache` runs full updates instead of incremental ones, so updates no longer depend on the `modifytimestamp` filter.