Can I run workloads directly on the login node?
No. The login node is a shared resource for submitting jobs, managing files, and running lightweight commands. Compute-h …
How do I monitor per-user resource usage on login nodes?
The Slurm Cluster dashboard in CoreWeave Managed Grafana includes a Slurm-Login Info section with CPU and memory usage p …
How do I set up Slurm accounting for priority scheduling?
SUNK enables Slurm accounting by default: accounting.enabled is true and SUNK deploys a local slurmdbd with MySQL automa …
What resources should I set for slurmd on login Pods?
SUNK login Pods do not run slurmd (that daemon runs in the slurmd Pods on compute nodes). The resources you size on a lo …
Why do slurmd Pod metrics show almost no CPU or memory usage?
This is expected on SUNK, not a broken metrics pipeline or a misconfigured Pod. Kubernetes Pod metrics come from cAdviso …
Why does my GB200 job segfault during checkpointing?
A segfault during checkpoint saving followed by an NCCL collective timeout points to a driver-level problem, not a fabri …
Why does my node keep draining for a missing InfiniBand device?
A node that drains with NHC: check_file_test: -e /dev/infiniband returned 1 has lost its InfiniBand device mount inside …
Why is my node drained or cordoned?
A Slurm node enters drained state when it stops accepting new jobs. The drain reason (visible in scontrol show node or t …