Provision users in SUNK

SUNK Standard SUNK Self-Service

​

Overview

This page explains how to provision users in CoreWeave Identity and Access Management (IAM) for SUNK so they receive POSIX identities, SSH keys, and Slurm accounts in your cluster. It also explains how SUP keeps Slurm users in step with IAM afterward, including when it deletes them. After you complete the steps here, you can continue to nsscache configuration so those identities are available on the cluster. SUNK access management refers to the provisioning of users in your SUNK cluster. Provisioned users can connect over SSH to Login Nodes and Individual Login Pods, manage Slurm settings, and use Slurm commands to run experiments. At CoreWeave, this encompasses the flow of user identities from its source of truth to the creation of POSIX and Slurm identities in your SUNK cluster. This end-to-end flow is implemented by Automated User Provisioning (AUP) and SUNK User Provisioning (SUP). You can use SUP alongside or independently of AUP. AUP connects your organization’s identity provider (IdP) to CoreWeave IAM through SCIM, the modern standard for secure identity synchronization. AUP continuously syncs users and groups from an upstream IdP, such as Okta or Microsoft Entra. Any upstream changes, such as new users, role updates, or deletions, propagate automatically to CoreWeave IAM. SUP provisions CoreWeave IAM data into SUNK, whether that data was federated from an IdP through AUP or manually created in the CoreWeave Cloud Console. The exception is users marked exempt from SCIM, which SUP doesn’t provision. SUP automatically creates POSIX users and groups, SSH keys, and Slurm users and accounts in your SUNK clusters as soon as a user is added directly to CoreWeave IAM or to an upstream IdP. The following diagram shows how all three provisioning paths (IdP federation via AUP, manual user and group management, and user SSH keys) flow through CoreWeave IAM and into SUNK clusters via SUP:

​

Provision users in CoreWeave IAM

The following sections describe how to get users into CoreWeave IAM. You can federate from an IdP with AUP, or create users manually in the Cloud Console. Follow the instructions for either method before configuring SUP so that SUP has IAM data to sync into the cluster.

​

Prerequisites

Before you provision users in CoreWeave IAM, enable the following settings on the SCIM Configuration page:

• Enable SCIM API
• Enable SUNK User Provisioning

​

Provision users from an IdP into CoreWeave IAM

This section assumes you’re federating users from an IdP with AUP. Complete configure AUP with Okta or configure AUP with Microsoft Entra first so users and groups exist in CoreWeave IAM before you continue with the steps in this section.

1. In the Cloud Console, navigate to the SCIM Configuration page.
2. If Automated User Provisioning isn’t already enabled, click the toggle buttons to enable it. All three toggles must be enabled if you’re federating from an IdP.
3. In your IdP, create a custom attribute for SSH public keys and map it to CoreWeave’s SCIM endpoint. CoreWeave IAM needs this mapping so SUP can install the correct keys on the cluster. The attribute mapping must meet these requirements:

   Requirement	Value
   Attribute name	sunkSshKeys (must be exact)
   SCIM namespace	urn:coreweave:params:scim:schemas:extension:coreweave:2.0:CoreWeaveUser
   Data type	String (the SCIM endpoint accepts a single key or a list of keys)

   For IdP-specific setup instructions, see:
   • Map SSH keys for SUNK with Microsoft Entra
   • Map SSH keys for SUNK with Okta
4. In your IdP, navigate to the target user’s profile and add their SSH public key to the custom attribute you created.
5. In the Provisioning section of your IdP, verify that the SSH key attribute is included in the attribute mappings for the CoreWeave application.
6. In the Cloud Console, navigate to the Users page to view a list of all users in your organization. Click the three-dot menu icon to the right of the target user’s name, and then click View Details.
7. Verify that the SSH Keys field under Slurm Attributes contains the SUNK public SSH key that you added in your IdP. When you use AUP, this field automatically synchronizes with CoreWeave IAM whenever changes are made in your IdP. You can manually edit this field in the CoreWeave Cloud Console to temporarily change the key, but the next synchronization overwrites the value. Don’t rely on manual updates for persistent SSH key configuration when using AUP.
8. Continue to nsscache configuration. At this point, federated users should appear in CoreWeave IAM with SSH keys that SUP can propagate to SUNK. For more information about AUP and integrating it with other CoreWeave services, see the Automated User Provisioning guide.

​

Provision users directly in CoreWeave IAM

1. In the Cloud Console, navigate to the SCIM Configuration page.
2. If they aren’t already enabled, click the toggle buttons to enable the following settings:
   • Enable SCIM API
   • Enable SUNK User Provisioning
   On the SCIM Configuration page, you don’t need to toggle Enable Automated User Provisioning if you aren’t federating from an IdP.
3. Each user who needs cluster access must add at least one public SSH key to the SSH Keys field of the Slurm Attributes section in their Settings page. They can find this page through the profile icon at the top right of the CoreWeave Cloud Console. To add multiple keys, enter one per line (newline-separated), not comma-separated. If the Slurm Attributes section doesn’t appear in their Settings page, verify that SUNK User Provisioning is enabled on the SCIM Configuration page.
4. Continue to nsscache configuration. Users who added SSH keys in their Settings page are ready for SUP to create matching POSIX and Slurm identities on the cluster.

​

Slurm user attributes

SUP automatically creates a POSIX User ID (UID), POSIX Group ID (GID), and POSIX username when a user is provisioned in CoreWeave IAM either through AUP or manual creation. These IDs belong to the CoreWeave user, not the email address. Don’t change a user’s email address only in your identity provider. If your IdP identifies users by email, that creates a second user with new IDs and breaks SSH access and file ownership. To change an email address without getting new IDs, see Change a user’s email address. Users can view these details in the Slurm Attributes section of their Settings page. They can open this page through the profile icon at the top right of the Cloud Console. Admins can view this information on the Users page of the Cloud Console. Click the vertical dot menu icon to the right of a user’s name, and then click View Details.

​

Slurm user and account lifecycle

SUP keeps the Slurm accounting database consistent with the POSIX users that nsscache resolves on the cluster. A sync process in the slurmctld Pod reconciles the two about every 60 seconds. Each sync run does the following:

1. Creates the default SUP Slurm account if it doesn’t already exist on the cluster. The account name comes from nsscache.slurmUserProvisioning.defaultSlurmAccount, which is cw-sup unless you override it.
2. Creates a Slurm user for every POSIX user who belongs to a provisioned group. The default account for each new Slurm user is the account from the preceding step. Provisioned groups are the groups you list in scim_users_parameters in the nsscache configuration. If you list no groups there, SUP uses slurm-users.
3. Deletes Slurm users that no longer map to a provisioned POSIX user, as described in When SUP deletes a Slurm user.

SUP never deletes Slurm accounts. The default account and any accounts you create yourself remain in place even after SUP deletes the last user associated with them. To stop SUP from managing Slurm users, set nsscache.slurmUserProvisioning.enabled to false. Slurm users that SUP already created stay in the accounting database, because the sync run is the only process that deletes them. POSIX user and group provisioning continues, and you manage Slurm users yourself with sacctmgr.

​

When SUP deletes a Slurm user

SUP deletes a Slurm user on the next sync run when either of the following is true:

• The POSIX user no longer resolves on the cluster. This happens after you remove or deactivate the user in CoreWeave IAM or in your upstream IdP, and nsscache stops publishing them.
• The POSIX user still resolves, but belongs to none of the provisioned groups. If you drop a group from scim_users_parameters, every user whose cluster access came from that group meets this condition.

If the user has non-terminal job records in the Slurm accounting database, such as a job stuck after a node failure, the delete attempt fails and SUP retries on the next sync run. The user remains in the accounting database until you clear the stuck job records. SUP removes the user from the Slurm accounting database, not only from the default Slurm account. It also removes every association that user held on the cluster, including any admin role and any additional account associations you added manually with sacctmgr. To restore access, provision the user again and reapply those changes.

​

Create SUNK user groups

Group names in CoreWeave IAM must match the names you reference in the nsscache configuration. If you’re federating users from an IdP, you can create your user groups there. AUP automatically syncs groups into CoreWeave IAM. If you aren’t federating users from an IdP, create your user groups in the CoreWeave Cloud Console:

1. In the CoreWeave Cloud Console, navigate to the Groups tab in the left sidebar.
2. Click the Create Group button at the top right of the page.
3. Enter the group name and click Create. These group names must match the names of the groups specified in the nsscache configuration. Each new group automatically receives a GID.