Manage users with nsscache

Enable automatic user provisioning for SUNK with nsscache

SUNK now uses nsscache, a lightweight directory service, to manage users across CoreWeave clusters. It supports two directory protocols: SCIM and LDAP. We recommend SCIM, as it enables automated user and group synchronization from your upstream identity provider (IdP) to SUNK. When a user is added or removed in your IdP, the change is reflected in your SUNK cluster within minutes, ensuring access stays accurate, secure, and up-to-date.

This implementation replaces the previous directory service based on SSSD, securing access and improving reliability. Whether your users are managed in a third-party IdP or directly in CoreWeave IAM, nsscache automatically provisions, updates, and removes POSIX users in your cluster without manual intervention.

This guide demonstrates how to configure a directory with nsscache with the following steps:

1. Create a Kubernetes Secret.
2. Add the Kubernetes Secret to the slurm chart.
3. Disable SSSD in the slurm chart and, if using individual login pods, the slurm-login chart.

Create a Kubernetes Secret for nsscache

Create a Kubernetes Secret that contains the configuration for your directory service.

Warning

CoreWeave strongly discourages including sensitive information, such as plaintext user credentials, directly in your values.yaml configuration. Instead, use a Kubernetes Secret for added security and manageability. CoreWeave provides encryption at rest for etcd data, which includes Secrets.

This secret can be a SCIM token or an LDAP service account password. Be sure to follow the appropriate naming conventions detailed below when creating your Secret.

Naming conventions for SCIM Secrets

For a SCIM Secret, the suggested Secretname is <release-name>-nsscache-scim-secret.

Inside the Secret, the key under data must be named nsscache-scim-auth-token. This contains the base64-encoded SCIM token.

Naming conventions for LDAP Secrets

For an LDAP Secret, the suggested Secret name is <release-name>-nsscache-ldap-secret.

Inside the Secret, the key under data must be named nsscache-ldap-password. This contains the base64-encoded password for the LDAP service.

Update the slurm chart with your Kubernetes Secret

After creating the Secret, you will need to update the slurm chart's values.yaml file.

Edit the nsscache.existingSecret parameter with the name of your Secret, as shown in the examples below.

Disable SSSD in the slurm and slurm-login charts

To enable nsscache, you must edit the slurm Helm chart. If using individual login pods, you must also edit the slurm-login Helm chart.

Update the slurm chart

In the slurm chart's values.yaml file, update the sssdContainer.enabled parameter to false and remove the directoryService section, as shown below:

Example

sssdContainer:
  enabled: false
# Remove the `directoryService` key
directoryService:

This disables SSSD, which is incompatible with nsscache.

You may remove the directoryCache.directoryService section in the values.yaml of the slurm chart, as nsscache does not use this configuration.

Disable SSSD

You must disable SSSD to use nsscache. For SUNK versions v6.x and below, SSSD is enabled by default.

Update the slurm-login chart

If using individual login pods, you will also need to edit the slurm-login chart's values.yaml file.

Set the directoryCache.source parameter to nsscache, as shown below:

Example

directoryCache:
  source: nsscache
# Remove the `directoryService` key
  directoryService:

You may remove the directoryCache.directoryService section in the values.yaml of the slurm-login chart, as nsscache does not use this configuration.

Example: SCIM values

Example SCIM values

sssdContainer:
  enabled: false
# Remove the `directoryService` key
directoryService:
nsscache:
  enabled: true
  existingSecret: scim-auth-token
  sudoGroups:
    - sunk-group-1
  nsscacheConfig:
    default:
      source: scim
      scim_base_url: https://api.coreweave.com/scim/<org>
      scim_users_parameters: filter=active eq "true"&groups=sunk-group-1
      scim_groups_parameters: excludeInactiveUsers=true&includeVirtualUserGroups=sunk-group-1

The suggested nsscache configuration for SCIM is set by default in the values.yaml file. For a full list of configuration options, see the Slurm parameter reference.

Example: Authentik LDAP values

Example Authentik LDAP values

sssdContainer:
  enabled: false
# Remove the `directoryService` key
directoryService:
nsscache:
  enabled: true
  existingSecret: nsscache-ldap-secret
  nsscacheConfig:
    default:
      source: ldap
      ldap_uri: ldap://authentik-outpost-ldap-outpost
      ldap_base: dc=coreweave,dc=cloud
      ldap_bind_dn: cn=ldapsvc,dc=coreweave,dc=cloud
      ldap_bind_password:
      ldap_rfc2307bis: 1
      ldap_default_shell: /bin/bash
    passwd:
      ldap_filter: (objectClass=user)
      ldap_override_home_dir: /mnt/home/%%u
    group:
      ldap_filter: (objectClass=group)
    shadow:
      ldap_filter: (objectClass=user)
    sshkey:
      ldap_filter: (objectClass=user)

Available parameters for SCIM

SCIM source options

The following options configure the behavior of the SCIM source:

Details

Parameter	Description
scim_base_url	The base URL of the SCIM server endpoint. This is the root URL that will be combined with the users and groups endpoints to form complete URLs.
scim_auth_token	The authentication token, or Bearer token, for SCIM API access. This can also be provided via the NSSCACHE_SCIM_AUTH_TOKEN environment variable.
scim_users_endpoint	The SCIM endpoint path for retrieving user data. Defaults to Users.
scim_groups_endpoint	The SCIM endpoint path for retrieving group data. Defaults to Groups.
scim_users_parameters	Optional URL parameters to be addedd to the users endpoint. Special characters (spaces, quotes, etc.) will be automatically URL-encoded. Example: groups=admin&filter=active eq true
scim_groups_parameters	Optional URL parameters to be added to the groups endpoint. Special characters (spaces, quotes, etc.) will be automatically URL-encoded. Example: filter=displayName eq users or displayName eq admin
scim_timeout	Timeout in seconds for SCIM requests. Defaults to 60.
scim_verify_ssl	Specifies whether to verify SSL certificates when making SCIM requests. Defaults to true. Set to false to disable SSL verification.
scim_retry_delay	Delay in seconds between retry attempts when SCIM requests fail. Defaults to 5.
scim_default_shell	Default shell to assign to users if not specified in SCIM data. Defaults to /bin/bash.
scim_override_home_directory	If specified in a [passwd] section, set every user's home directory to the given value. Optionally, use %%u to substitute the username. For example, /mnt/home/%%u would set user john to /mnt/home/john, while /shared/home would set all users to the same directory. Note that changing this value will change it for the entire cluster, not per user.

SCIM path configuration options

The following path configuration options allow customization of how data is extracted from SCIM responses. These can be set per-map in [passwd], [group], [shadow], and [sshkey] sections.

Details

Parameter	Description
scim_path_username	Path within SCIM user/group resources to extract the username. Defaults to userName.
scim_path_uid	Path within SCIM user resources to extract the user ID (UID). Defaults to id.
scim_path_gid	Path within SCIM user/group resources to extract the group ID (GID).
scim_path_home_directory	Path within SCIM user resources to extract the home directory. Defaults to /home/username format.
scim_path_login_shell	Path within SCIM user resources to extract the login shell. If not specified, uses the scim_default_shell value.
scim_path_ssh_keys	Path within SCIM user resources to extract the SSH public keys. Should point to an array of SSH key strings or a single SSH key string.

[shadow] map parameters for SCIM

The shadow map creates shadow(5) format entries for user data from the SCIM users endpoint. This requires only the scim_path_username configuration in the [shadow] section, as other shadow fields are typically not available from SCIM sources. All shadow entires are created in the format username:*:::::::, where * indicates that authentication is handled elsewhere (not through local password files).

The following optional configuration parameters are available for the [shadow] section to provide default values for shadow fields:

Details

Parameter	Description
scim_shadow_default_lstchg	Default value for the last password change field (days since January 1, 1970). Defaults to empty string.
scim_shadow_default_min	Default value for the minimum password age field, in days. Defaults to empty string.
scim_shadow_default_max	Default value for the maximum password age field, in days. Defaults to empty string.
scim_shadow_default_warn	Default value for the password warning period field, in days. Defaults to empty string.
scim_shadow_default_inact	Default value for the password inactivity period field, in days. Defaults to empty string.
scim_shadow_default_expire	Default value for the account expiration date field (days since January 1, 1970). Defaults to empty string.
scim_shadow_default_flag	Default value for the reserved flag field. Defaults to empty string.

Optional parameters for LDAP

SUNK adds optional parameters to nsscache for LDAP.

Default shell

The nsscache.nsscacheConfig.default.ldap_default_shell parameter provides a default shell for all users. Users may specify a different shell with the loginShell value in the user attributes configuration.

This parameter differs from the ldap_override_shell, which sets a shell that overrides the user's preferences in their LDAP profile.

Home directory override

Use the nsscache.nsscacheConfig.default.ldap_override_home_dir parameter for a home directory override. Use %%u to use the username in the home directory.

Typically, this is used to set a mount point for the home directories. For example, /mnt/home/%%u. This should match the mount specified in compute.VolumeMounts.

Verify and troubleshoot nsscache

You do not need to manually sync data with nsscache. It takes about two minutes for data to sync from an identity provider to SUNK through nsscache. If you encounter an issue with user data not being available, wait a few minutes and check again.

To validate that nsscache is working,

1. Log in to the Login pod for your cluster.

   Example

   $
    kubectl exec -it <LOGIN-POD> -c sshd -- /bin/bash
   

2. Access the /etc/nsscache directory and list the files within:

   Example

   $
   cd /etc/nsscache && ls

   This should return the following files:

   Example

   group.cache passwd.cache shadow.cache sshkey.cache

3. Check the contents of the group.cache, passwd.cache, shadow.cache, and sshkey.cache files directly for information about the users and groups in your directory, as shown with the cat command below:

   Example

   $
   cat sshkey.cache

   Alternatively, use the getent command with the group, passwd, or shadow options to list the system users and groups in addition to the data pulled by nsscache, as shown below:

   Example

   $
   getent passwd

   Note that the getent command does not directly retrieve SSH keys. To view the contents of the sshkey.cache file, you will need to use the cat command, as demonstrated above.