Use Prolog and Epilog Scripts

Set up a Slurm environment with prolog and epilog scripts

In a Slurm environment, prolog and epilog scripts are handy tools that run automatically at the start (prolog) and end (epilog) of every job. You can use these scripts for various tasks, like preparing the environment or cleaning up after a job. This guide will show you how to deploy and configure these scripts in SUNK using Kubernetes ConfigMaps.

First, you'll make separate ConfigMaps for your prolog and epilog scripts. Each ConfigMap should contain all the scripts you plan to use. Then, you'll configure SUNK to use the scripts in the ConfigMaps.

Create a prolog ConfigMap

Consider this example prolog script, which will be executed before every job.

Example

#!/usr/bin/env bash
set -e

echo "Prolog test executed"

Convert that script into a Kubernetes ConfigMap, like this:

Example

apiVersion: v1
kind: ConfigMap
metadata:
  name: slurm-prolog
data:
  prolog-test.sh: |
    #!/usr/bin/env bash
    set -e

    echo "Prolog test executed"

Replace prolog-test.sh with your script's name and the content with your actual script.

Save this YAML as slurm-prolog-configmap.yaml. Now apply the ConfigMap to your Kubernetes cluster:

Example

$ kubectl apply -f slurm-prolog-configmap.yaml

This step creates the ConfigMap in your cluster, but doesn't activate it in SUNK yet. We'll do that in the section below.

Create an epilog ConfigMap

Creating an epilog ConfigMap is similar to creating the prolog. Here's an example epilog script that will be executed after every job:

Example

#!/usr/bin/env bash
set -e

echo "Epilog test executed"

Convert that script into a Kubernetes ConfigMap, like this:

Example

apiVersion: v1
kind: ConfigMap
metadata:
  name: slurm-epilog
data:
  epilog-test.sh: |
    #!/usr/bin/env bash
    set -e

    echo "Epilog test executed"

Replace epilog-test.sh with your script's name and the content with your actual script.

Save this YAML as slurm-epilog-configmap.yaml, then apply the ConfigMap to your Kubernetes cluster:

Example

$ kubectl apply -f slurm-epilog-configmap.yaml

As before, this creates the ConfigMap, but it's not yet active in SUNK.

Configure SUNK to use the scripts

Once the scripts are in ConfigMaps, SUNK needs to know where to find them.

In the values.yaml file for the SUNK deployment, set the slurmConfig.slurmd.prologConfigMap and slurmConfig.slurmd.epilogConfigMap values to the ConfigMap names.

Here's an example:

Example

slurmConfig:
  slurmd:
    prologConfigMap: slurm-prolog
    epilogConfigMap: slurm-epilog

Replace slurm-prolog and slurm-epilog with the names of your ConfigMaps.

After updating values.yaml, apply the changes to the deployment:

Example

$ helm upgrade my-sunk-deployment chart/path -f values.yaml

Replace my-sunk-deployment with the name of your Slurm deployment and chart/path with the path to your Slurm chart. In the following example, we run this command from the root of the SUNK repository:

Example

$ helm upgrade slurm chart/slurm -f values.yaml

More information

Prolog and epilog scripts automatically execute at the start and end of each Slurm job, helping you manage job environments automatically. Remember to test these scripts to ensure they perform as expected in your job workflow.