Skip to main content
This tutorial shows you how to run Grouped Relative Policy Optimization (GRPO) training with torchforge using the Qwen3 1.7B model on a SUNK cluster. GRPO is a reinforcement learning technique for fine-tuning language models, and torchforge provides a PyTorch-based framework to run this training at scale. By the end of this tutorial, you have a working torchforge environment, a submitted Slurm batch job that runs GRPO training, and training metrics streamed to Weights & Biases. This guide is intended for ML practitioners with access to a SUNK cluster who want to experiment with reinforcement learning workflows.
Experimental status and tested version
  • torchforge is experimental. Expect potential bugs, incomplete features, and API changes.
  • This documentation and these instructions were tested with torchforge commit 8bd8d5d3c793ca6e2930b471b8ada67ce2458784.

​
Prerequisites

  • Access to a SUNK cluster with GPU nodes.
  • Minimum one H100 node for GRPO training.
  • 10 GB available disk space.
  • A GitHub access token.
  • A Weights & Biases API key.
  • Conda.

​
Initialize GitHub and Weights & Biases credentials

The torchforge installation pulls dependencies from GitHub, and the training job reports metrics to Weights & Biases. Export your credentials so both services are available during installation and training. At a Slurm login node, run the following commands:
  1. Export your GitHub token. Replace [GITHUB-TOKEN] with your GitHub access token:
  2. Export your Weights & Biases API key: Get your Weights & Biases API key from wandb.ai (User Settings > API keys). Replace [WANDB-API-KEY] with your API key:

​
Install torchforge

Next, set up an isolated conda environment, clone the torchforge repository at the tested commit, and run the project’s installation script. Using a dedicated conda environment keeps torchforge’s dependencies separate from the rest of the system. To install torchforge, run the following commands:
  1. Initialize conda:
  2. Create a conda environment:
    You should see output similar to the following:
  3. Activate torchforge:
  4. Clone the repository:
  5. Run the installation script: The installation script takes 5 to 15 minutes to complete.
    You should see output similar to the following. You don’t need to re-activate the conda environment:
  6. Verify the installation:
    You should see output similar to the following:

​
Run GRPO training

With torchforge installed, you can launch a short GRPO training run as a Slurm batch job. The following steps reduce the training step count for a quick test, define a batch script that requests an H100 node, submit the job, and tail the logs so you can watch training progress. To run GRPO training, complete the following steps:
  1. Edit the training configuration to reduce steps for testing: The default configuration runs for 1,000,000 steps. Lower this to 10 steps to verify the end-to-end setup quickly without waiting for a full training run.
  2. Create a Slurm batch script torchforge-training.sbatch:
  3. Submit the job:
  4. Monitor logs:
    Eventually, you see logs like the following:
You now have a completed GRPO training run on SUNK. The job produced training logs locally and reported metrics to your Weights & Biases project, where you can inspect loss, rewards, and other training statistics. View training metrics in your Weights & Biases dashboard at wandb.ai.

​
Additional resources

Last modified on May 27, 2026