> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Hugging Face
> Track Hugging Face models with Weights & Biases to visualize hyperparameters, metrics, and GPU utilization.
export const ColabLink = ({url}) =>
Try in Colab
;
This tutorial shows you how to use the Weights & Biases integration with [Hugging Face Transformers](https://github.com/huggingface/transformers) to automatically track training and evaluation metrics, hyperparameters, and system stats while fine-tuning a model. By following this tutorial, you learn how to visualize your model's performance through the [Weights & Biases](https://wandb.ai/site) dashboard so you can compare experiments and iterate on your models with confidence.
You can compare hyperparameters, output metrics, and system stats like GPU utilization across your models.
## Why use Weights & Biases
* **Unified dashboard**: Central repository for all your model metrics and predictions.
* **Lightweight**: No code changes required to integrate with Hugging Face.
* **Accessible**: Free for individuals and academic teams.
* **Secure**: All projects are private by default.
* **Trusted**: Used by machine learning teams at OpenAI, Toyota, Lyft, and more.
Weights & Biases works like GitHub for machine learning models. Save machine learning experiments to your private, hosted dashboard. Experiment with the confidence that Weights & Biases saves all versions of your models, no matter where you run your scripts.
Weights & Biases lightweight integrations work with any Python script. Sign up for a free Weights & Biases account to start tracking and visualizing your models.
In the Hugging Face Transformers repository, Weights & Biases has instrumented the Trainer to automatically log training and evaluation metrics to Weights & Biases at each logging step.
Here's an in-depth look at how the integration works: [Hugging Face + W\&B Report](https://forge.coreweave.com/wandb/jxmorris12/huggingface-demo/reports/Train-a-model-with-Hugging-Face-and-Weights-%26-Biases--VmlldzoxMDE2MTU).
## Install, import, and log in
This section sets up the environment you need to run the tutorial. Install the Hugging Face and Weights & Biases libraries, and download the GLUE dataset and training script for this tutorial:
* [Hugging Face Transformers](https://github.com/huggingface/transformers): Natural language models and datasets.
* [Weights & Biases](/forge-home): Experiment tracking and visualization.
* [GLUE dataset](https://gluebenchmark.com/): A language understanding benchmark dataset.
* [GLUE script](https://raw.githubusercontent.com/huggingface/transformers/refs/heads/main/examples/pytorch/text-classification/run_glue.py): Model training script for sequence classification.
```notebook theme={"system"}
!pip install datasets wandb evaluate accelerate -qU
!wget https://raw.githubusercontent.com/huggingface/transformers/refs/heads/main/examples/pytorch/text-classification/run_glue.py
```
```notebook theme={"system"}
# the run_glue.py script requires transformers dev
!pip install -q git+https://github.com/huggingface/transformers
```
Before continuing, you must [sign up for a free account](https://id.coreweave.com/signup). An account is required to send your run data to a Weights & Biases dashboard.
## Add your API key
Authenticating with your API key links this notebook to your Weights & Biases account so that runs are logged to your projects. After you sign up, run the next cell and click the link to get your API key and authenticate this notebook.
```python theme={"system"}
import wandb
wandb.login()
```
Optionally, you can set environment variables to customize what Weights & Biases logs during training. For example, you can log both gradients and parameters by setting `WANDB_WATCH=all`. See the [Hugging Face integration guide](/products/wandb/integrations/huggingface) for the full list of options.
```python theme={"system"}
# Optional: log both gradients and parameters
%env WANDB_WATCH=all
```
## Train the model
With the environment configured and authentication complete, you're ready to start a training run. Call the downloaded training script [`run_glue.py`](https://huggingface.co/transformers/examples.html#glue) and see training automatically get tracked to the Weights & Biases dashboard. This script fine-tunes BERT on the Microsoft Research Paraphrase Corpus (pairs of sentences with human annotations indicating whether they're semantically equivalent).
```python theme={"system"}
%env WANDB_PROJECT=huggingface-demo
%env TASK_NAME=MRPC
!python run_glue.py \
--model_name_or_path bert-base-uncased \
--task_name $TASK_NAME \
--do_train \
--do_eval \
--max_seq_length 256 \
--per_device_train_batch_size 32 \
--learning_rate 2e-4 \
--num_train_epochs 3 \
--output_dir /tmp/$TASK_NAME/ \
--overwrite_output_dir \
--logging_steps 50
```
## Visualize results in the dashboard
After training starts, you can monitor metrics in real time. Click the link printed out by the previous cell, or go to [wandb.ai](https://app.wandb.ai) to see your results stream in live. The link to see your run in the browser appears after all the dependencies are loaded. Look for the following output: "**wandb**: View run at \[URL to your unique run]"
### Visualize model performance
Look across experiments, zoom in on findings, and visualize high-dimensional data.
### Compare architectures
Here's an example comparing [BERT versus DistilBERT](https://forge.coreweave.com/wandb/jack-morris/david-vs-goliath/reports/Does-model-size-matter%3F-Comparing-BERT-and-DistilBERT-using-Sweeps--VmlldzoxMDUxNzU). The automatic line plot visualizations show how different architectures affect the evaluation accuracy throughout training.
## Track key information by default
This section describes what Weights & Biases captures automatically so you know what data is available in your dashboard without additional configuration. Weights & Biases saves a new run for each experiment. Here's the information saved by default:
* **Hyperparameters**: Weights & Biases saves settings for your model in Config.
* **Model metrics**: Weights & Biases saves time series data of metrics streaming in to Log.
* **Terminal logs**: Weights & Biases saves command line outputs and makes them available in a tab.
* **System metrics**: GPU and CPU utilization, memory, and temperature.
## Learn more
* [Video walkthroughs on YouTube](http://wandb.me/youtube)
* **Unified dashboard**: Central repository for all your model metrics and predictions.
* **Lightweight**: No code changes required to integrate with Hugging Face.
* **Accessible**: Free for individuals and academic teams.
* **Secure**: All projects are private by default.
* **Trusted**: Used by machine learning teams at OpenAI, Toyota, Lyft, and more.
Weights & Biases works like GitHub for machine learning models. Save machine learning experiments to your private, hosted dashboard. Experiment with the confidence that Weights & Biases saves all versions of your models, no matter where you run your scripts.
Weights & Biases lightweight integrations work with any Python script. Sign up for a free Weights & Biases account to start tracking and visualizing your models.
In the Hugging Face Transformers repository, Weights & Biases has instrumented the Trainer to automatically log training and evaluation metrics to Weights & Biases at each logging step.
Here's an in-depth look at how the integration works: [Hugging Face + W\&B Report](https://forge.coreweave.com/wandb/jxmorris12/huggingface-demo/reports/Train-a-model-with-Hugging-Face-and-Weights-%26-Biases--VmlldzoxMDE2MTU).
## Install, import, and log in
This section sets up the environment you need to run the tutorial. Install the Hugging Face and Weights & Biases libraries, and download the GLUE dataset and training script for this tutorial:
* [Hugging Face Transformers](https://github.com/huggingface/transformers): Natural language models and datasets.
* [Weights & Biases](/forge-home): Experiment tracking and visualization.
* [GLUE dataset](https://gluebenchmark.com/): A language understanding benchmark dataset.
* [GLUE script](https://raw.githubusercontent.com/huggingface/transformers/refs/heads/main/examples/pytorch/text-classification/run_glue.py): Model training script for sequence classification.
```notebook theme={"system"}
!pip install datasets wandb evaluate accelerate -qU
!wget https://raw.githubusercontent.com/huggingface/transformers/refs/heads/main/examples/pytorch/text-classification/run_glue.py
```
```notebook theme={"system"}
# the run_glue.py script requires transformers dev
!pip install -q git+https://github.com/huggingface/transformers
```
Before continuing, you must [sign up for a free account](https://id.coreweave.com/signup). An account is required to send your run data to a Weights & Biases dashboard.
## Add your API key
Authenticating with your API key links this notebook to your Weights & Biases account so that runs are logged to your projects. After you sign up, run the next cell and click the link to get your API key and authenticate this notebook.
```python theme={"system"}
import wandb
wandb.login()
```
Optionally, you can set environment variables to customize what Weights & Biases logs during training. For example, you can log both gradients and parameters by setting `WANDB_WATCH=all`. See the [Hugging Face integration guide](/products/wandb/integrations/huggingface) for the full list of options.
```python theme={"system"}
# Optional: log both gradients and parameters
%env WANDB_WATCH=all
```
## Train the model
With the environment configured and authentication complete, you're ready to start a training run. Call the downloaded training script [`run_glue.py`](https://huggingface.co/transformers/examples.html#glue) and see training automatically get tracked to the Weights & Biases dashboard. This script fine-tunes BERT on the Microsoft Research Paraphrase Corpus (pairs of sentences with human annotations indicating whether they're semantically equivalent).
```python theme={"system"}
%env WANDB_PROJECT=huggingface-demo
%env TASK_NAME=MRPC
!python run_glue.py \
--model_name_or_path bert-base-uncased \
--task_name $TASK_NAME \
--do_train \
--do_eval \
--max_seq_length 256 \
--per_device_train_batch_size 32 \
--learning_rate 2e-4 \
--num_train_epochs 3 \
--output_dir /tmp/$TASK_NAME/ \
--overwrite_output_dir \
--logging_steps 50
```
## Visualize results in the dashboard
After training starts, you can monitor metrics in real time. Click the link printed out by the previous cell, or go to [wandb.ai](https://app.wandb.ai) to see your results stream in live. The link to see your run in the browser appears after all the dependencies are loaded. Look for the following output: "**wandb**: View run at \[URL to your unique run]"
### Visualize model performance
Look across experiments, zoom in on findings, and visualize high-dimensional data.
### Compare architectures
Here's an example comparing [BERT versus DistilBERT](https://forge.coreweave.com/wandb/jack-morris/david-vs-goliath/reports/Does-model-size-matter%3F-Comparing-BERT-and-DistilBERT-using-Sweeps--VmlldzoxMDUxNzU). The automatic line plot visualizations show how different architectures affect the evaluation accuracy throughout training.
## Track key information by default
This section describes what Weights & Biases captures automatically so you know what data is available in your dashboard without additional configuration. Weights & Biases saves a new run for each experiment. Here's the information saved by default:
* **Hyperparameters**: Weights & Biases saves settings for your model in Config.
* **Model metrics**: Weights & Biases saves time series data of metrics streaming in to Log.
* **Terminal logs**: Weights & Biases saves command line outputs and makes them available in a tab.
* **System metrics**: GPU and CPU utilization, memory, and temperature.
## Learn more
* [Video walkthroughs on YouTube](http://wandb.me/youtube)