Training and inference at scale made simple, efficient, and adaptable
Hugging Face Accelerate is a library that enables the same PyTorch code to run across any distributed configuration, to simplify model training and inference at scale.Accelerate includes a W&B Tracker, which this page shows how to use to log metrics, configuration, and artifacts from distributed training runs to Weights & Biases. For more information, see Accelerate Trackers in Hugging Face.
This section shows how to configure Accelerate to log experiment data to Weights & Biases during training. To get started with Accelerate and Weights & Biases, follow this pseudocode:
from accelerate import Accelerator# Tell the Accelerator object to log with wandbaccelerator = Accelerator(log_with="wandb")# Initialise your wandb run, passing wandb parameters and any config informationaccelerator.init_trackers( project_name="my_project", config={"dropout": 0.1, "learning_rate": 1e-2}, init_kwargs={"wandb": {"entity": "my-wandb-team"}} )...# Log to wandb by calling accelerator.log(); step is optionalaccelerator.log({"train_loss": 1.12, "valid_loss": 0.8}, step=global_step)# Make sure that the wandb tracker finishes correctlyaccelerator.end_training()
In more detail:
Pass log_with="wandb" when you initialize the Accelerator class.
Once Accelerate logs to Weights & Biases, you may want direct access to the underlying W&B run object to log artifacts, custom charts, or other data that the tracker doesn’t expose. To access the W&B tracker, use the Accelerator.get_tracker() method. Pass in the string corresponding to a tracker’s .name attribute, which returns the tracker on the main process.
wandb_tracker = accelerator.get_tracker("wandb")
From there, you can interact with the wandb run object as usual:
wandb_tracker.log_artifact(some_artifact_to_log)
Trackers built in Accelerate automatically execute on the correct process, so if a tracker only needs to run on the main process it does so automatically.To remove Accelerate’s wrapping entirely, you can achieve the same outcome with:
