Start a sweep agent

This page explains how to start a sweep agent on one or more machines so that Weights & Biases can run your hyperparameter search. Sweep agents use the sweep configuration that you define when you initialize a sweep to explore different hyperparameter combinations. Weights & Biases creates a new run for each hyperparameter combination that the sweep agent runs. See Manage sweeps to learn how to pause, resume, stop, or cancel a running sweep agent. Before you continue:

Configure your training script to create and track hyperparameter combinations with Weights & Biases. See Add Weights & Biases to your code for examples.
Define a configuration file for your sweep.
Initialize the sweep and note its sweep ID.

Start a scheduler

If you specify a custom scheduler in your sweep configuration, you need to start the scheduler before launching any agents.

Sweeps that use the default hosted controller don’t require starting a local scheduler. Only start wandb sweep-scheduler if your sweep configuration includes a custom scheduler.

In the scheduler environment, install the dependencies for your engine. If you configured scheduler.source, make the source file and its dependencies available there, and run the scheduler from a directory where the relative source path resolves. Replace ENTITY, PROJECT, and SWEEP_ID with the entity, project, and ID of your initialized sweep:

wandb sweep-scheduler ENTITY/PROJECT/SWEEP_ID

Use one scheduler per sweep. Multiple schedulers for the same sweep don’t share optimizer state and can generate duplicate runs. Keep the scheduler running as a separate process while agents execute your training code.

Option	Description
`--batch-size`	Number of unfinished runs to keep available for agents. For example, if nine runs are running and none are queued, the scheduler generates one more.
`--poll-interval`	Seconds between polls of the sweep’s runs.

For all command options and default values, see wandb sweep-scheduler. The scheduler respects sweep lifecycle commands, including pausing, resuming, and stopping the sweep. If you kill or delete the sweep, the scheduler exits.

Restart a scheduler

To restart a scheduler, run wandb sweep-scheduler with the same sweep path. The scheduler reads the existing run data to initialize the engine, then continues optimization. This restores observations from Weights & Biases rather than a saved optimizer checkpoint. If you use a custom optimizer factory, see Handle scheduler restarts for requirements on optimizer state and source files.

Start an agent

Each agent environment requires the Weights & Biases Python SDK (wandb), your training program, and its dependencies. If you started a scheduler, start agents in separate terminals or environments. Start additional agents with the same sweep path to evaluate configurations concurrently. See Parallelize agents. These examples show how to start a sweep agent with the W&B CLI or from Python. You need the sweep ID that Weights & Biases returns when you initialize the sweep. You can use the short sweep ID on its own (for example, dtzl1o7u), or include the entity and project as a path:

entity/project/sweep_ID

Where:

entity: Your Weights & Biases team name.
project: The name of the project where you want Weights & Biases to store the output of the run. If you don’t specify the project, Weights & Biases puts the run in a project called “Uncategorized”.
sweep_ID: The pseudo-random, unique ID that Weights & Biases generates.

CLI
Python script or notebook

Use the wandb agent command to start a sweep agent. Provide the sweep ID that Weights & Biases returns when you initialize the sweep.Replace [SWEEP-ID] with your sweep ID in the following command:

wandb agent [SWEEP-ID]

For graceful shutdown when you interrupt the agent (for example, with Ctrl+C), use wandb agent --forward-signals [SWEEP-ID] so the current run receives the signal and can shut down cleanly. See Signal handling and sweep runs for details.

Use wandb.agent() to start a sweep agent. Provide the sweep ID that Weights & Biases returns when you initialize the sweep, along with the name of the function that serves as the entrypoint to your training script.Replace [SWEEP-ID] with your sweep ID and [FUNCTION-NAME] with the name of your training function in the following code:

wandb.agent(sweep_id="[SWEEP-ID]", function="[FUNCTION-NAME]")

Weights & Biases only supports signal forwarding from the agent to the training run when you use the CLI (wandb agent --forward-signals). Weights & Biases doesn’t support signal forwarding for wandb.agent() in Python because the training function runs in a thread, not as a child process.See the Python script or notebook tab in Add Weights & Biases to your code for an example of how to set up your training script if you use this method.

MultiprocessingYou must wrap your wandb.agent() and wandb.sweep() calls with if __name__ == '__main__': if you use the Python standard library’s multiprocessing or PyTorch’s pytorch.multiprocessing package. For example:

if __name__ == '__main__':
    wandb.agent(sweep_id="[SWEEP-ID]", function="[FUNCTION]", count="[COUNT]")

Wrapping your code with this convention ensures that Python only runs it when you run the script directly, and not when a worker process imports it as a module.For more information about multiprocessing, see Python standard library multiprocessing or PyTorch multiprocessing. For more information about the if __name__ == '__main__': convention, see Defining main functions in Python.

Limit the number of runs a sweep agent tries

By default, random and Bayesian searches run indefinitely, so cap how many runs an agent attempts. Specify the number of runs a sweep agent should try to bound its work. The following code snippets demonstrate how to set a maximum number of runs with the CLI and within a Jupyter notebook or Python script.

Random and Bayesian searches run indefinitely. If you don’t set a run count, you must stop the process from the command line, from within your Python script, or from the Sweeps UI.

CLI
Python script or notebook

First, initialize your sweep with the wandb sweep command. For more information, see Initialize sweeps.

wandb sweep config.yaml

Next, pass an integer value to the count flag to set the maximum number of runs to try.

NUM=10
SWEEPID="dtzl1o7u"
wandb agent --count $NUM $SWEEPID

First, initialize your sweep. For more information, see Initialize sweeps.

sweep_id = wandb.sweep(sweep_config)

Next, start the sweep job. Provide the sweep ID that sweep initiation generates. Pass an integer value to the count parameter to set the maximum number of runs to try.

sweep_id, count = "dtzl1o7u", 10
wandb.agent(sweep_id, count=count)

If you start a new run after the sweep agent finishes, within the same script or notebook, call wandb.teardown() before you start the new run.

Weights & Biases

Guides

Integrations

Reference

Support

Start a scheduler

Restart a scheduler

Start an agent

Limit the number of runs a sweep agent tries

​Start a scheduler

​Restart a scheduler

​Start an agent

​Limit the number of runs a sweep agent tries

Start a scheduler

Restart a scheduler

Start an agent

Limit the number of runs a sweep agent tries