Log media and objects

Log media with the Weights & Biases Python SDK to inspect model inputs and outputs and compare results across runs, models, and datasets. Supported types include images, audio, video, histograms, 3D point clouds, molecules, and HTML. For API details, see the Data Types reference. For examples, see the model predictions report or watch the video walkthrough.

Pre-requisites

Some media types require additional dependencies. To install the Weights & Biases SDK with media dependencies, run:

pip install wandb[media]

Images

Use wandb.Image to log images from NumPy arrays, PIL images, or local files. Log images to inspect inputs, outputs, filter weights, and activations.

CoreWeave recommends logging fewer than 50 images per step to reduce the impact on training performance and image loading times.

The following tabs demonstrate how to log images using different input types.

Arrays
PIL Images
Image files

Provide arrays directly when constructing images manually, such as by using make_grid from torchvision.

Weights & Biases converts arrays to PNG images using Pillow.

import wandb
import numpy as np

with wandb.init(project="image-log-example") as run:
    images = wandb.Image(image_array, caption="Top: Output, Bottom: Input")
    run.log({"examples": images})

For NumPy arrays, use the shape (height, width, channels). Weights & Biases infers grayscale, RGB, or RGBA from a channel count of 1, 3, or 4, respectively. Images are saved as PNG files by default.Pixel values are normalized to the range 0–255 if values fall outside that range or all values fall within 0–1. To turn off normalization, pass normalize=False. For more control over array conversion, create a PIL image as shown in the PIL images tab.

Create a PIL.Image and pass it to wandb.Image.

from PIL import Image

image_array = np.random.randint(0, 256, (128, 128, 3), dtype=np.uint8)

with wandb.init(project="pil-image-example") as run:
    # Create a PIL image from a NumPy array
    image = Image.fromarray(image_array)

    # Optionally, convert to RGB if needed
    if image.mode != "RGB":
        image = image.convert("RGB")

    # Log the image
    run.log({"example": wandb.Image(image, caption="My Image")})

To log an existing image file, pass its path to wandb.Image.

with wandb.init(project="media-log-example") as run:
    run.log({"example": wandb.Image("myimage.jpg")})

To compare images across training steps, expand the image panel in the Weights & Biases App UI and use the step slider. Select an image to open it in full-screen view, where you can zoom and pan with the mouse or keyboard shortcuts. To compare images or videos across runs, steps, or indices in one view, use Compare mode.

Image overlays

Log semantic segmentation masks to adjust their opacity and inspect changes across steps in the Weights & Biases App UI.

Segmentation Masks
Bounding Boxes

Log images with segmentation masks or bounding boxes to inspect model predictions and ground truth in the Weights & Biases App UI.

Pass a dictionary to the masks argument of wandb.Image. Each key names a mask, such as predictions or ground_truth. Each value is a dictionary with the following fields:

Field	Description
`mask_data`	A 2D NumPy array containing an integer class label for each pixel. Provide either `mask_data` or `path`.
`path`	The path to a saved mask image. Provide either `path` or `mask_data`.
`class_labels`	Optional dictionary that maps integer class labels to readable names.

To log multiple masks for an image, provide a separate dictionary for each mask:

mask_data = np.array([[1, 2, 2, ..., 2, 2, 1], ...])

class_labels = {1: "tree", 2: "car", 3: "road"}

mask_img = wandb.Image(
    image,
    masks={
        "predictions": {"mask_data": mask_data, "class_labels": class_labels},
        "ground_truth": {
            # ...
        },
        # ...
    },
)

Segmentation masks for each key are defined at each step (each call to wandb.Run.log()).

If steps provide different values for the same mask key, only the most recent value for that key is applied to the image.
If steps provide different mask keys, the visibility controls show all keys, but only masks defined at the selected step are applied to the image. Toggling a mask that is not defined at that step does not change the image.

For a complete example, see the segmentation masks report and example notebook.

Log images with bounding boxes, then use filters and visibility controls to inspect groups of boxes in the Weights & Biases App UI.

Pass a dictionary to the boxes argument of wandb.Image. Each key names a group, such as predictions or ground_truth. Each value contains these fields:

Field	Description
`box_data`	A list of dictionaries, one for each bounding box.
`class_labels`	Optional dictionary that maps class IDs to readable names. If omitted, Weights & Biases generates names such as `class_0` and `class_1`.

Each dictionary in box_data describes a box:

Field	Description
`position`	The box position and size. Use either `minX`, `maxX`, `minY`, and `maxY`, or `middle`, `width`, and `height`. For `middle`, provide `[x, y]` coordinates. Boxes can use different formats.
`class_id`	An integer class ID that corresponds to an entry in `class_labels`.
`scores`	A dictionary that maps score names to numeric values. Use these scores to filter boxes in the Weights & Biases App UI.
`domain`	Coordinate units. Set to `"pixel"` for pixel coordinates. By default, coordinates are fractions of the image dimensions, from 0 to 1.
`box_caption`	Optional text to display on the box.

This example logs one box with relative coordinates and another with pixel coordinates:

import wandb

class_id_to_label = {
    1: "car",
    2: "road",
    3: "building"
}

img = wandb.Image(
    image,
    boxes={
        "predictions": {
            "box_data": [
                {
                    # one box expressed in the default relative/fractional domain
                    "position": {"minX": 0.1, "maxX": 0.2, "minY": 0.3, "maxY": 0.4},
                    "class_id": 2,
                    "box_caption": class_id_to_label[2],
                    "scores": {"acc": 0.1, "loss": 1.2},
                    # another box expressed in the pixel domain
                    # (for illustration purposes only, all boxes are likely
                    # to be in the same domain/format)
                    "position": {"middle": [150, 20], "width": 68, "height": 112},
                    "domain": "pixel",
                    "class_id": 3,
                    "box_caption": "a building",
                    "scores": {"acc": 0.5, "loss": 0.7},
                    # ...
                    # Log as many boxes as needed
                }
            ],
            "class_labels": class_id_to_label,
        },
        # Log each meaningful group of boxes with a unique key name
        "ground_truth": {
            # ...
        },
    },
)

with wandb.init(project="my_project") as run:
    run.log({"driving_scene": img})

For an interactive example, see the bounding boxes report.

Image overlays in Tables

To display image overlays in a Weights & Biases Table, add a wandb.Image object with masks or bounding boxes to each row.

Segmentation Masks
Bounding Boxes

Interactive Segmentation Masks in Tables

This example logs one image with segmentation masks for each row in the table:

table = wandb.Table(columns=["ID", "Image"])

for id, img, label in zip(ids, images, labels):
    mask_img = wandb.Image(
        img,
        masks={
            "prediction": {
                "mask_data": label,
                "class_labels": class_labels
                }
            # ...
        },
    )

    table.add_data(id, mask_img)

with wandb.init(project="my_project") as run:
    run.log({"Table": table})

This example logs one image with bounding boxes for each row in the table:

table = wandb.Table(columns=["ID", "Image"])

for id, img, boxes in zip(ids, images, boxes_set):
    box_img = wandb.Image(
        img,
        boxes={
            "prediction": {
                "box_data": [
                    {
                        "position": {
                            "minX": box["minX"],
                            "minY": box["minY"],
                            "maxX": box["maxX"],
                            "maxY": box["maxY"],
                        },
                        "class_id": box["class_id"],
                        "box_caption": box["caption"],
                        "domain": "pixel",
                    }
                    for box in boxes
                ],
                "class_labels": class_labels,
            }
        },
    )

Histograms

Use wandb.Histogram to log numeric distributions, such as model gradients. In the Weights & Biases App UI, the x-axis shows training steps, the y-axis shows metric values, and color represents the count in each bin.

Basic Histogram Logging
Flexible Histogram Logging

Pass a sequence of numbers, such as a list, array, or tensor, to wandb.Histogram. Weights & Biases flattens arrays and tensors and uses numpy.histogram() to calculate the histogram.By default, histograms use 64 bins. To customize the number of bins, set num_bins a value up to 512.

with wandb.init() as run:
    run.log({"gradients": wandb.Histogram(grads)})

To control histogram calculation, call np.histogram() and pass the returned tuple to np_histogram:

import numpy as np

np_hist_grads = np.histogram(grads, density=True, range=(0.0, 1.0))

with wandb.init() as run:
    run.log({"gradients": wandb.Histogram(np_hist_grads)})

Optionally set density=True to represent bin values as a probability density rather than counts.

3D visualizations

Use wandb.Object3D to log 3D point clouds and lidar scenes with bounding boxes. Provide a NumPy array with point coordinates and optional colors or category labels. Point clouds are interactive in the Weights & Biases App UI. Drag to rotate the view, zoom to inspect details, or hold Control while using the mouse to move through the scene. The following report shows scenes from the Lyft dataset with 3D bounding boxes:

The Weights & Biases App UI displays up to 300,000 points per point cloud.

NumPy array formats

Use one of the following array formats, where n is the number of points:

Shape	Point format	Description
`(n, 3)`	`[x, y, z]`	Point coordinates.
`(n, 4)`	`[x, y, z, category]`	Coordinates and a category from 1 to 14, such as a segmentation class.
`(n, 6)`	`[x, y, z, r, g, b]`	Coordinates and red, green, and blue values from 0 to 255.

Python object

Use wandb.Object3D.from_point_cloud() to create a scene with points, bounding boxes, and vectors.

import wandb
import numpy as np

points = np.array([
    [0.2, 0.3, 0.4, 255, 0, 0],
    [0.5, 0.6, 0.7, 0, 255, 0],
    [0.8, 0.9, 0.2, 0, 0, 255],
])

boxes = [{
    "corners": [
        [0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0],
        [0, 0, 1], [1, 0, 1], [1, 1, 1], [0, 1, 1],
    ],
    "color": [0, 0, 255],
    "label": "example region",
    "score": 0.6,
}]

scene = wandb.Object3D.from_point_cloud(
    points=points,
    boxes=boxes,
    vectors=[{"start": [0, 0, 0], "end": [0.1, 0.2, 0.5], "color": [255, 0, 0]}],
    point_cloud_type="lidar/beta",
)

with wandb.init() as run:
    run.log({"my_first_point_cloud": scene})

To move through a point cloud in the Weights & Biases App UI, hold Control while moving the mouse.

Point cloud files

Use wandb.Object3D.from_file() to load point cloud data from a JSON file:

run.log({"my_cloud_from_file": wandb.Object3D.from_file(
     "./my_point_cloud.pts.json"
)})

The following JSON illustrates the format for point cloud data:

{
  "points": [
    [0.2, 0.3, 0.4, 255, 0, 0],
    [0.5, 0.6, 0.7, 0, 255, 0],
    [0.8, 0.9, 0.2, 0, 0, 255]
  ],
  "boxes": [
    {
      "corners": [
        [0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0],
        [0, 0, 1], [1, 0, 1], [1, 1, 1], [0, 1, 1]
      ],
      "color": [0, 0, 255],
      "label": "example region",
      "score": 0.6
    }
  ],
  "type": "lidar/beta"
}

NumPy arrays

Use wandb.Object3D.from_numpy() with any of the supported array formats:

# Coordinates only.
xyz_points = np.array([[0.4, 1, 1.3], [1, 1, 1], [1.2, 1, 1.2]])

# Coordinates and category labels.
category_points = np.array([[0.4, 1, 1.3, 1], [1, 1, 1, 1], [1.2, 1, 1.2, 12]])

# Coordinates and RGB colors.
rgb_points = np.array([
    [0.4, 1, 1.3, 255, 0, 0],
    [1, 1, 1, 0, 255, 0],
    [1.2, 1, 1.2, 0, 0, 255],
])

with wandb.init() as run:
    run.log({
        "xyz_point_cloud": wandb.Object3D.from_numpy(xyz_points),
        "category_point_cloud": wandb.Object3D.from_numpy(category_points),
        "rgb_point_cloud": wandb.Object3D.from_numpy(rgb_points),
    })

Molecules

Use wandb.Molecule to log molecular data and inspect 3D structures in the Weights & Biases App UI.

run.log({"protein": wandb.Molecule("6lu7.pdb")})

Supported file types are pdb, pqr, mmcif, mcif, cif, sdf, sd, gro, mol2, and mmtf. You can also log SMILES strings, RDKit mol files, and rdkit.Chem.rdchem.Mol objects. These examples require RDKit.

from rdkit import Chem

resveratrol = rdkit.Chem.MolFromSmiles("Oc1ccc(cc1)C=Cc1cc(O)cc(c1)O")

with wandb.init() as run:
    run.log(
        {
            "resveratrol": wandb.Molecule.from_rdkit(resveratrol),
            "green fluorescent protein": wandb.Molecule.from_rdkit("2b3p.mol"),
            "acetaminophen": wandb.Molecule.from_smiles("CC(=O)Nc1ccc(O)cc1"),
        }
    )

For an interactive example, see the AlphaFold workspace.

2D view of a molecule

You can log a 2D view of a molecule using the wandb.Image data type and rdkit:

from rdkit import Chem
from rdkit.Chem import AllChem, Draw

molecule = Chem.MolFromSmiles("CC(=O)O")
AllChem.Compute2DCoords(molecule)
pil_image = Draw.MolToImage(molecule, size=(300, 300))

with wandb.init() as run:
    run.log({"acetic_acid": wandb.Image(pil_image)})

Other media

Weights & Biases also supports logging of a variety of other media types.

Audio

Use wandb.Audio to log audio data. Set sample_rate to the recording’s sample rate in hertz.

with wandb.init() as run:
    run.log({"whale songs": wandb.Audio(np_array, caption="OooOoo", sample_rate=32)})

You can log up to 100 audio clips per step. For more information, see audio-file.

Video

Use wandb.Video to log a video file:

with wandb.init() as run:
    run.log({"video": wandb.Video("myvideo.mp4")})

To log a NumPy array of video frames, specify the output format and frame rate:

with wandb.init() as run:
    run.log({"video": wandb.Video(video_array, fps=4, format="gif")})

Use the shape (frames, channels, height, width) for a single video or (batch, frames, channels, height, width) for a batch. Encoding NumPy arrays requires FFmpeg and MoviePy. Supported formats are "gif", "mp4", "webm", and "ogg". When passing an array or an in-memory stream, specify format. When encoding raw frames, the default frame rate is 4 frames per second. When you provide a file path, Weights & Biases checks that the file exists and uses a supported format. When you provide a BytesIO object, Weights & Biases creates a temporary file with the specified format as its extension. To view and play videos, open a media panel in your project workspace, run workspace, or report. Use Compare mode to compare videos across runs or steps.

Text

Use wandb.Table to display text alongside predictions, labels, or other values. Specify column names when you create the table. By default, the column headers are ["Input", "Output", "Expected"]. To ensure optimal UI performance, the default maximum number of rows is set to 10,000. However, you can explicitly override the maximum with wandb.Table.MAX_ROWS = {DESIRED_MAX}.

columns = ["Text", "Predicted Sentiment", "True Sentiment"]
data = [["I love my phone", "1", "1"], ["My phone is terrible", "0", "-1"]]

with wandb.init(project="my_project") as run:
    table = wandb.Table(data=data, columns=columns)
    run.log({"examples": table})

Alternatively, add rows with wandb.Table.add_data():

table = wandb.Table(columns=columns)
table.add_data("I love my phone", "positive", "positive")
table.add_data("My phone stopped working", "negative", "negative")
run.log({"text_examples": table})

To create a table from a pandas DataFrame, pass it to dataframe:

table = wandb.Table(dataframe=my_dataframe)
with wandb.init() as run:
    run.log({"examples": table})

For more usage information, see string.

HTML

Use wandb.Html to display custom HTML in a panel on the run page. Pass an HTML file or a string:

with wandb.init() as run:
    run.log({"custom_string": wandb.Html('<a href="https://mysite">Link</a>')})
    
    with open("some.html") as f:
        run.log({"custom_file": wandb.Html(f)})

By default, Weights & Biases adds styles to the HTML. To disable this behavior, pass inject=False. To turn off this behavior pass inject=False:

with wandb.init() as run:
    with open("some.html") as html_file:
        run.log({"custom_file": wandb.Html(html_file, inject=False)})

For more usage information, see html-file.

Weights & Biases

Guides

Integrations

Reference

Support

Pre-requisites

Images

Image overlays

Image overlays in Tables

Histograms

3D visualizations

NumPy array formats

Python object

Point cloud files

NumPy arrays

Molecules

2D view of a molecule

Other media

Audio

Video

Text

HTML

​Pre-requisites

​Images

​Image overlays

​Image overlays in Tables

​Histograms

​3D visualizations

​NumPy array formats

​Python object

​Point cloud files

​NumPy arrays

​Molecules

​2D view of a molecule

​Other media

​Audio

​Video

​Text

​HTML

Pre-requisites

Images

Image overlays

Image overlays in Tables

Histograms

3D visualizations

NumPy array formats

Python object

Point cloud files

NumPy arrays

Molecules

2D view of a molecule

Other media

Audio

Video

Text

HTML