Use Weave to automatically track and log calls made using DSPy modules and functions
DSPy is a framework for algorithmically optimizing language model (LM) prompts and weights, especially when you use LMs one or more times within a pipeline. Weave automatically tracks and logs calls made with DSPy modules and functions. This guide shows you how to enable Weave tracing for DSPy programs, track custom modules and signatures, and capture traces from DSPy optimizers and evaluations. With these traces, you can debug, analyze, and improve your DSPy applications.
This section shows you how to enable automatic tracing of DSPy calls in Weave. Store traces of language model applications in a central location, both during development and in production. These traces are useful for debugging and as a dataset that helps you improve your application.Weave automatically captures traces for DSPy. To start tracking, call weave.init(project_name="[YOUR-WANDB-PROJECT-NAME]") and use the library as normal. Replace [YOUR-OPENAI-API-KEY] with your OpenAI API key and [YOUR-WANDB-PROJECT-NAME] with your Weights & Biases project name.
import osimport dspyimport weaveos.environ["OPENAI_API_KEY"] = "[YOUR-OPENAI-API-KEY]"weave.init(project_name="[YOUR-WANDB-PROJECT-NAME]")lm = dspy.LM('openai/gpt-4o-mini')dspy.configure(lm=lm)classify = dspy.Predict("sentence -> sentiment")classify(sentence="it's a charming and often affecting journey.")
With tracing enabled, Weave logs every LM call your DSPy program makes to your Weave project, where you can inspect inputs, outputs, and metadata.
Beyond built-in calls, Weave also traces the custom modules and signatures you define. A Module is the building block with learnable parameters for DSPy programs that abstracts a prompting technique. A Signature is a declarative specification of input/output behavior of a DSPy Module. Weave automatically tracks all built-in and custom Signature and Module objects in your DSPy programs. Replace [YOUR-OPENAI-API-KEY] with your OpenAI API key and [YOUR-WANDB-PROJECT-NAME] with your Weights & Biases project name.
import osimport dspyimport weaveos.environ["OPENAI_API_KEY"] = "[YOUR-OPENAI-API-KEY]"weave.init(project_name="[YOUR-WANDB-PROJECT-NAME]")class Outline(dspy.Signature): """Outline a thorough overview of a topic.""" topic: str = dspy.InputField() title: str = dspy.OutputField() sections: list[str] = dspy.OutputField() section_subheadings: dict[str, list[str]] = dspy.OutputField( desc="mapping from section headings to subheadings" )class DraftSection(dspy.Signature): """Draft a top-level section of an article.""" topic: str = dspy.InputField() section_heading: str = dspy.InputField() section_subheadings: list[str] = dspy.InputField() content: str = dspy.OutputField(desc="markdown-formatted section")class DraftArticle(dspy.Module): def __init__(self): self.build_outline = dspy.ChainOfThought(Outline) self.draft_section = dspy.ChainOfThought(DraftSection) def forward(self, topic): outline = self.build_outline(topic=topic) sections = [] for heading, subheadings in outline.section_subheadings.items(): section, subheadings = ( f"## {heading}", [f"### {subheading}" for subheading in subheadings], ) section = self.draft_section( topic=outline.title, section_heading=section, section_subheadings=subheadings, ) sections.append(section.content) return dspy.Prediction(title=outline.title, sections=sections)draft_article = DraftArticle()article = draft_article(topic="World Cup 2002")
Weave also captures traces for DSPy optimizers and evaluation calls, which you can use to improve and evaluate your DSPy program’s performance on a development set. Replace [YOUR-OPENAI-API-KEY] with your OpenAI API key and [YOUR-WANDB-PROJECT-NAME] with your Weights & Biases project name.
