1. Build a dataset
Open the task’s Datasets page and choose the traffic that best represents the behavior you want to preserve. Review a sample before training. Remove bad examples and make sure the dataset covers the situations customers actually encounter.Datasets Quick Start
Create and review your first dataset in Studio.
2. Improve the answers when needed
Your current model’s answers can be used directly as training targets. If they are inconsistent or you want to teach better behavior, relabel the dataset with a stronger model. Relabeling creates an alternative set of answers without replacing the originals, so you can inspect both and choose which one to train on.3. Train several candidates
Choose the models you want to train. A small sweep is usually more useful than a single run because the best model for one task is not always the largest model. Studio handles the training workflow and makes finished models available for comparison and deployment.Fine-tuning Quick Start
Start a training run from a ready dataset.
4. Compare quality
Create an evaluation and compare every candidate with the answers you want to preserve. Look at both the overall score and individual examples to understand where a model improves or regresses. Do not deploy from an incomplete evaluation. Resolve failed comparisons and make sure the result represents the behavior that matters to your product.Evaluations Quick Start
Run a head-to-head comparison in Studio.
5. Deploy the winner safely
Add the best model to task routing. Start with part of the traffic when you want a production canary, then increase its share as confidence grows. Keep the previous model in the routing configuration at zero weight after a full rollout. This makes rollback quick without sending it new traffic.6. Automate the next cycle
After running the workflow manually once, configure Automation. It can prepare the next dataset, train several candidates, compare them, and deploy a winner only when it meets the threshold you choose.API: Agent workflow
API: Agent workflow
Give the agent the task alias, W&B entity, desired dataset window, candidate base models, judge model, and rollout threshold. Then direct it to the copyable requests in:Ask it to wait for each asynchronous resource to reach its successful terminal state before starting the dependent step. Automation combines the full loop in one configuration.