> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluations Quick Start

> Run a head-to-head evaluation between your reference output and one or more candidate models.

Head-to-head evaluation is the best starting point for open-ended tasks where exact text equality would be too strict.

<Note>
  Before you begin, create a dataset with validation rows and add every fine-tune or provider model you want to compare.
</Note>

<Steps>
  <Step title="Open the dataset's Evaluate tab">
    Select **New evaluation**, choose **Head to Head**, and give the evaluation a descriptive name.
  </Step>

  <Step title="Choose the reference">
    Select original or relabeled dataset output as the primary participant. Candidates will be scored against this desired behavior.
  </Step>

  <Step title="Add candidate models">
    Add fine-tunes from the dataset first, then any base or provider models you want to compare. Fine-tunes are grouped by base model in the selector.
  </Step>

  <Step title="Configure the judge">
    Select a capable judge model and write instructions that define what a better answer means for this task. Start with 200 validation rows when enough are available.
  </Step>

  <Step title="Run and inspect">
    Start the evaluation. Scores update while rows finish. Open individual rows to read the judge explanation and use the failure indicator to inspect provider or parsing errors.
  </Step>

  <Step title="Decide whether to deploy">
    A 50% score means parity with the reference: wins count as 1, ties as 0.5, and losses as 0. Promote only after the completed score and per-row behavior meet your threshold.
  </Step>
</Steps>

Failed cases retry automatically up to three attempts. Once the run settles, **Retry failed cases** requeues only failures and preserves completed results.

<Accordion title="API: Create an evaluation (POST /evals)">
  The first participant is the primary. Replace the candidate model reference with the artifact or provider model you want to evaluate:

  ```bash theme={"system"}
  curl --request POST \
    --url "https://distillation.training.wandb.ai/v1/evals" \
    --header "Authorization: Bearer $WANDB_API_KEY" \
    --header "Wandb-Entity: your-team" \
    --header "Content-Type: application/json" \
    --data "{
      \"name\": \"ticket-classifier-comparison\",
      \"dataset_id\": \"$DATASET_ID\",
      \"spec\": {
        \"type\": \"h2h_judge\",
        \"judge_model_ref\": \"openai/gpt-5.6-sol\",
        \"judge_prompt\": \"Choose the more correct and helpful answer. Return a tie when they are equally good.\"
      },
      \"participants\": [
        {\"kind\": \"original\"},
        {\"kind\": \"model\", \"model_ref\": \"wandb-inference/your-fine-tuned-model\"}
      ],
      \"sample_size\": 200
    }"
  ```

  The response contains the evaluation ID. Poll `GET /evals/{evalId}` until the evaluation settles; see [Create an evaluation](/model-distillation/reference/management).
</Accordion>