feat: eval mode for systematic WebMCP tool testing

[madmath]

Summary

• Adds an eval system that lets users upload JSON eval suites in Settings and run them from the sidebar via /eval
• Each scenario sends a prompt to the agent, checks which tools were called (expected vs forbidden), and optionally uses LLM-as-judge to verify post-conditions
• Full batch auto-run with progress bar, per-scenario pass/fail rows (showing the prompt at a glance), and a summary dashboard at the end
• Escape key aborts mid-run

Eval Suite Format

{
  "name": "Mock Shop WebMCP Eval",
  "baseUrl": "https://demostore.mock.shop",
  "scenarios": [
    {
      "id": "search-basic",
      "prompt": "Search for sneakers",
      "startPage": "/",
      "expectations": {
        "toolCalls": ["search_store"],
        "forbiddenToolCalls": ["add_to_cart"],
        "postConditions": "The agent called search_store with a query related to sneakers."
      },
      "tags": ["search", "single-tool"]
    }
  ]
}

Scoring

• Tool call score: (expected tools called / total expected) minus 0.5 penalty for any forbidden tool called
• Judge score: LLM evaluates post-conditions (0–1)
• Combined: 40% tool calls + 60% judge (or 100% of whichever is present)

New files (5)

File	Purpose
src/lib/eval/types.ts	Type definitions for suites, scenarios, scoring
src/lib/eval/scoring.ts	Pure tool-call matching + combined score functions
src/lib/eval/runner.ts	Sequential batch orchestrator with navigation + judge
src/sidebar/EvalReportBox.ts	Progress bar, scenario rows with prompt, summary UI
src/options/eval-suites.ts	Import/delete eval suite JSON files in Settings

Modified files (11)

• Storage — evalSuites field + CRUD methods on ConfigStorage
• Types — EvalJudgeMessage added to ExtensionMessage union
• AIClient — generateTextForAgent() for non-streaming judge calls
• Background — EVAL_JUDGE message handler
• Builtins — /eval registered so /help lists it
• Sidebar — /eval intercept, startEvalMode(), Escape to abort
• Options page — "Eval Suites" section in HTML, init call, CSS

Test plan

• Build passes (npm run build — typecheck + lint + vite)
• All 447 existing tests pass
• Settings → Import Suite → upload a JSON file → card appears with name/count → delete works
• Sidebar → /eval with no suites → shows error message
• Sidebar → /eval after importing suite → progress bar advances, scenarios run, summary appears
• Escape mid-run → aborts cleanly
• /help lists /eval

🤖 Generated with Claude Code