Skip to content

chore(claude): enable provided playwright test agents#911

Open
carlosthe19916 wants to merge 1 commit intoguacsec:mainfrom
carlosthe19916:claude/playwright-agents
Open

chore(claude): enable provided playwright test agents#911
carlosthe19916 wants to merge 1 commit intoguacsec:mainfrom
carlosthe19916:claude/playwright-agents

Conversation

@carlosthe19916
Copy link
Collaborator

@carlosthe19916 carlosthe19916 commented Feb 2, 2026

Addresses: #909

Enabling Playwright provided Test Agents as documented in the official webpage https://playwright.dev/docs/test-agents

This PR is the result of executing npx playwright init-agents --loop=claude with no customization to the default agent definition.

Notice that this PR also includes a new MCP server

"playwright-test": {
      "command": "npx",
      "args": [
        "playwright",
        "run-test-mcp-server"
      ]
    }

which is different from

"playwright": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "@playwright/mcp@latest"
      ],
      "env": {}
    },

Summary by Sourcery

Integrate Playwright-provided test agents for planning, generating, and healing browser tests, and wire them up via a dedicated Playwright test MCP server.

New Features:

  • Add a Playwright test generator agent for creating Playwright test files from high-level test plans.
  • Add a Playwright test planner agent for exploring web apps and producing structured test plans.
  • Add a Playwright test healer agent for diagnosing and fixing failing Playwright tests using Playwright MCP tools.

Enhancements:

  • Register a new playwright-test MCP server configuration to back the Playwright testing agents.

Signed-off-by: Carlos Feria <2582866+carlosthe19916@users.noreply.github.com>
@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Feb 2, 2026

Reviewer's Guide

Adds Playwright-provided Claude test agents (planner, generator, healer) and wires them to a new playwright-test MCP server configuration, enabling automated planning, generation, and healing of Playwright tests via Claude.

File-Level Changes

Change Details Files
Configure new Playwright Test MCP server for Claude separate from existing generic Playwright MCP server.
  • Add playwright-test MCP server entry that runs npx playwright run-test-mcp-server.
  • Keep existing playwright MCP entry that runs @playwright/mcp@latest, clarifying there are now two distinct Playwright-related MCP endpoints.
.mcp.json
Introduce a Claude agent for generating Playwright tests from existing test plans and seed files using the new Playwright Test MCP tools.
  • Create playwright-test-generator agent definition with detailed role instructions for turning markdown test plans into single-file Playwright tests.
  • Wire generator-focused MCP tools (generator_setup_page, generator_read_log, generator_write_test, and browser_* tools) into the agent.
  • Document example input/output workflow for converting a sample test plan into a generated test file with best-practice structure.
.claude/agents/playwright-test-generator.md
Introduce a Claude agent for planning comprehensive Playwright test suites by exploring a web app via Playwright browser tools.
  • Create playwright-test-planner agent definition with responsibilities for exploring the UI, mapping user flows, and drafting test scenarios.
  • Attach planner-specific tools (planner_setup_page, planner_save_plan, and browser_* tools) and describe constraints around screenshots and exploration depth.
  • Define required structure and quality standards for saved markdown test plans including scenario breakdown, steps, and negative tests.
.claude/agents/playwright-test-planner.md
Introduce a Claude agent for debugging and healing failing Playwright tests using Playwright Test MCP debugging tools.
  • Create playwright-test-healer agent definition focused on running, debugging, and fixing failing tests via test_run, test_debug, test_list, and browser inspection tools.
  • Lay out a step-by-step healing workflow: run tests, debug failures, inspect context, identify root cause, edit tests, and re-run until green.
  • Codify best practices such as robust selectors, avoiding discouraged APIs, documenting fixes, and using test.fixme() when the app is broken rather than the test.
.claude/agents/playwright-test-healer.md

Possibly linked issues

  • #claude: The PR implements the issue’s decision by enabling Playwright-provided test agents and configuring the new MCP server.

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The description field in playwright-test-generator frontmatter is extremely long and contains example markup inline; consider moving the example into the body text so the description stays concise and the YAML remains more readable and maintainable.
  • The hard‑coded tools lists for the three new agents are quite verbose and tightly coupled to the MCP server API; consider adding a brief comment or central reference near .mcp.json to keep these in sync if Playwright’s MCP tool names change (e.g., when regenerating via init-agents).
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `description` field in `playwright-test-generator` frontmatter is extremely long and contains example markup inline; consider moving the example into the body text so the description stays concise and the YAML remains more readable and maintainable.
- The hard‑coded `tools` lists for the three new agents are quite verbose and tightly coupled to the MCP server API; consider adding a brief comment or central reference near `.mcp.json` to keep these in sync if Playwright’s MCP tool names change (e.g., when regenerating via `init-agents`).

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@carlosthe19916 carlosthe19916 linked an issue Feb 2, 2026 that may be closed by this pull request
@codecov
Copy link

codecov bot commented Feb 2, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.91%. Comparing base (9e82125) to head (ab2a6c6).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #911      +/-   ##
==========================================
- Coverage   64.40%   63.91%   -0.49%     
==========================================
  Files         187      187              
  Lines        3295     3295              
  Branches      746      746              
==========================================
- Hits         2122     2106      -16     
- Misses        875      891      +16     
  Partials      298      298              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

claude: use Playwright provided test agents

1 participant