|
| 1 | +# Proposal: Migrate UI E2E Tests from Robot + Selenium to Playwright |
| 2 | + |
| 3 | +Author: Prasanth Baskar/[bupd](https://github.com/bupd) |
| 4 | + |
| 5 | +Tracking: [goharbor/harbor#22134](https://github.com/goharbor/harbor/issues/22134) |
| 6 | + |
| 7 | +Timeline: ~3 months (one release cycle v2.15 for Phase 1) |
| 8 | + |
| 9 | +## Summary |
| 10 | + |
| 11 | +Migrate Harbor’s UI end-to-end (E2E) tests from Robot Framework + Selenium to Playwright. The goal is to improve test reliability, reduce technical debt, and enable more contributors to participate in test development. Migration will be incremental; existing Robot testcases will remain until equivalent Playwright coverage exists. |
| 12 | + |
| 13 | +## Motivation |
| 14 | + |
| 15 | +### Current challenges |
| 16 | +- Robot testcases rely heavily on XPath and are fragile. |
| 17 | +- Difficult to edit or extend, discouraging contributions. |
| 18 | +- Poor developer experience compared to modern tooling. |
| 19 | + |
| 20 | +### Why Playwright |
| 21 | +- Rich debugging and tracing. |
| 22 | +- Strong community adoption. |
| 23 | +- Accessible for both frontend and backend contributors. |
| 24 | + |
| 25 | +## Goals |
| 26 | +- Replace UI/XPath-based Robot tests with Playwright. |
| 27 | +- Migrate testcases incrementally by multiple PRs in a non-blocking manner. |
| 28 | +- Maintain coverage during migration. |
| 29 | +- Improve contributor experience and onboarding. |
| 30 | +- Retire Robot tests after achieving 1:1 coverage. |
| 31 | + |
| 32 | +## Non-goals |
| 33 | +- Immediate deletion of Robot tests. |
| 34 | +- Creating a single PR including all changes. |
| 35 | + |
| 36 | +> Note: Robot tests will eventually be deleted, but this will occur in a later phase, not during Phase 1. |
| 37 | +
|
| 38 | +## Proposal |
| 39 | + |
| 40 | +### Phase 1 – Migration (This Release Cycle) |
| 41 | +(Setup): Add a GitHub Action to run Playwright tests |
| 42 | +- Start with nightly UI Robot tests. |
| 43 | +- Incrementally migrate testcases to Playwright in multiple PRs. |
| 44 | +- Contributors submit migration in multiple small PRs. |
| 45 | +- Robot tests remain until Playwright test cases reach 1:1 parity. |
| 46 | + |
| 47 | +### Phase 2 – Retirement (Next Release Cycle) |
| 48 | +- Plan to retire Robot UI testcases after migrating all testcases to Playwright. |
| 49 | + |
| 50 | +## Requirements |
| 51 | +- Infra Needs: access to AWS Harbor environments for setup (servers, webhooks). |
| 52 | +- Contributors: Community-driven migration. |
| 53 | + |
| 54 | +## Checklist for Phase 1 |
| 55 | +- [ ] Add GitHub Action to run Playwright pipeline. |
| 56 | +- [ ] Configure CI to run both Robot and Playwright tests. |
| 57 | +- [ ] Configure CI to run Playwright tests on PRs. |
| 58 | +- [ ] Break down test suite into smaller subtests. |
| 59 | +- [ ] Migrate Robot UI tests to Playwright in multiple PRs. |
| 60 | +- [ ] Ensure main branch stays green (non-blocking, incremental merges). |
| 61 | +- [ ] Achieve full 1:1 parity with Robot UI tests. |
| 62 | +- [ ] Upgrade Angular frontend to latest version. |
| 63 | +- [ ] Ensure all UI E2E cases pass in CI with latest Angular frontend. |
| 64 | + |
| 65 | +## Outcome |
| 66 | + |
| 67 | +- Reliable UI tests. |
| 68 | +- Better developer experience. |
| 69 | +- Easier contributions. |
| 70 | +- Reduced technical debt by retiring Robot + Selenium. |
0 commit comments