Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup Playwright tests, misc fixes #4

Merged
merged 31 commits into from
Aug 28, 2024
Merged

Setup Playwright tests, misc fixes #4

merged 31 commits into from
Aug 28, 2024

Conversation

vincerubinetti
Copy link
Collaborator

@vincerubinetti vincerubinetti commented Aug 13, 2024

Note that the diff here is a pain because I had to do a bulk chmod due to a playwright setup command messing with a bunch of file permissions, and couldn't find a way to reset the permissions to their defaults (pre-playwright-command) without going through one file at a time.

  • pull in various improvements from Review comments, bug fixes, tweaks krishnanlab/geneplexus-app-v2#37 and Misc frontend fixes krishnanlab/geneplexus-app-v2#38
  • update test workflow: fix trigger, update action versions, update debugging steps, add e2e playwright testing with caching
  • setup playwright for e2e and accessibility testing
  • add status indicator to analysis card component
  • enable msw network request mocking
  • don't allow empty "lookup analysis"
  • install and setup axe-core (library that powers chrome dev tools' lighthouse accessibility check)
  • check all major pages of app (so far) for accessibility issues
  • add basic e2e test for demonstration

Copy link

netlify bot commented Aug 13, 2024

Deploy Preview for molevolvr ready!

Name Link
🔨 Latest commit ff5b1a8
🔍 Latest deploy log https://app.netlify.com/sites/molevolvr/deploys/66cf5eb0ca2cc90008668819
😎 Deploy Preview https://deploy-preview-4--molevolvr.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@vincerubinetti
Copy link
Collaborator Author

I've just added a commit that pulls in updates from krishnanlab/geneplexus-app-v2#38, which includes unrelated fixes, but also some accessibility fixes that would affect the Axe testing here. It also adds /testbed to the routes that are tested with Axe.

@vincerubinetti vincerubinetti requested review from falquaddoomi and removed request for falquaddoomi August 21, 2024 17:46
@falquaddoomi
Copy link
Contributor

Hey, do you expect people to run these tests you've added locally, or are they only for the GitHub action? If I can run them locally, how do I do that? I'm getting the following when I try to do what seems obvious from reading the scripts you've defined in package.json:

[ faisala@gleipnir molevolvr2.0/app ] % bun run install-playwright
$ bunx playwright install --with-deps
[ faisala@gleipnir molevolvr2.0/app ] % bun run test:e2e                  
[WebServer] $ vite --port 1234

SyntaxError: Unexpected token 'with'

SyntaxError: Unexpected token 'with'

Error: No tests found

To open last HTML report run:

  npx playwright show-report

error: script "test:e2e" exited with code 1
[ faisala@gleipnir molevolvr2.0/app ] % bun run test    
$ bun run test:lint && bun run test:types && bun run test:e2e
$ eslint . && prettier **/*.{tsx,ts,css,html,md,json,yaml} --check --no-error-on-unmatched-pattern

Oops! Something went wrong! :(

ESLint: 9.8.0

ConfigError: Config (unnamed): Key "rules": Key "constructor-super": structuredClone is not defined
    at rethrowConfigError (/Users/faisala/Checkouts/chai/jravilab/molevolvr2.0/app/node_modules/@eslint/config-array/dist/cjs/index.cjs:303:8)
    at /Users/faisala/Checkouts/chai/jravilab/molevolvr2.0/app/node_modules/@eslint/config-array/dist/cjs/index.cjs:1098:5
    at Array.reduce (<anonymous>)
    at FlatConfigArray.getConfigWithStatus (/Users/faisala/Checkouts/chai/jravilab/molevolvr2.0/app/node_modules/@eslint/config-array/dist/cjs/index.cjs:1091:43)
    at FlatConfigArray.getConfig (/Users/faisala/Checkouts/chai/jravilab/molevolvr2.0/app/node_modules/@eslint/config-array/dist/cjs/index.cjs:1120:15)
    at /Users/faisala/Checkouts/chai/jravilab/molevolvr2.0/app/node_modules/eslint/lib/eslint/eslint-helpers.js:346:56
    at Array.reduce (<anonymous>)
    at /Users/faisala/Checkouts/chai/jravilab/molevolvr2.0/app/node_modules/eslint/lib/eslint/eslint-helpers.js:333:36
    at /Users/faisala/Checkouts/chai/jravilab/molevolvr2.0/app/node_modules/eslint/lib/eslint/eslint-helpers.js:296:32
    at Object.isAppliedFilter (/Users/faisala/Checkouts/chai/jravilab/molevolvr2.0/app/node_modules/@nodelib/fs.walk/out/readers/common.js:12:31)
[ faisala@gleipnir molevolvr2.0/app ] % 

@vincerubinetti
Copy link
Collaborator Author

Yes you should be able to run them locally. Do you have node v22 installed?

@falquaddoomi
Copy link
Contributor

falquaddoomi commented Aug 23, 2024

Aha; I wasn't using node v22 at the time, but I switched to v22.7.0, wiped out node_modules, reinstalled using bun install, and ran bun run test with the following (successful?) output:

% bun run test
$ bun run test:lint && bun run test:types && bun run test:e2e
$ eslint . && prettier **/*.{tsx,ts,css,html,md,json,yaml} --check --no-error-on-unmatched-pattern
Checking formatting...
All matched files use Prettier code style!
$ tsc
[WebServer] $ vite --port 1234

(node:9206) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)

Running 27 tests using 5 workers
(node:9279) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9282) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9278) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9281) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9280) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9455) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9456) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9485) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9486) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9487) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
[webkit] › home.spec.ts:4:1 › Example analyses show
(node:9557) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9556) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9579) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9584) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
  16 skipped
  11 passed (9.6s)

To open last HTML report run:

  npx playwright show-report

Perhaps you should specify the version of node that's required via https://docs.npmjs.com/cli/v10/configuring-npm/package-json#engines, or some other way like in the README?

Also, adding something to your PR description that specifies what you'd like me to test and how to do it would be nice.

@vincerubinetti
Copy link
Collaborator Author

Sorry, I thought that was already in the readme. I've updated the readme to add some clarifications.

Also, adding something to your PR description that specifies what you'd like me to test and how to do it would be nice.

What I can do from now on is make a comment near sections of the diff that are particularly complex/error-prone/high-consequence. I did this for monarch and it helped them ignore irrelevant changes. I will do this now for this PR.

* is element covering anything "important" (above anything besides a
* "background" element)
*/
export const isCovering = (
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Util func that checks a grid of points under an element.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting solution, and it seems to work well.

I'm not suggesting changing it, but if you're interested you might consider trying the intersection observer API, which seems to have good support from all the major browsers. Basically it allows you to register a target element that gets checked against all the elements under some node in the DOM (by default it's the browser window, i.e. all elements) for intersections and fires a callback when an intersection occurs. Supposedly it's more performant than checking yourself, but since you're just sampling a few points I think your solution is fine.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if ResizeObserver can be used in this manner, because although you can specify a root element other than the viewport, it seems that the root must be an ancestor. That is, the element being covered (let's say an svg chart element) would have to be a child of the table of contents.

Idk if there is a browser API I could use to make this less kludgy. I could do a simple bbox overlap check, but then I'd have to do it against all the elements in the viewport. It might end up being faster than this method though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I thought it could check intersections between the target and any child of the root element, but I guess if it's just the root vs. the target then it's not very useful.

I'm curious if iterating over all the visible elements and doing bbox checks would be faster than elementsFromPoint() (no idea how that function works under the hood), but IMHO what you have is good. I'd only explore it if you're curious and have time, or if you run into performance issues.

Copy link
Collaborator Author

@vincerubinetti vincerubinetti Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I switched to the bbox method. I found the elementsFromPoint method to range from < 1ms to 10ms, and the bbox method to always be < 1ms. Since this function is run in a very debounced fashion, this should never really matter, but it could matter if we needed to call it on every scroll event in the future. The bbox method, with the default "important" element selector, should definitely do less work on average than elementsFromPoint.

Bbox method, at least with the default section > * selector, will be less "accurate", in the sense that it's based on DOM hierarchy and not actual page layout, which are not always the same. For example:

<section>
  <!-- narrow box in center of screen. gets checked by util func by default, b/c is a direct descendant of section -->
  <div style="width: 100px">
    <!-- image that is "popped out" of regular document flow, positioned outside its parent div, floating on the left side of the screen. not checked by the util func by default, but could still be under the floating table of contents panel -->
    <img style="position: fixed; left: 0"/>
  </div>
</section>

This is unlikely HTML/CSS layout though. If something needs to be popped out like this, its likely to be done at the top level (the div in the example).

If it becomes a problem, could change the selector to section * (would have to check many more elements, becoming possibly slower than the elementsFromPoint method).

@falquaddoomi
Copy link
Contributor

What I can do from now on is make a comment near sections of the diff that are particularly complex/error-prone/high-consequence. I did this for monarch and it helped them ignore irrelevant changes. I will do this now for this PR.

Thanks; I realize this took effort, and it is helpful. What I meant, though, was just some simple directions for things you'd want tried out by a reviewer, e.g. "Run bun run test and see that the tests complete" or "try out feature X in widget Y". I'm not much of a frontend dev anymore, and just simple things like "use node version X" or "run this command to launch the test suite" would save me a little time.

You're of course welcome to continue to describe the significant code changes in your PR and why you made them, but I don't expect you to.

@vincerubinetti
Copy link
Collaborator Author

Gotcha. I'll still try to do the comment-by-comment when possible, still seems helpful on large PRs so you can sort of skip over lower risk/impact things. And if not I'll have a paragraph or so summarizing the salient things.

@falquaddoomi
Copy link
Contributor

Just FYI, I've looked through the changes and I still approve it all. Feel free to merge.

@vincerubinetti vincerubinetti merged commit cff0650 into main Aug 28, 2024
4 checks passed
@vincerubinetti vincerubinetti deleted the playwright branch August 28, 2024 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants