-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: Implement end-to-end testing with Playwright #420
Commits on Aug 28, 2024
-
test: Implement end-to-end testing with Playwright
This is an in-progress cleaned version of the final PR (numberscope#361) of the Delft student user interface project, which it supersedes. Comments from the original PR that remain relevant: * The end-to-end tests run using Firefox and Chromium. * New tests are in the e2e folder. * The tests often depend on specific classes and IDs, so they may need to be updated upon changes to Numberscope. * The tests can be executed as the following npm script: `npm run test:e2e` * An interactive testing UI and debugger can be executed as the following npm script: `npm run test:e2e:ui` Caveats concerning trying this cleaned PR and its status: * Make certain to run `npm install` after pulling this PR. * Many of the tests do not yet pass, perhaps because of the "specific classes and IDs" point mentioned above and the fact that ui2 has diverged significantly from the Delft PR series. * Tests are not yet run automatically prior to commit. * I do not think there are any image tests yet, we need to try to add them. * Tests are not yet performed in the continuous integration checks to be run on GitHub; they should be.
Configuration menu - View commit details
-
Copy full SHA for 1b6aea7 - Browse repository at this point
Copy the full SHA 1b6aea7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1249a46 - Browse repository at this point
Copy the full SHA 1249a46View commit details -
Configuration menu - View commit details
-
Copy full SHA for fd32bf9 - Browse repository at this point
Copy the full SHA fd32bf9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3b81741 - Browse repository at this point
Copy the full SHA 3b81741View commit details -
Configuration menu - View commit details
-
Copy full SHA for 44869dc - Browse repository at this point
Copy the full SHA 44869dcView commit details
Commits on Aug 29, 2024
-
The strong desire for image tests led to a cascade of changes in this commit, mostly driven by the need to have reproducible images: - Removes all use of `sketch.noLoop()` and `sketch.loop()` in favor of the previously existing `stop()` and `continue()` visualizer methods, to allow: - Adds a `frames=NNN` query parameter to URLs to set the maximum number of frames a visualization may draw - Switches from the "static" instance of mathjs to a "dynamic" one, to allow its random number configuration to be controlled. In conjunction with this, moves all math functions into a single math module, as extensions of mathjs. - Removes all use of `Math.random()` in favor of the mathjs random generator - Adds a `randomSeed=AAAA` query parameter to URLs to make the mathjs random generator reproducible. - Documents all of the above changes.
Configuration menu - View commit details
-
Copy full SHA for b98114a - Browse repository at this point
Copy the full SHA b98114aView commit details
Commits on Aug 30, 2024
-
doc: describe running and creating code tests
Also adds new tests for `src/shared/defineFeatured.ts` and corrects the documentation extraction facility for the package manager scripts. Resolves numberscope#25. Resolves numberscope#73. Resolves numberscope#246.
Configuration menu - View commit details
-
Copy full SHA for 11a1def - Browse repository at this point
Copy the full SHA 11a1defView commit details -
test: Test that the caching mechanism won't double-calculate
In other words, it should never call calculate twice for the same index. This is tested by 10K random accesses to indices less than 1M, followed by accessing the first 10K entries, followed by accessing the last 10K entries. Hopefully that should suffice. Resolves numberscope#54.
Configuration menu - View commit details
-
Copy full SHA for c988513 - Browse repository at this point
Copy the full SHA c988513View commit details -
fix: Prevent ModFill from hanging on extremely large input
This is an initial pass at addressing numberscope#113. Note, however, that ModFill is not reporting to the person doing visualization that it is running with different parameter values than shown. So that still must be done, but for that part we will need a resolution to numberscope#112, which will be a sufficiently involve change that we should leave it to a spearate PR from this numberscope#420.
Configuration menu - View commit details
-
Copy full SHA for de5f16c - Browse repository at this point
Copy the full SHA de5f16cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3ca1474 - Browse repository at this point
Copy the full SHA 3ca1474View commit details -
Configuration menu - View commit details
-
Copy full SHA for d8ccce2 - Browse repository at this point
Copy the full SHA d8ccce2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5cccf8c - Browse repository at this point
Copy the full SHA 5cccf8cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 571390a - Browse repository at this point
Copy the full SHA 571390aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3d1d6fa - Browse repository at this point
Copy the full SHA 3d1d6faView commit details
Commits on Sep 5, 2024
-
test: Add OEIS Sequence-Visualizer test grid
Use a list of "stressful sequences" to perform at least two image tests on each visualizer. As this change uncovered several existing errors, there are numerous other changes in this commit to return to a state of all tests passing. Here is an enumeration of other changes, in no particular order: * Change husky pre-commit actions so that the end-to-end tests are not run if there is an existing successful test since any files in the project last changed. * On most image tests, snapshot only the visualizer canvas. * Make 'axios' a runtime dependency, rather than just development, as it is used when obtaining OEIS sequence values in the running frontend. * So many other changes are made to Histogram.ts that this PR also adds hatching to show the elements with unknown factorization. Adds the "p5.brush" package to draw the hatching. Since this necessitates WebGL, adds a new 'P5GLVisualizer' base class for visualizers that wish to use a WebGL canvas. * Updates several dependencies to latest to make sure their out-of-dateness was not contributing to test issues. * Switches to the (ES) "module" import system from "commonjs" to ensure that was not contributing to test issues. Resolves numberscope#294. * Specifies font locations in a way that vite understands how to relocate in both the 'dev' and 'build' versions of the app. * Properly marks both factor cache and value cache as empty at the initialization of an OEIS sequence. * Allows ±Infinity as valid values of INTEGER param fields for convenience. * Allows specimenQuery to take the output of parseSpecimenQuery to recreate the same specimen query as was parsed, for the sake of testing. * Turns off the browser default context menu on visualizers, which wasn't particularly useful and was interfering with the UI for some visualizers. * Fixes typos in Differences visualizer (this.first -> sequence.first) * Makes FactorHistogram visualizer a p5 WebGL-based visualizer, and adds hatching to the "0 factors" bar to indicate the terms with unknown factorizations. * Uses the WebGL default controls to implement panning, zooming, and (rather uselessly but somewhat spectacularly) rotating the plot in three dimensions for the FactorHistogram visualizer. * On FactorHistogram's first pass over the sequence data, collects the counts for each possible number of factors, as opposed to the number of factors for each entry, to avoid a possibly disastrously long loop on the second pass that accumulates bin counts. This also streamlines the computation of the largest number of factors in the data. * Factors out repeated code in FactorHistogram, for labeling bars and displaying text. * Fits the hover box in FactorHistogram to the text to be displayed. * FactorHistogram displays a temporary message if factoring is taking a long time. * Moves the bar labels of FactorHistogram just under their respective bars, to make sure they are visible. * Selects y-axis ticks at round numbers for FactorHistogram, and moves the tick labels closer to their ticks. * Prevents FactorHistogram from looping except when there is mouse activity. * Prevents ModFill from freezing up if too large a "mod dimension" is chosen. Note TODO: display a warning when a smaller mod dimension than requested is actually used. * Prevents Vue's reactivity system from attempting to modify the behavior of p5 sketches, using the Vue `markRaw` method. This change prevents some instances of infinite loops caused by cascading change notifications. * Tightens up the typing of P5Visualizer so that it is possible to derive another visualizer base class P5GLVisualizer from it. Also splits up the inhabit method so that P5GLVisualizer can modify it as needed. * Allow negative start indices for Show Factors visualizer. * Show at most 100 terms in Show Factors (no room on screen for more than that). * Updates TypeScript target versions of JavaScript Resolves numberscope#226.
Configuration menu - View commit details
-
Copy full SHA for 044a054 - Browse repository at this point
Copy the full SHA 044a054View commit details -
Configuration menu - View commit details
-
Copy full SHA for f1f7d30 - Browse repository at this point
Copy the full SHA f1f7d30View commit details
Commits on Sep 6, 2024
-
Configuration menu - View commit details
-
Copy full SHA for bb45388 - Browse repository at this point
Copy the full SHA bb45388View commit details -
Configuration menu - View commit details
-
Copy full SHA for 95d8f57 - Browse repository at this point
Copy the full SHA 95d8f57View commit details
Commits on Sep 9, 2024
-
feat: Uniform sequence bounds controls
All params for controlling which terms of a sequence will be used in the visualization are removed from individual visualizers. Instead, there are uniform params in the Sequence classes themselves. Resolves numberscope#411. In this implementation, several other changes are necessary and/or expedient to make. In particular, the type of sequence indices is changed to bigint and bounds to ExtendedBigint (the union of bigint and ±Infinity), in preparation for addressing numberscope#455. Additional changes include: * Removal of SequenceDefault class, as everything now derives from Cached. * More care in caching of OEIS sequences, to provide some help with numberscope#459. * Individual-parameter validation functions now take a validation status to update based on the finding. This refactor avoids a frequent need to merge status objects. * Individual-parameter validation functions are called in a context of `this` set to the Paramable object, so that other data from the paramable can be accessed. * New `math` functions for dealing with ExtendedBigints. * NumberGlyph visualizer now tries to display as many terms as are available and will fit on the screen, except in case there are infinitely many terms available, in which case it still defaults to 64 terms. This change is needed so that the general length parameter for Sequences would affect what NumberGlyph displayed. * Puts reactivation of "known" OEIS sequences into a function rather than at top level of sequences.ts so that the timing of when it occurs can be controlled. * Renames property `_size` of P5Visualizer to `size` as multiple derived classes seem to need this value (i.e., not just used internally in the base class). * Makes sure the p5 loop is restarted if need be every time `.show()` is called on a visualizer. * Temporarily disables A000521 transversal tests until we can get incremental OEIS loading to work. * Adds a test for starting a visualization deep into a sequence.
Configuration menu - View commit details
-
Copy full SHA for d811eba - Browse repository at this point
Copy the full SHA d811ebaView commit details
Commits on Sep 10, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 33386c0 - Browse repository at this point
Copy the full SHA 33386c0View commit details
Commits on Sep 17, 2024
-
test: First working e2e tests with text and transversal
Unfortunately, due to the intricacies of end-to-end testing with image comparison, this is a very large commit. It introduces running tests in a Docker container for the sake of reproducibility. (So note that henceforth you must have docker installed and running on your machine to perform testing, and hence to make a commit.) However, it turns out not all tests can be run in a Docker container -- Firefox is not able to supply a WebGL context inside of Docker. Hence, some tests still need to be run directly on the host machine. To deal with all this, I felt it necessary to introduce the "make" tool. In particular, creating the necessary Docker image is slow, and I didn't want to have to repeat that except when truly necessary. That, in turn, means there are now many more configuration files, as well as auxiliary files created by the Makefile. The directory tree was becoming hopelessly cluttered. To keep things straight in my head, I felt it was really necessary to reorganize the directory structure. The biggest change is to put as many of the configuration files as possible in a new etc/ directory (the old-school unix name for where to put such things). I managed to get almost everything put in there. The one major holdout is the tsconfig files; I just couldn't find a way to get TypeScript to run without its config files at the top level of the project. Slightly annoying, but then, there are so many more annoying things about TypeScript. Anyhow, accomplishing the configuration file rearrangement resulted in updating eslint. Unfortunately, it went through a _major_ rewrite from version 8 to 9, and prettier-eslint has not been updated to work with the new version. I tried to work on that update myself, but prettier-eslint is incredibly intricate in the way that it handles configurations because it is trying to provide a great deal of flexibility and power, and it is trying to infer prettier configuration from eslint configuration (and maybe even vice-versa as well, I'm not sure). Since we were only using a tiny fraction of that power, it turned out to be significantly easier to just replace prettier-eslint with a custom script tools/prettiest.js that runs first prettier, then eslint. It also meant we could get rid of the lint-staged tool, so there is that. (While I was at it, I ran the "depcheck" tool and got rid of some other unneeded cruft that had accumulated.) But now it all works, and as a side benefit since just about every npm script now runs through make, you should never need to worry about running install before dev or build before preview, etc. Make keeps track of what depends on what and whether/how to redo the prerequisites before taking the requested action. Even better, if you just ran successful end-to-end tests, which take a while now, then `git commit` will not have to re-run them. Those are the major points. In addition, there are a number of minor changes resulting from errors uncovered in getting all the tests to work, etc: - Reordering of element attributes and other reformatting in vue components, because with the lint updates and reconfiguration, it now seems significantly stricter about vue templates. - First pass at an updated User Guide for ui2; it definitely needs more work. - Some other more minor documentation updates. - Updates tools/editor/autoformat.el so that emacs can use the new "prettiest" tool for formatting (even though I know I am likely the only one in the world who will ever use that).
Configuration menu - View commit details
-
Copy full SHA for 63ddc6c - Browse repository at this point
Copy the full SHA 63ddc6cView commit details
Commits on Sep 18, 2024
-
test: Add skipped tests reflecting known shortcomings
As per Numberscope discussion today, this is the last missing element of the end-to-end testing PR. With these skipped tests, we are flagging that there are known concerns we want to resolve at least by beta, if not alpha. Also fixes a small gap in the docs for running from source that Aaron noticed in the meeting. With this, the PR will be marked ready for final review.
Configuration menu - View commit details
-
Copy full SHA for bca3bf4 - Browse repository at this point
Copy the full SHA bca3bf4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 22d77fb - Browse repository at this point
Copy the full SHA 22d77fbView commit details -
Configuration menu - View commit details
-
Copy full SHA for b5cfc4a - Browse repository at this point
Copy the full SHA b5cfc4aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8f3f58e - Browse repository at this point
Copy the full SHA 8f3f58eView commit details -
Configuration menu - View commit details
-
Copy full SHA for c02cfd8 - Browse repository at this point
Copy the full SHA c02cfd8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4a212e3 - Browse repository at this point
Copy the full SHA 4a212e3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5214d32 - Browse repository at this point
Copy the full SHA 5214d32View commit details -
Configuration menu - View commit details
-
Copy full SHA for acb49bb - Browse repository at this point
Copy the full SHA acb49bbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 68d9294 - Browse repository at this point
Copy the full SHA 68d9294View commit details -
Configuration menu - View commit details
-
Copy full SHA for ff375c7 - Browse repository at this point
Copy the full SHA ff375c7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 345a72b - Browse repository at this point
Copy the full SHA 345a72bView commit details
Commits on Oct 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 599662b - Browse repository at this point
Copy the full SHA 599662bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7374837 - Browse repository at this point
Copy the full SHA 7374837View commit details
Commits on Oct 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for be2a7d5 - Browse repository at this point
Copy the full SHA be2a7d5View commit details
Commits on Oct 4, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7567594 - Browse repository at this point
Copy the full SHA 7567594View commit details -
Configuration menu - View commit details
-
Copy full SHA for 14b24f6 - Browse repository at this point
Copy the full SHA 14b24f6View commit details
Commits on Oct 8, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 31bbe08 - Browse repository at this point
Copy the full SHA 31bbe08View commit details -
Configuration menu - View commit details
-
Copy full SHA for 94669e5 - Browse repository at this point
Copy the full SHA 94669e5View commit details -
Configuration menu - View commit details
-
Copy full SHA for f72c78c - Browse repository at this point
Copy the full SHA f72c78cView commit details
Commits on Oct 9, 2024
-
Configuration menu - View commit details
-
Copy full SHA for fbdaedc - Browse repository at this point
Copy the full SHA fbdaedcView commit details -
Configuration menu - View commit details
-
Copy full SHA for feb8084 - Browse repository at this point
Copy the full SHA feb8084View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1fb03d2 - Browse repository at this point
Copy the full SHA 1fb03d2View commit details
Commits on Oct 10, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 4c78deb - Browse repository at this point
Copy the full SHA 4c78debView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4322aa7 - Browse repository at this point
Copy the full SHA 4322aa7View commit details