Skip to content

Commit

Permalink
doc: describe running and creating code tests
Browse files Browse the repository at this point in the history
  Also adds new tests for `src/shared/defineFeatured.ts` and corrects
  the documentation extraction facility for the package manager
  scripts.

  Resolves numberscope#25.
  Resolves numberscope#73.
  Resolves numberscope#246.
  • Loading branch information
gwhitney committed Aug 30, 2024
1 parent b98114a commit 11a1def
Show file tree
Hide file tree
Showing 5 changed files with 193 additions and 5 deletions.
166 changes: 166 additions & 0 deletions doc/code-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
# Code tests

For its front end, Numberscope uses [`vitest`](https://vitest.dev/) for unit
tests and low-level integration tests that run without a browser, and
[`Playwright`](https://playwright.dev/) for higher-level integration and
end-to-end integration tests that run in a browser

### Running tests

The package manager is set up with scripts that make basic running of all
tests very convenient. To run all unit (vitest) tests, execute
`npm run test:unit`. To run all end-to-end (Playwright) tests, use
`npm run test:e2e`.

#### Customizing vitest runs

If some situation in the code has led to one of the unit tests failing, you
may find yourself in a situation in which you need to run that test
repeatedly, and would rather not run all of the other tests. Fortunately, the
tests are organized by source directory. Hence, if the failure is occurring in
a test associated with the `src/shared` directory (say), then you can run just
the tests associated with that directory via `npm run test:unit src/shared`.
For a brief overview of other options you might use with running unit tests,
try `npm run test:unit -- --help`. (Note the extra `--` stuck in there, which
is needed because the test option you want to use here itself begins with
`--`.)

#### Customizing Playwright runs

You may find yourself in a similar situation with end-to-end testing. In this
case, all of the tests are specified in the top-level `e2e` directory.
However, they are organized into thematic files, which you can see via
`ls e2e`, for example. You can run just one of these collections of tests, say
the ones for the Gallery view in `e2e/gallery.spec.ts`, by executing
`npm run test:e2e gallery`. And similarly to unit tests, you can obtain an
overview of other possibilities via `npm run test:e2e -- --help`.

## Creating tests

If you are considering contributing code to Numberscope, you will very likely
need to create tests. The [pull request checklist](pull-request-checklist.md)
notes that any new feature (including a new visualizer) requires some test
that exercises it; and any bug fix should include a test that would have
failed with the buggy code but which now succeeds on the improved code. There
are somewhat different processes for adding each of the two kinds of tests.

### Adding a unit test

As mentioned above, these tests are organized per source directory, in
subdirectories named `__tests__`. More than likely such a directory will
already exist alongside the source file you want to test, but if not, simply
create one -- vitest will find it automatically. Within the **tests**
directory, the tests are organized by source file, and named accordingly, with
file extension `.spec.ts`. Again, if you are adding tests related to a file
that already has associated tests, you can just insert new tests in an
existing file. If there are no tests yet for the file you are concerned with,
you can just create the `.spec.ts` file and the framework will automatically
execute it.

Let's go through the latter case. At the time of this writing, there was no
test specification for the `src/shared/defineFeatured.ts` file that provides
the specimens for the Featured Gallery. So, we simply created
`src/shared/__tests__/defineFeatured.spec.ts`. At the top of the file you need
to import the test facilities from `vitest`, and typically all of the exported
entities from the module being tested, like so:

```
{! ../src/shared/__tests__/defineFeatured.spec.ts extract: { stop: ibe..g } !}
```

In this case, there is only one export, `getFeatured`, so the test
specification is pretty brief. Here it is in full:

```
{! ../src/shared/__tests__/defineFeatured.spec.ts extract: { start: tured } !}
```

As you can see, tests are arranged in groups, each consisting of a call to
`describe`. The first argument is a string saying what is being tested in that
group, typically one of the exports. The second argument is a function that
executes the test.

Within that function, each test is specified by a call to `it`. The first
argument is again descriptive, giving the property that is being tested. And
the second argument is again a function that runs the test. The body of this
function performs some trial computation using the item being tested,
typically followed by one or more assertions about the outcome of that
computation. The vitest framework provides a
[mini-language](https://vitest.dev/api/expect.html) for specifying assertions,
each triggered by a call to the `expect` function. You can see a few examples
in the code above (admittedly some a bit contrived for the sake of this
exposition).

And that's all there is to it. With this spec file newly in place,
`npm run test:unit` reports three more passed tests than before. It's ready to
be committed and made part of a pull request.

### Adding an end-to-end test

The Playwright framework for end-to-end testing is broadly similar. In this
case, the spec files are all in the `e2e` directory, and again, they will
automatically be discovered and run when placed there. The file extension is
the same, and the main names are essentially arbitrary but should give an idea
of what's being tested.

Let's take a look at one test in `e2e/gallery.spec.ts` that tests proper
operation of the page reached by clicking on the "Gallery" item in
Numberscope's navigation bar. As with vitest, there's a preamble importing the
test framework:

```
{! ../e2e/gallery.spec.ts extract: { stop: before } !}
```

but this time there's not necessarily a need to explicitly import any of the
Numberscope code. That's because Playwright builds the entire system and opens
up its root URL in a browser running in "headless" mode -- that is to say, it
simulates rendering the resulting pages, and needs no attached display. The
Playwright tests manipulate that browser and verify it operates as expected.

Next, we see a new test framework feature: the ability to run code before
every one of the tests in this file:

```
{! ../e2e/gallery.spec.ts extract: { start: play, stop: describe} !}
```

You'll notice that this code takes a `page` argument. That's common to all
Playwright actions: page represents the browser tab that you are (virtually)
working with in your tests. In this case, we direct the tab to visit the
gallery URL, and we clear out its (simulated) browser-local storage to provide
a clean environment for testing.

Note that vitest has a [similar ability](https://vitest.dev/api/#beforeeach)
to call a block of code before each test, if needed.

Finally, let's look at the second test in this suite:

```
{! ../e2e/gallery.spec.ts extract: { start: '(.*describe.*)', stop: title } !}
...
{! ../e2e/gallery.spec.ts extract: { start: '(.*minim.*)', stop: clicking } !}
...
```

As with vitest, tests are arranged in groups, and here `test.describe` is the
direct analogue of `describe` in vitest, with `test` the analogue of `it`. The
second call to `test` in the "Gallery" group is shown above. Also like vitest,
it consists of a sequence of assertions, with reminiscent syntax. In this
case, most of the assertions have to be `await`ed, because they necessitate
various actions being performed in the browser, which may take some time to go
into effect.

For example, the third item in the test is the direction to click on the first
element in the page with id `featured-arrow`. Whatever that does (it's
supposed to collapse the first group of images in the Gallery), it may take a
little while to occur and have its effects (who knows, it might load some
external webpage, which could take time). Hence the await. The test goes on to
check that the resulting page has the expected organization (correct elements
in the DOM tree), and then clicks again to see that the display goes back to
the way it was.

Hopefully this example demystifies the process of creating a test for
Numberscope. You can also find
[more details](https://playwright.dev/docs/test-assertions) on the actions and
assertions you can put in a Playwright test.
6 changes: 3 additions & 3 deletions doc/working-with-pm.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ an alternate package manager.

<!-- prettier-ignore -->
{! ../package.json extract: {start: 'scripts', stop: '},', replace: [
['help:(\D*)".*".*echo (.*)"', '### npm run \1\n\2'], # one-liner like test
'"help:\D*:\d*".*echo (.*)"', # a numbered continuation line of help
['"help:(\D*)":.*"', '### npm run \1\n'], # header of a block of help lines
['help:(.*\D)".*".*echo (.*)"', '### npm run \1\n\2'], # 1-liner help
'"help:.*[-:]\d*".*echo \\?"?(.*?)\\?"?"', # a continuation line of help
['"help:(.*\D)":.*"', '### npm run \1\n'], # header of block of help lines
'.' # ignore the actual scripts that don't start with help
]} !}
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ nav:
- doc/gitting-it-right.md
- doc/code-principles.md
- doc/code-style.md
- doc/code-tests.md
- doc/pull-request-checklist.md
- doc/visual-studio-code-setup.md
- Internal code and APIs:
Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,21 @@
"help:dev-1": "echo Compiles frontscope and starts a server running it.",
"help:dev-2": "echo Note the server will hot-reload the code from any",
"help:dev-3": "echo source files as they change, to ease development.",
"dev": "npm run typecheck && vite",
"help:dev:workbench": "run-s -s help:dev:workbench-*",
"help:dev:workbench-1": "echo Compiles frontscope and starts a server running it in workbench mode,",
"help:dev:workbench-2": "echo \"with the visualizers in the 'visualizers-workbench' directory loaded.\"",
"help:dev:workbench-3": "echo \"Hot-reloads code changes, just like the 'dev' script.\"",
"dev": "npm run typecheck && vite",
"dev:workbench": "npm run typecheck && vite --mode workbench",
"help:doc:serve": "run-s -s help:doc:serve-*",
"help:doc:serve-1": "echo \"This is like the 'dev' script, but for the\"",
"help:doc:serve-2": "echo embedded documentation rather than the",
"help:doc:serve-3": "echo Numberscope interface itself.",
"doc:serve": "cd tools && node pyrun.mjs mkdocs serve",
"help:doc:serve:workbench": "run-s -s help:doc:serve:workbench-*",
"help:doc:serve:workbench-1": "echo \"This is like the 'dev:workbench' script, but\"",
"help:doc:serve:workbench-2": "echo for the embedded documentation rather than the",
"help:doc:serve:workbench-3": "echo Numberscope interface itself.",
"doc:serve": "cd tools && node pyrun.mjs mkdocs serve",
"doc:serve:workbench": "cd tools && node pyrun.mjs mkdocs serve --config-file=mkdocs-workbench.yml",
"help:build": "run-s -s help:build:*",
"help:build:1": "echo Compiles frontscope and minifies the result,",
Expand Down
21 changes: 21 additions & 0 deletions src/shared/__tests__/defineFeatured.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import {describe, it, expect} from 'vitest'

import {getFeatured} from '../defineFeatured'

describe('getFeatured', () => {
it('returns an array of specimen-in-memory structures', () => {
const featured = getFeatured()
expect(featured[0]).toHaveProperty('query')
expect(featured[0]).toHaveProperty('date')
expect(featured[0]).toHaveProperty('canDelete')
})
it('that are not delete-able', () => {
const featured = getFeatured()
const last = featured.length - 1
expect(featured[last].canDelete).toBeFalsy()
})
it('should generate at least three specimens', () => {
const featured = getFeatured()
expect(featured.length).toBeGreaterThan(2)
})
})

0 comments on commit 11a1def

Please sign in to comment.