-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: adding some basic documentation #10
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,4 +4,129 @@ A tool for managing many combinations of real and fake NPM packages within your | |
|
||
Extracted from [ember-auto-import](https://github.com/ef4/ember-auto-import). No docs here yet, but: | ||
- you can see how it's used in ember-auto-import/test-scenarios | ||
- it's all typescript so you can follow the types to learn what's possible | ||
- it's all typescript so you can follow the types to learn what's possible | ||
|
||
## Getting started | ||
|
||
`scenario-tester` works especially well with a monorepo. We recommend that you create a new workspace package for your scenarios e.g. : | ||
|
||
``` | ||
mkdir test-packages/test-scenarios | ||
``` | ||
|
||
Next you need to create some scenarios. The best way to create a scenario is from a Project, and the best way to create a project is to have a base project on disk. For example, in EmberJS you can create a new project with `ember new base-project` and it will create a new empty project for you. We can use this as a starting point for our other scenarios. | ||
|
||
``` | ||
cd test-packages/test-scenarios | ||
ember new base-project | ||
``` | ||
|
||
Next you can create a scenarios file that you can export a base project function: | ||
|
||
|
||
```js | ||
// test-packages/tests-scenarios/scenarios.js | ||
import { Project } from 'scenario-tester'; | ||
import { dirname } from 'path'; | ||
|
||
// This is a way to allow for `require.resolve` to work in esm modules in node. | ||
// If you are using CJS you can skip these two lines | ||
import { createRequire } from 'node:module'; | ||
const require = createRequire(import.meta.url); | ||
|
||
export function baseApp() { | ||
return Project.fromDir(dirname(require.resolve('./classic-app-template/package.json')), { linkDevDeps: true }); | ||
} | ||
``` | ||
|
||
While it's not quite that useful (yet) you can use this base app to build a specific test scenario. Let's do a simple test that boots your app. | ||
|
||
```js | ||
// test-packages/test-scenarios/boot-app-test.js | ||
|
||
import { baseApp } from './scenarios'; | ||
import { Scenarios } from 'scenario-tester'; | ||
import qunit from 'qunit'; | ||
|
||
const { module: Qmodule, test } = qunit; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do we alias these? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. so in a real ESM setup you absolutely do because qunit isn't ESM 😞 In typescript we have been getting away with it for a while because it's so loosey-goosey with the whole thing 🤪 I've been testing this in #11 and ember-fastboot/ember-cli-fastboot#919 and it doesn't work any other way 👍 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @NullVoxPopuli vite papers over a lot of incompatibilities for you so not sure if it's a good base to go off of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. oh sorry I thought you were talking about something else 🤔 I thought you were just telling me to do this:
and that's what I was saying just doesn't work. I didn't realise you were commenting on the fact that I was renaming module 🙃 The reason I did this is because I copy and pasted from... either ember-auto-import or embroider... I can't remember. I never questioned it, but if it's not an issue I am happy to just use module directly 🤷 I assume Ed probably used it because there were a billion and a half uses of the word |
||
|
||
Scenarios.fromProject(baseApp) | ||
.map('boot-app-test', project => { | ||
// TODO add some specifics for your test scenarios here | ||
}) | ||
.forEachScenario(scenario => { | ||
Qmodule(scenario.name, function (hooks) { | ||
let app; // PreparedApp | ||
|
||
hooks.before(async () => { | ||
app = await scenario.prepare(); | ||
// any custom setup that you have for each scenario | ||
}); | ||
|
||
test('it works', async function (assert) { | ||
// your custom test code | ||
assert.ok(true); | ||
}); | ||
}); | ||
}); | ||
``` | ||
|
||
The power in this kind of setup comes when you start adding in different dependencies that you might want to test this app with. We'll go into that a bit more in the next section. For now to run these tests we can run the following command. | ||
|
||
```bash | ||
npx qunit *-test.js | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would favour package.json scripts even though npx may function |
||
``` | ||
|
||
This will run each of the tests in your setup sequentially. This can take a while, so on CI it can be useful to run all your scenarios in paralell. For this we need your CI to be able to discover all the possible tests that are available and run one specific test. | ||
|
||
**Note:** all of our examples are using Gihub Actions so your CI provider may need a different setup | ||
|
||
## CI Setup | ||
|
||
The first job in your CI will need to discover all the tests that are available to use: | ||
|
||
```yaml | ||
jobs: | ||
discover_matrix: | ||
runs-on: ubuntu-latest | ||
outputs: | ||
matrix: ${{ steps.set-matrix.outputs.matrix }} | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- uses: actions/setup-node@v3 | ||
with: | ||
node-version: '16' | ||
cache: npm | ||
- run: npm install | ||
- id: set-matrix | ||
working-directory: test-packages/test-scenarios | ||
run: echo "::set-output name=matrix::$(npm run --silent test:list -- --matrix 'npm run test -- --filter %s:')" | ||
``` | ||
|
||
This will generate a matrix for you that will contain a command to filter down to just one test. You can then use this to run the test in another job: | ||
|
||
```yaml | ||
jobs: | ||
scenarios: | ||
needs: discover_matrix | ||
name: ${{ matrix.name }} | ||
runs-on: ubuntu-latest | ||
strategy: | ||
fail-fast: false | ||
matrix: ${{fromJson(needs.discover_matrix.outputs.matrix)}} | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- uses: actions/setup-node@v3 | ||
with: | ||
node-version: '16' | ||
cache: npm | ||
- run: npm ci | ||
- name: test | ||
run: ${{ matrix.command }} | ||
working-directory: test-packages/test-scenarios | ||
``` | ||
|
||
|
||
|
||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably want --skip-npm and --skip-git flags on that ember command
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did that when I personally ran the command but I didn't want to make this all about ember 🤔 there is nothing tying scenario-tester to ember and I thought that was an extra detail that we shouldn't include here. Does that make sense?
I don't feel all that strongly about this so if you (or anyone) does I'm happy to add them 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it does make sense, but as is, the example instruction of
ember new base-project
is misleading / wrong, which is where I'm coming from.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for the purpose of having a less frustrating experience and a working set of commands I would add them
OR provide a command to achieve the same without using the ember-cli