Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests running in a GH action sharing a nuxt server a bit faster #925

Open
4 tasks done
acidjazz opened this issue Aug 24, 2024 · 5 comments
Open
4 tasks done

Tests running in a GH action sharing a nuxt server a bit faster #925

acidjazz opened this issue Aug 24, 2024 · 5 comments

Comments

@acidjazz
Copy link

acidjazz commented Aug 24, 2024

Describe the feature

In my current Nuxt project I have 6 test files each testing about 8-10 endpoints, each file currently spins up its own nuxt instance making the entire action require about 10 minutes to run, and vitest itself taking 8m 46s

So I tweaked my GH action to instead run a nuxt server in an action in the background, and then sleep for 10 seconds to make sure it completes its run

  - name: Start Server
    run: pnpm run dev:test &
  - name: Sleep for 10 seconds
    run: sleep 10

Update: sleep no longer needed

Now instead of vitest taking 8 minutes 46 seconds, it finished in 18 seconds.

You can find the entire action file here

I found this pretty amazing so I thought I'd share starting with an issue with the questions:

  • Is there any way to improve how I'm doing this?
  • Where should we document this so the community can easily find it?

Very happy to provide a PR anywhere to help share this strategy

BTW shot out to @tobiasdiez for Making this possible

Additional information

  • Would you be willing to help implement this feature?
  • Could this feature be implemented as a module?

Final checks

@acidjazz acidjazz changed the title Tests running in a GH action sharing a nuxt server WAY faster Tests running in a GH action sharing a nuxt server a bit faster Aug 24, 2024
@antfu
Copy link
Member

antfu commented Aug 24, 2024

That's an interesting observation; thanks for sharing!

Unfortunately, I am not able to set up your project to run locally (Prisma related). It would be great if you could set up a minimal reproduction.

On the other hand, I suggest you could try the following two approaches:

@acidjazz
Copy link
Author

Move the server start to a global setup file, so you don't need an extra step in your action: https://vitest.dev/config/#globalsetup

I wasnt able to get globalSetup working, here is what I tried:

vitest.config.ts

...
  test: {
    globalSetup: './test/globalSetup.ts',
...

in ./test/globalSetup.ts i tried

import { setup as nuxtSetup } from '@nuxt/test-utils'

export default async function setup() {
  await nuxtSetup()
}

(i tried both named and default)

and I get

Error: Vitest failed to access its internal state.

One of the following is possible:
- "vitest" is imported directly without running "vitest" command
- "vitest" is imported inside "globalSetup" (to fix this, use "setupFiles" instead, because "globalSetup" runs in a different context)
- Otherwise, it might be a Vitest bug. Please report it to https://github.com/vitest-dev/vitest/issues

 ❯ getWorkerState ../node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/vitest/dist/chunks/utils.Ck2hJTRs.js:5:11
 ❯ getTestFile ../node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/vitest/dist/chunks/vi.fiQ7lMRF.js:523:17
 ❯ createExpect ../node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/vitest/dist/chunks/vi.fiQ7lMRF.js:466:20
 ❯ ../node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/vitest/dist/chunks/vi.fiQ7lMRF.js:526:22
 ❯ ModuleJob.run ../node:internal/modules/esm/module_job:222:25
 ❯ ModuleLoader.import ../node:internal/modules/esm/loader:316:24
 ❯ ViteNodeRunner.interopedImport ../node_modules/.pnpm/[email protected]_@[email protected][email protected]/node_modules/vite-node/dist/client.mjs:421:28
 ❯ ViteNodeRunner.directRequest ../node_modules/.pnpm/[email protected]_@[email protected][email protected]/node_modules/vite-node/dist/client.mjs:280:24
 ❯ ViteNodeRunner.cachedRequest ../node_modules/.pnpm/[email protected]_@[email protected][email protected]/node_modules/vite-node/dist/client.mjs:206:14
 ❯ ViteNodeRunner.dependencyRequest ../node_modules/.pnpm/[email protected]_@[email protected][email protected]/node_modules/vite-node/dist/client.mjs:259:12

@acidjazz
Copy link
Author

acidjazz commented Aug 24, 2024

When I do this I get prisma errors which I believe are when they are trying to create the same user at the exact same time:

FAIL  ../test/cartridge.test.ts > /api/cartridge > post /api/cartridge - create a cartridge
PrismaClientKnownRequestError:
Invalid `prisma.user.upsert()` invocation:


Unique constraint failed on the constraint: `users_email_key`
 ❯ _n.handleRequestError ../node_modules/.pnpm/@[email protected][email protected]/node_modules/@prisma/client/runtime/library.js:121:7749
 ❯ _n.handleAndLogRequestError ../node_modules/.pnpm/@[email protected][email protected]/node_modules/@prisma/client/runtime/library.js:121:7057
 ❯ _n.request ../node_modules/.pnpm/@[email protected][email protected]/node_modules/@prisma/client/runtime/library.js:121:6741
 ❯ l ../node_modules/.pnpm/@[email protected][email protected]/node_modules/@prisma/client/runtime/library.js:130:9355
 ❯ Module.createUser utils/user.ts:11:10
      9|   let user: User | null = null
     10|
     11|   user = await prisma.user.upsert({
       |          ^
     12|     where: { email: info.email },
     13|     create: {
 ❯ userFromEmail ../test/auth.ts:38:28
 ❯ Module.actingAs ../test/auth.ts:44:16
 ❯ ../test/cartridge.test.ts:12:28

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { code: 'P2002', clientVersion: '5.18.0', meta: { modelName: 'User', target: 'users_email_key' }, batchRequestIdx: undefined }
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

to avoid this in the action or locally, i have to set this in vitest:

vitest.config.ts

...
  test: {
    poolOptions: {
      forks: {
        minForks: 1,
        maxForks: 1,
      },
    },
...

Created an issue in more detail here

@acidjazz
Copy link
Author

acidjazz commented Aug 25, 2024

UPDATE I was able to swap out the nuxt dev & and sleep 10 actions with starting nuxt in the build:

The entire action now runs in 2m 11s

- name: Build and start server in the background
  run: pnpm run build:test:start:bg

here is build:test:start:bg in my package.json

"nuxt build --dotenv .env.test; node --env-file=.env.test .output/server/index.mjs &"

notice i had to use --env-file= which requires node v20+ so i also added actions/setup-node-v4

Updated action found here

@adamdehaven
Copy link
Contributor

adamdehaven commented Nov 29, 2024

FWIW I run my e2e tests against a locally running wrangler environment. Playwright allows you to pass options to start up a webServer, but you could alternatively utilize a package like start-server-and-test to first spin up your test environment, and then run the tests once it's up.

Utilizing a package like this will also take care of shutting down the dev server when the tests complete or fail, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants