experimentalJustInTimeCompile feedback

[AtofStryker]

The experimentalJustInTimeCompile option was released in Cypress 13.14.0. Full details on its implementation can be found here: https://docs.cypress.io/guides/references/experiments#Experimental-Just-In-Time-Compile

We would like to move forward with making this default, but want to hear everyone's feedback on how this impacts their Component Tests, especially in terms of performance.

Please leave feedback!

[mikeyavorsky]

I am seeing great improvements to performance, for a codebase with 132 component test specs. I'd be excited to make this default.

Initial Compile

Before

468 assets
3282 modules
webpack 5.94.0 compiled successfully in 21129 ms

After

29 assets
862 modules
webpack 5.94.0 compiled successfully in 2999 ms

Recompiling

Before

468 assets
3282 modules
webpack 5.94.0 compiled successfully in 22028 ms

After

33 assets
912 modules
webpack 5.94.0 compiled successfully in 2582 ms

[SergeyVolynkin]

@AtofStryker @jennifer-shehane Quick question if Cypress team evaluated migrating from Webpack to Turbopack/ESBuild/SWC for bundling? Adopting Turbopack could potentially yield significant performance improvements, particularly in reducing startup times for both component and end-to-end tests.

[jasedwards]

I am running nx component tests for my angular components through an azure devops pipeline while connected to cypress cloud. Before setting this value we would randomly get the chunk load error. I don't know if this fixed it or not because I can't actually complete a run. Because webpack now builds everything before each spec it errors out after 30 minutes. I get a "disconnected" error which is fine as after 30 minutes it only ran 5 out of 60 specs. Without the setting it is 30 minutes to run all specs. I am searching for ways to make webpack cache in hoping that helps but have not found anything yet. Ideally I could just use the same tools for cypress that I use for my apps (vite) and then I could move on from this problem. Any suggestions how to run 1 spec in under 5 minutes with this setting would be greatly appreciated.

[AtofStryker]

@jasedwards since your app is built on vite, any reason you can't use @cypress/vite-dev-server and use vite as the bundler value?

[jasedwards]

Right now only webpack dev server is supported for angular component tests. To align with my app builds I would need cypress to support it and also for nx to remove the hard coding where it sets webpack as the bundler. though I assume they will do that once other options are supported.

There was a feature request opened a few weeks ago but does not appear to be any acknowledgement of it yet.

[AtofStryker]

Ah I see. Do you have a sample reproduction we can look at or CI logs? What exactly do you mean by "Webpack now builds everything?"

A user in the chunk load error issue ran into something similar #28644 (comment). Would you be able to try the patch to see how things respond? My guess is since you are also using angular the support file is also likely very large and might be causing problems

[jasedwards]

Sorry, will give more detail on our setup. For component testing instead of having a configuration for every nx library we just have one app called test-app. this contains all of our component tests. With this new setting webpack will rebuild test-app before every spec file (we have 60 right now). This means before each spec there is a 5 minute wait before it can run. I did figure out how to get caching working for webpack but that caused some test failures. Honestly though if I could move to vite it still would not work with that setting because it even though vite only take 1 minute to build my apps it would still be 60 minutes of build time.

Regarding the patch, I was unclear on how to run with the patch if you could provide some detail on that I would appreciate it.

[HerrGustav]

@AtofStryker we still see these webpack chunk loading errors, even with the experimental jit compile feature enabled for component testing in Cypress v13.15.1. E.g. this one from today:

It happens quite regularly and makes our ci really flaky. We use the default runners from Github actions, so they are not the most perfomant out there. But we do split our tests using cypress-split.

We use the normal webpack dev server provided by Cypress, configured for Next:

 devServer: {
      framework: 'next',
      bundler: 'webpack',
    },

Looking at the settings, the jit feature should be enabled:

Our code base is not so large in this specific project, we are talking about 236 component test files split across 8 runners.