Best practices for very large test suites - optimizing the DX

[harvitronix]

Hey all, we use Cypress on a big e-commerce, CRM, scheduling app that we've developed in house. Our test suite contains ~50 spec files with on the order of 700 individual tests. Running the whole suite in serial takes over an hour on an M1.

As our test suite has grown, the probability of a test failing approaches 1. And when it takes an hour, and then you have to go back to re-running a bunch of spec files, what happens is our team just stops running tests after making PRs. Imagine spending 15 minutes on a small PR and then waiting an hour for the tests to run. Horrible DX.

One remedy we put in place is we have a "critical tests" file, where we can just run those ones, which only takes ~15 minutes. Still not ideal but at least it gets run.

I figure we must be doing something wrong. Should we be converting everything to component tests to run faster? Should we be doing fewer e2e tests in general through Cypress? Is cloud-based (and expensive) parallelization the right or only solution?

Would love to hear how others with large apps and test suites are solving for this. Thanks!

[harvitronix]

@jennifer-shehane any thoughts or advice on this?