More in-depth analysis of tests

[grug]

Hello there!

A project that we're working on (multiple teams contributing) has been struggling in the e2e department. To give some idea around how we're currently using Cypress:

• We have lots of test suites
• We use lots of custom Cypress commands
• We run the tests frequently

We are facing the current problems:

• Tests are slow to execute
• Tests are flakey

None of this is Cypress' fault obviously! The thing is that we keep getting into a situation where we fix the flakey or slow tests and then in a few weeks we wind back up in a position where we have a bunch more slow and flakey tests.

One thing that can help us out is on the Cypress side, we can try and offer some insights to our developers around what things are slow. Obviously from the Cypress dashboard (and CircleCI's dashboards) we can see

• Which tests are the flakey ones
• Which ones are the slowest/most failing)

But it would be great to drill deeper than that. Ideally we'd be able to offer an insight into which commands are taking the most time to execute (both Cypress-provided and our custom commands) to work out where we can shave time off and start to develop habits where we aren't writing such slow/flakey tests.

To pick a starting point for this endeavour, I'm writing some utilities around capturing the execution times of our custom commands and generating reports on that.

My questions are:

• Is there a way to hook into just before and after Cypress commands are executed
• Has anyone looked into anything like this sort of thing before (delivering more granular analytics around Cypress)
• Does anyone know why Cypress doesn't report the time it took for each individual test to run? I know it reports how long a suite took to run, but it would be nice to go deeper than that