Infinite runtime with afterEach and retries enabled

[qlimlass]

Current behavior

My ticket on cypress-cucumber-preprocessor

Semi-reproducible behavior: With retries enabled and anything in afterEach on a test, if that test fails, then the test will run infinitely on the retry attempt whether the second attempt fails or succeeds.

The behavior on my project: I've got timeouts set to 60 seconds and if I have a bad network request hanging longer than that, then it will produce this infinite run on the retry. I'm only able to reproduce it consistently on a handful of websites with "pageLoadTimeout": 1000. The behavior is the same though.

Desired behavior

I am trying to use cypress-cucumber-preprocessor which runs with logic in afterEach. I currently can't put these cucumber tests into CI because of the risk of running jobs infinitely. Desired behavior is that the tests fail or succeed on retry attempts and the test concludes, which is what happens if I remove the afterEach block from the test code below.

Any suggestions for workarounds?

Test code to reproduce

Infinite.js

describe('Infinite', function () {
    afterEach(() => {
        cy.then(() => {});
    });
    it('Infinite', function () {
        cy.visit('https://www.starwars.com/');
    });
});

cypress.json

{
	"retries": 2,
    "pageLoadTimeout": 1000
}

Cypress Version

8.6.0

[adrienjoly]

I have the same problem with Cypress 8.6.0 and 9.1.1, since I enabled retries!

You can find public run logs on https://github.com/openwhyd/openwhyd/runs/4493699111?check_suite_focus=true

Below, you'll see (based on timestamps) that tests should have timed out!

The project is open source, so you'll be able to inspect the tests, configuration and plugins used with Cypress.

[Regaddi]

This issue is currently blocking us from utilizing retries. When we hit this issue, our GitHub Action runner keeps running for hours, unless we notice it's hanging, causing unnecessary usage of our Actions quota. Disabling retries has the side effect that some tests fail randomly due to overall network or infrastructure flakiness, forcing us to re-run the whole test suite again (which again causes unnecessary usage of our Actions and Cypress Dashboard quota).

[flotwig]

Agreed that this seems like an issue. Closing this as duplicate of #21619 since that has more information in the thread.