Test run can hang/freeze when browser closes unexpectedly

[bsnyder0]

What is your Scenario?

Our team utilizes GitHub Actions to run a series of TestCafe tests nightly against our environments. During random runs; not often, but potentially once a month after 10+ runs a day; the test run will freeze, resulting in GHA force closing the runner after 6 hours. Investigating into this, we haven't been able to fully reproduce this issue, outside of manually closing the browser during a test. Our tests on GHA (ubuntu:latest) utilize Chrome in Headless mode, and our tests when developing (Windows 11) use Chrome standard (some tests using --guest), but not headless. We aren't yet convinced the browser is closing during the GHA tests, but this is our current/best lead.

What is the Current behavior?

During our tests in Windows, closing the browser mid session (once the fixture/test start their actions), calls to await t functions will hang, as they cannot be completed.

What is the Expected behavior?

Ideally, the closed browser / lost connection, should be detected as a failure, and either one of the following actions:

• Quit the run entirely
• Attempt to run the fixture 'after' function, and abort the rest of the queued tests
• Likely, attempting to run the test/fixture afterEach can't run, due to the lack of t environment anymore.

Reporters should be able to finalize their run and export their results of the tests that [did] run.

What is the public URL of the test page? (attach your complete example)

Can be repro'd on example.com

What is your TestCafe test code?

runner.js

const fs = require('fs');
const path = require('path');
const createTestCafe = require('testcafe');

/* Required for tests that spawn child tests */
global.createTestCafe = createTestCafe;

/* Required for running via proxy, and fake media tests */
const selfSigned = require('openssl-self-signed-certificate');
const sslOptions = {
    key: selfSigned.key,
    cert: selfSigned.cert
};


(async () => {

  const testcafe = await createTestCafe({
    hostname: 'localhost',
    sslOptions
  });

  const browsers = [];
  let chrome = [];
  let runnerOptions = {};
  if(process.platform==='win32') { // windows
      chrome.push('chrome --start-maximized');
  } else { // not windows / github actions likely
      chrome.push('chrome:headless');
      runnerOptions = {
        quarantineMode: {
          attemptLimit: 3,
          successThreshold: 1
      }
      }
  }
  chrome.push('--allow-insecure-localhost');

  /* If NODE_MEDIA is set, require fake media sources in tests */
  /* Optional: Set NODE_AUDIO or NODE_VIDEO to files in ~/assets, to override the default files */
  if(process.env.NODE_MEDIA) {
    // check if media files exist
    let audioFile = path.resolve(`assets/${process.env.NODE_AUDIO ?? 'source_audio.wav'}`);
    let videoFile = path.resolve(`assets/${process.env.NODE_VIDEO ?? 'source_video2.y4m'}`);
    
    // add if exists
    if(fs.existsSync(audioFile) && fs.existsSync(videoFile)) {
        console.log("Using Chrome with fake video and audio");
        chrome.push('--use-fake-ui-for-media-stream');
        chrome.push('--use-fake-device-for-media-stream');
        chrome.push(`--use-file-for-fake-video-capture=${videoFile}`);
        chrome.push(`--use-file-for-fake-audio-capture=${audioFile}`)
    }
  }
  chrome = chrome.join(' ');
  let failedCount = 0;
  try {
      function recurringLogger(x, ms) {
        let onceLock = false;
        const logIt = async () => {
          if(onceLock) return;
          onceLock = true;
          let openSession = x.proxy?.openSessions.keys().next().value;
          if(openSession) {
            let session = x.proxy.openSessions.get(openSession);
            let sessionCount = session.proxy.server1._connections
            console.log(`Connection count: ${sessionCount}`);
            if(sessionCount===0) {
              console.log("Oh no, browser closed?!?!? Stopping");
              await runner.stop();
              console.log("Stop finished?");
              failedCount.cancel();
              console.log("Cancel called");
              await testcafe.close();
              console.log("Testcafe closed?");
            }
          } else {
            console.log("No session open");
          }
          
          onceLock = false;
          // console.log(x.proxy?.openSessions);
        }
        let loop = setInterval(logIt, ms);
        process.on('beforeExit', () => {
          console.log("Final exit thoughts");
          clearInterval(loop);
      
        })
      
      }

      const runner = testcafe.createRunner();
      const testSources = [
        'tests/**/*.js'
      ];
      
      failedCount = runner
          .src(testSources)
          .browsers(chrome)
          .concurrency(1)
          .video('failures/videos', {failedOnly: true})
          .run(runnerOptions);

      recurringLogger(runner, 500);

      failedCount = await failedCount;

      console.log('Tests failed: ' + failedCount);
      await testcafe.close();
  }
  catch(e){
    console.log('Error in running tests',e);
  }
  finally {
      process.exit(failedCount);
  }
})();

test.js

import { Selector } from 'testcafe';

fixture.only('Fixture')
  .page('http://example.com')
;

test('test', async t => {
    for(let i = 0; i < 20; i++) {
        await t
            .expect(Selector('h1').innerText).eql('Example Domain')
            .wait(1000);
        console.log("Completed check: ", i);
    }
});

Your complete configuration file

No response

Your complete test report

No response

Screenshots

No response

Steps to Reproduce

1. run node runner.js
2. note the logging showing the browser connection initializing, becoming available, and the test counter running
3. close the browser before the test finishes, but after the counting started; within 20 seconds of the test run

TestCafe version

3.6.2

Node.js version

v16.20.2

Command-line arguments

node runner.js

Browser name(s) and version(s)

Chrome 128

Platform(s) and version(s)

No response

Other

This test has been done with both Native Automation enabled and disabled with no difference.

[github-actions]

We appreciate you taking the time to share information about this issue. We reproduced the bug and added this ticket to our internal task queue. We'll update this thread once we have news.