api: Test Debug Flag / Pause between tests among other behaviors

[michael-lloyd-morris]

🤔 What's the problem you're trying to solve?

While tests are being developed they sometimes need some debugging, especially when a browser driver is in play. At my last job I installed such a feature that was invoked using an npm config flag, but I think this feature is a good candidate to be moved into the direct core suite.

✨ What's your proposed solution?

Cucumber would have a --debug flag and would provide a means for userland code to react to it, possibly just a simply importable function that will return boolean whether the test is running in debug. The major reason for including this is configuring the various browser drivers out there (Selenium, Puppeteer, Playwright) is out of the scope of Cucumber, but tests using them are the most likely consumer of this feature. The userland script would run the tests with the browser GUI displayed (Headed mode) if the debug flag is set, and turn off any automatic expiry of the session so that the user is free to inspect what is going on in the browser.

In addition to providing this flag Cucumber would pause between test runs and wait for the user to press a key. This can be done in userland with hooks and npm configuration options and looks like this:

  async function waitforPrompt(prompt) {
    const rl = readline.createInterface({
      input: process.stdin,
      output: process.stdout,
    });

    return new Promise((resolve) => {
      rl.question(prompt, () => {
        rl.close();
        resolve();
      });
    });
  }

 if (process.env.npm_config_ucx_debug === "true") {
    After({ timeout: -1 }, async function (scenario) {
      await waitforPrompt(`Feature: ${scenario.gherkinDocument.feature.name}\nScenario: ${scenario.pickle.name}\nComplete. Press Enter to Continue.`);
      await this.tearDown();
    });
  } else {
    After(async function (scenario) {
      await this.tearDown();
    });
  }

This behavior isn't compatible with parallel running, so if the base debug flag and parallel are used the parallel flag is suppressed with a warning ("Cannot create parallel runners in debug mode")

⛏ Have you considered any alternatives or workarounds?

This is doable in userland, but it's awkward and having a direct setup in the core library would be useful

📚 Any additional context?

I would like to have some discussion on this at length here or in slack before beginning. The addition of this flag has major implications and there are doubtless other areas of the suite I haven't thought of that might want to alter their behavior when it is present.

[davidjgoss]

I would see this being more of an add-on than a core feature. @mattwynne was looking at something similar a while ago but with an Electron-based UI rather than an interactive terminal.