Incorrect exit code when failing to install npm dependencies

[christemple]

I noticed that when I've forgotten to update the npm_dependencies versions in the browserstack.js, and Browserstack fails to run the build, running browserstack-cypress run --sync --config-file browserstack.js does not return a failure/error exit code on the commandline, it proceeds as if everything is fine.

Here's an example of the output on one of these runs if it helps:

[2021-10-01T11:20:56.943Z] $ browserstack-cypress run --sync --config-file browserstack.js --build-name test-pipeline-changes-2021-10-01-9283ba5
[2021-10-01T11:20:57.504Z] [10/1/2021, 11:20:58 AM] - info: Reading config from /tmp/workspace/ata-webapp_test-pipeline-changes/browserstack.js
[2021-10-01T11:20:57.504Z] [10/1/2021, 11:20:58 AM] - info: Reading username from the environment variable BROWSERSTACK_USERNAME
[2021-10-01T11:20:57.504Z] [10/1/2021, 11:20:58 AM] - info: Reading access key from the environment variable BROWSERSTACK_ACCESS_KEY
[2021-10-01T11:20:57.504Z] [10/1/2021, 11:20:58 AM] - info: Reading local identifier from the environment variable BROWSERSTACK_LOCAL_IDENTIFIER
[2021-10-01T11:20:57.504Z] [10/1/2021, 11:20:58 AM] - info: Validating the config
[2021-10-01T11:21:05.608Z] [10/1/2021, 11:21:06 AM] - info: Creating tests.zip with files in .
[2021-10-01T11:21:15.543Z] [10/1/2021, 11:21:15 AM] - info: Uploading the tests to BrowserStack
[2021-10-01T11:21:18.804Z] [10/1/2021, 11:21:18 AM] - info: Uploaded tests successfully (bs://8e7fb92dceed99cb94b4559048750c0616804a42)
[2021-10-01T11:21:18.804Z] [10/1/2021, 11:21:18 AM] - info: Deleted tests.zip successfully.
[2021-10-01T11:21:18.804Z] [10/1/2021, 11:21:19 AM] - info: Setting up Local testing...
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:24 AM] - info: Browsers list: OS X Catalina / Chrome (latest), Windows 10 / Edge (latest)
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:24 AM] - info: Local testing set up in always-on mode.
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:24 AM] - info: Local testing identifier: prdmgt-data-webapp-test-pipeline-changes-2021-10-01-9283ba5
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:24 AM] - info: Local is set to: true (you will now be able to test localhost / private URLs)
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:24 AM] - info: Running your tests in headless mode. Use --headed arg to run in headful mode.
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:24 AM] - info: Project name is: prdmgt-data-webapp
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:24 AM] - info: Build name is: test-pipeline-changes-2021-10-01-9283ba5
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:24 AM] - info: Parallels limit specified: 2
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:25 AM] - warn: Your build will run using Cypress 8.4.1 instead of Cypress 8. Read more about supported versions here: http://browserstack.com/docs/automate/cypress/supported-versions
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:25 AM] - info: Success! Build created with build id: b558ca63c68e012f640d29aaf82e2c79159a6f39
[2021-10-01T11:21:25.333Z] [10/1/2021, 11:21:25 AM] - info: Visit the Automate dashboard for real-time test reporting: https://automate.browserstack.com/dashboard/v2/builds/b558ca63c68e012f640d29aaf82e2c79159a6f39
[2021-10-01T11:21:29.493Z] [10/1/2021, 11:21:29 AM] - error: The report cannot be generated as the build b558ca63c68e012f640d29aaf82e2c79159a6f39 is still running. Try again later once the build run is completed.
[2021-10-01T11:21:31.380Z] See the entire build report here:
[2021-10-01T11:21:31.380Z] https://automate.browserstack.com/dashboard/v2/builds/b558ca63c68e012f640d29aaf82e2c79159a6f39

Running this in our Jenkins pipeline doesn't report any errors, however looking at the Browserstack build shows there was a NPM_INSTALL_FAILED issue. Can we get an exit code that reflects the failure in this scenario?

[karanshah-browserstack]

@christemple Can you please update to the latest CLI version v1.10.1? There was a bug in v1.9.0 where on some of the ci machines the report wasn't generating properly.

[christemple]

@karanshah-browserstack I've updated to the latest version, it has fixed the previous --sync issue I experienced in v1.9.0, but I'm still experiencing an issue with v1.10.1... this may not have been introduced recently, it's possible I've just discovered it after investigating build issues.

Whenever there is a NPM_INSTALL_FAILED failure, it does not send back the correct exit code to cause the CI tool to fail (Jenkins in this case), it continues as though the build was successful when it failed to even start because of the npm dependencies failure

[karanshah-browserstack]

@christemple Noted.

[nagpalkaran95]

@christemple We made some changes in recent CLI releases which handles the case if npm install fails on the build level (for all sessions) we exit with non zero exit code (with exception as well).
Can you update the CLI to latest version and let us know if it's still an issue?

[CRoulandSpiral]

This appears to be resolved with 1.18.0 for me. The proper error codes are being passed back on failed npm dependency installs.