Command line exit code

[jasonw-rr]

https://docs.cypress.io/guides/guides/command-line#Exit-code
Anyone know about the reasons behind this design decision?

CI is having a hard time determining if the test runner is having a problem or there is a case failing?
I would suggest Cypress command line to respect POSIX-compatible exit code convention, and deal with assertion failed case count separately for 2 reasons:

1. The CI needs to be able to tell if the process executed successfully to ensure pipeline integrity, as test result is subjective and should be handled elsewhere (sometimes certain failed cases are acceptable, and the build is flagged as unstable instead of failing the entire build). If Cypress has problem finish running all specs, such as script error or invalid env format, the CI could identify/escalate the issue and then halt the pipeline.
2. Valid exit code is 0~255, there is a chance the failed case count is greater than 255 and does not return the actual number.

[jennifer-shehane]

I believe we took this behavior from mocha. Seems the decision for that was equally not explained well. mochajs/mocha#3559

And yah, the 'correct' error code doesn't print for failed tests beyond 255, it just prints exitCode 1.

255 failures

257 failures