Running on CI is different than running on local

[zeyupan888]

Even though the cypress on CI's version is 13.2.0 which is the same as the one I used in my local machine, the CI failed all tests and local machine doesn't fail any test at all.
The error comes from

Timed out retrying after 4000ms: Cannot read properties of undefined (reading 'origin')
cy.visit() failed trying to load:

http://localhost:8080/

We attempted to make an http request to this URL but the request failed without a response.

We received this error at the network level:

  > Error: ESOCKETTIMEDOUT

Common situations why this would fail:

you don't have internet access
you forgot to run / boot your web server
your web server isn't accessible
you have weird network configuration settings on your computer
Because this error occurred during a before all hook we are skipping all of the remaining tests.

Cannot read properties of undefined (reading 'KeyboardEvent')

why the CI is so different than local? Thanks.

[hadiimran00]

There are a few reasons why Cypress tests on CI might fail while passing locally:

Different environment: CI environments are often very different from local development environments, in terms of the underlying operating system, hardware, and software configuration. This can lead to subtle differences in how Cypress behaves, which can cause tests to fail.

Network connectivity issues: CI environments can also have network connectivity issues, which can prevent Cypress from reaching the application under test. This can be caused by a variety of factors, such as firewalls, proxies, and DNS issues.

Race conditions: Cypress tests are often written to be asynchronous, and this can lead to race conditions in CI environments. Race conditions occur when two or more events happen at the same time, and the order in which they happen can affect the outcome of the test.

In the case of the error message you provided, it seems likely that the issue is related to network connectivity. The error message ESOCKETTIMEDOUT means that Cypress was unable to connect to the application under test within the specified timeout period. This could be due to a firewall, proxy, or DNS issue.

Here are some things you can try to troubleshoot the issue:

Check your network connectivity: Make sure that your CI environment has internet access and that the application under test is accessible. You can do this by running a simple command like ping or curl.

Check your firewall and proxy settings: Make sure that your CI environment is configured to allow Cypress to connect to the application under test. You may need to open additional ports or configure a proxy.

Check your DNS settings: Make sure that the DNS settings in your CI environment are correct. You can do this by running a command like nslookup to resolve the hostname of the application under test.

Increase the timeout period: You can try increasing the timeout period for Cypress tests in your CI environment. This will give Cypress more time to connect to the application under test.

If you are unable to resolve the issue, you may need to contact your CI provider for assistance.

Here are some additional tips for running Cypress tests in CI environments:

Use a CI runner that is compatible with Cypress: Some CI runners are not compatible with Cypress, so it is important to choose one that is.

Use a consistent environment: Try to use the same environment for your CI tests as you use for your local development. This will help to reduce the number of variables that can cause tests to fail.

Run your tests in parallel: Cypress supports running tests in parallel, which can help to improve the performance of your CI pipeline.

Use a CI reporting tool: There are a number of CI reporting tools that can be used to generate reports from Cypress tests. This can help you to identify and debug test failures.

I hope this helps!

[zeyupan888]

I still have trouble triagingthe root cause

[Dusste]

Hi, on the same topic - CI /= local, how is possible to have significantly less coverage on CI then on local, all tests are passing btw. So on CI the average is 54% but on local is 75%