Success locally but failed on github actions CI

[convers39]

I am trying to access the redux store when doing the e2e test (may sound weird but there is no way to reach the HTML element as it renders as WebGL layers on a canvas), now the pretty strange thing is that:

for the same test code running at

• localhost against the server of localhost
• localhost against the server of the deployed dev env website
• localhost against the server of the deployed staging env website
• github actions CI against the server of the deployed dev env website

all succeeded without any problem, but only when running at github actions CI against the server of the deployed staging env website failed with one single test case.

Here is my code, I am trying to compare the road highlight length before and after the CREATE operation. The length should be +1 if the code works as expected.

// get redux store comand
Cypress.Commands.add('getReduxStore', (key: string) => {
  return cy
    .window()
    .its('store')
    .invoke('getState')
    .then(state =>  state[key])
})

// test code
      cy.getReduxStore('road').then(({ list }) => {
        const lengthBefore = list.length
        // CREATE operation ...
        cy.wait('@addRoad').then(interception => { // the data is mocked so the response is consistent despite envs
          const { response } = interception
          // request succeeded
          expect(response.statusCode).eq(200)
          expect(response.body.status).eq('success')

          // check react state if the new avoid data exists
          cy.getReduxStore('road').then(({ list }) => {
            // A new road should be added to the state
            expect(list.length).eq(lengthBefore + 1) // -> failed here
            const ids = Cypress._.map(list, 'roadId')
            // avoidId should match the response
            expect(ids).include(response.body.roadId)
          })

          // should show success notification
          cy.matchNotification(/success/i)
        })
      })

I know this sounds very crazy ( and it does drive me crazy), but from the downloaded video, visually you can tell that the test should pass.

This is the state before CREATE operation, which will increase the highlight on a certain road.

This is the state after CREATE operation, and it shows the highlight is added as expected.

This does not make any sense since the highlight roads are reactive to the road list in the redux store, it cannot be visually changed if the store does not change, and the test failed as the road list length somehow did NOT change, which is contradictory.

Here is the CI workflow, the only difference is the baseUrl as it points to different domains for tests against each env.

      - name: Run tests for dev
        if: ${{ (inputs.environment == 'dev') || (github.base_ref == 'release') }}
        uses: cypress-io/github-action@v4
        env:
          CI: true
          TZ: Asia/Tokyo
          LOG_LEVEL: ${{ inputs.logLevel }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          CYPRESS_username: ${{ secrets.CYPRESS_USERNAME }}
          CYPRESS_password: ${{ secrets.CYPRESS_PASSWORD }}
          CYPRESS_PROJECT_ID: ${{ secrets.CYPRESS_PROJECT_ID }}
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
        with:
          config: baseUrl=https://xxx-dev.domain.com
          browser: chrome
          record: true
          parallel: true
          working-directory: e2e
          spec: |
            cypress/**/*cy.{js,ts}
          wait-on-timeout: 300
      - name: Run tests for stg
        if: ${{ (inputs.environment == 'stg') || (github.base_ref == 'master') }}
        uses: cypress-io/github-action@v4
        env:
          CI: true
          TZ: Asia/Tokyo
          LOG_LEVEL: ${{ inputs.logLevel }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          CYPRESS_username: ${{ secrets.CYPRESS_USERNAME }}
          CYPRESS_password: ${{ secrets.CYPRESS_PASSWORD }}
          CYPRESS_PROJECT_ID: ${{ secrets.CYPRESS_PROJECT_ID }}
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
        with:
          config: baseUrl=https://xxx-stg.domain.com
          browser: chrome
          record: true
          parallel: true
          working-directory: e2e
          spec: |
            cypress/**/*cy.{js,ts}
          wait-on-timeout: 300

As I cannot reproduce this in other envs, I am not sure what is going on and what should be the problem.

If there are any similar issues/hints it would be really appreciated!

Or may have to consider visual testing instead of peeking at the redux store...