[UI] investigate issue filtering tests

[yonasberhe23]

Investigate issue filtering tests - even though we are filtering tests using grep tags in CI, we end up with a combination of tests that we don't expect.

ref: rancher/dashboard#12934 (comment)

[yonasberhe23]

It seems like the issue is caused by exclusions (--@jenkins), which might not be properly interpreted when combined with + filters (e.g. @standardUser+@manager --@jenkins). Without the @jenkins exclusion, filtering works correctly.

To work around this, I tried using the CYPRESS_grepExcludeTags option instead of --@jenkins, hoping it would handle exclusions more effectively. However, I encountered the same issue.

I've attempted to fix this by:

• Explicitly passing the options via CLI (in package.json).
• Defining the options in the workflow YAML file.
• Using a combination of both methods.

Despite these attempts, I'm still seeing the issue specifically with the @standardUser tag, where all spec files are being found and searched instead of being filtered properly.

Example from package.json:

"e2e:prod2": "CYPRESS_grepTags=@standardUser+@manager CYPRESS_grepExcludeTags=@jenkins CYPRESS_grepOmitFiltered=true CYPRESS_grepFilterSpecs=true TEST_USERNAME=$TEST_USERNAME TEST_BASE_URL=$TEST_BASE_URL yarn cy:run"

Example from workflow (test.yaml):

env: 
    CYPRESS_grepExcludeTags: "@jenkins"
    CYPRESS_grepFilterSpecs: "true"
    CYPRESS_grepOmitFiltered: "true"

I'm open to suggestions if anyone has ideas on why the exclusion logic isn't working correctly for @standardUser

@rancher-max @izaac

[izaac]

@yonasberhe23 is there anything that could help using the debug mode?
https://github.com/cypress-io/cypress/tree/develop/npm/grep#debugging

[yonasberhe23]

everything checks out there.

one thing I haven't tried is upgrading to the latest @cypress/grep version. we're currently on 3.1.5 and latest is 4.1.0. I'll try that

[yonasberhe23]

Upgrading to the latest @cypress/grep version did not help.

[yonasberhe23]

Imo I think we're following the Cypres documentation for tag-based filtering correctly so there may be an issue with the way Cypress is processing the tags. There's not much we can do from here.

@IsaSih at this point I suggest that you proceed with rancher/dashboard#12934 (comment)

[yonasberhe23]

Closing this out as we have a document describing this issue and a workaround. Also, I've opened a feature improvement ticket in Cypress to address the root issue.
see confluence doc at Cypress+Grep+Filtering+Issues+and+Workarounds