Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FLAKE e2e (2)] Operator Group [It] insufficient permissions resolve via RBAC #3091

Closed
tmshort opened this issue Nov 6, 2023 · 3 comments
Closed

[FLAKE e2e (2)] Operator Group [It] insufficient permissions resolve via RBAC #3091

tmshort opened this issue Nov 6, 2023 · 3 comments
Labels
kind/flake Categorizes issue or PR as related to a flaky test.

Comments

@tmshort
Copy link
Contributor

tmshort commented Nov 6, 2023

Flaky Test Report

Also; too many similar/identical logs.

Failure Log Link
Failure Log

Relevant Failure Log

------------------------------
• [FAILED] [302.403 seconds]
Operator Group [It] insufficient permissions resolve via RBAC
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/operator_groups_e2e_test.go:1612

  Timeline >>
  created the operator-group-e2e-m679s testing namespace
  14:59:21.6505: Creating CRD
  14:59:21.6708: Creating operator group
  14:59:21.7011: Creating CSV
  14:59:21.8668: wait for CSV to fail
  14:59:21.9798:  (): 
  waited 9.38511127s for CSV operator-group-e2e-m679s-zr5ms/another-csv-jwr88: to be in phases [Failed], in phase  (): 
  14:59:22.0721:  (): 
  14:59:22.1724:  (): 
  14:59:22.282:  (): 
  14:59:22.3772:  (): 
  14:59:22.473:  (): 
  14:59:22.5726:  (): 
  14:59:22.7073:  (): 
  14:59:22.9107:  (): 
  14:59:23.1086: InstallReady (AllRequirementsMet): all requirements found, attempting install
  waited 1.128787182s for CSV operator-group-e2e-m679s-zr5ms/another-csv-jwr88: to be in phases [Failed], in phase InstallReady (AllRequirementsMet): all requirements found, attempting install
  14:59:23.3095: Failed (InstallComponentFailedNoRetry): install strategy failed: install strategy failed: deployments.apps "operator-deploymentzkkxr" is forbidden: User "system:serviceaccount:operator-group-e2e-m679s-zr5ms:nginx-sansp5b" cannot get resource "deployments" in API group "apps" in the namespace "operator-group-e2e-m679s-zr5ms"
  waited 200.854629ms for CSV operator-group-e2e-m679s-zr5ms/another-csv-jwr88: to be in phases [Failed], in phase Failed (InstallComponentFailedNoRetry): install strategy failed: install strategy failed: deployments.apps "operator-deploymentzkkxr" is forbidden: User "system:serviceaccount:operator-group-e2e-m679s-zr5ms:nginx-sansp5b" cannot get resource "deployments" in API group "apps" in the namespace "operator-group-e2e-m679s-zr5ms"
  14:59:23.3211: wait for CSV to succeeed
  14:59:23.5102: Failed (InstallComponentFailedNoRetry): install strategy failed: install strategy failed: deployments.apps "operator-deploymentzkkxr" is forbidden: User "system:serviceaccount:operator-group-e2e-m679s-zr5ms:nginx-sansp5b" cannot get resource "deployments" in API group "apps" in the namespace "operator-group-e2e-m679s-zr5ms"
  waited 5.402819873s for CSV operator-group-e2e-m679s-zr5ms/another-csv-jwr88: to be in phases [Succeeded], in phase Failed (InstallComponentFailedNoRetry): install strategy failed: install strategy failed: deployments.apps "operator-deploymentzkkxr" is forbidden: User "system:serviceaccount:operator-group-e2e-m679s-zr5ms:nginx-sansp5b" cannot get resource "deployments" in API group "apps" in the namespace "operator-group-e2e-m679s-zr5ms"
  14:59:23.7102: Failed (InstallComponentFailedNoRetry): install strategy failed: install strategy failed: deployments.apps "operator-deploymentzkkxr" is forbidden: User "system:serviceaccount:operator-group-e2e-m679s-zr5ms:nginx-sansp5b" cannot get resource "deployments" in API group "apps" in the namespace "operator-group-e2e-m679s-zr5ms"
  14:59:23.9075: Failed (InstallComponentFailedNoRetry): install strategy failed: install strategy failed: deployments.apps "operator-deploymentzkkxr" is forbidden: User "system:serviceaccount:operator-group-e2e-m679s-zr5ms:nginx-sansp5b" cannot get resource "deployments" in API group "apps" in the namespace "operator-group-e2e-m679s-zr5ms"
  14:59:24.1069: Failed (InstallComponentFailedNoRetry): install strategy failed: install strategy failed: deployments.apps "operator-deploymentzkkxr" is forbidden: User "system:serviceaccount:operator-group-e2e-m679s-zr5ms:nginx-sansp5b" cannot get resource "deployments" in API group "apps" in the namespace "operator-group-e2e-m679s-zr5ms"

... (lots of repeated lines)

  15:04:23.3076: Failed (InstallComponentFailedNoRetry): install strategy failed: install strategy failed: deployments.apps "operator-deploymentzkkxr" is forbidden: User "system:serviceaccount:operator-group-e2e-m679s-zr5ms:nginx-sansp5b" cannot get resource "deployments" in API group "apps" in the namespace "operator-group-e2e-m679s-zr5ms"
  15:04:23.5087: Failed (InstallComponentFailedNoRetry): install strategy failed: install strategy failed: deployments.apps "operator-deploymentzkkxr" is forbidden: User "system:serviceaccount:operator-group-e2e-m679s-zr5ms:nginx-sansp5b" cannot get resource "deployments" in API group "apps" in the namespace "operator-group-e2e-m679s-zr5ms"
  15:04:23.708: Failed (InstallComponentFailedNoRetry): install strategy failed: install strategy failed: deployments.apps "operator-deploymentzkkxr" is forbidden: User "system:serviceaccount:operator-group-e2e-m679s-zr5ms:nginx-sansp5b" cannot get resource "deployments" in API group "apps" in the namespace "operator-group-e2e-m679s-zr5ms"
  [FAILED] in [It] - /home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/operator_groups_e2e_test.go:1746 @ 11/06/23 15:04:23.858
  cleaning up ephemeral test resources...
  deleting test subscriptions...
  deleting test installplans...
  deleting test catalogsources...
  deleting test crds...
  deleting test csvs...
  test resources deleted
  << Timeline

  [FAILED] 
  	Error Trace:	/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/operator_groups_e2e_test.go:1746
  	            				/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/onsi/ginkgo/v2/internal/node.go:463
  	            				/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/onsi/ginkgo/v2/internal/suite.go:863
  	            				/opt/hostedtoolcache/go/1.20.10/x64/src/runtime/asm_amd64.s:1598
  	Error:      	Received unexpected error:
  	            	timed out waiting for the condition
  	Test:       	Operator Group insufficient permissions resolve via RBAC
  
  In [It] at: /home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/operator_groups_e2e_test.go:1746 @ 11/06/23 15:04:23.858

  Full Stack Trace
    github.com/stretchr/testify/require.NoError({0x7fa3ac8bddb0, 0xc000cf5440}, {0x40901c0, 0xc0001ba160}, {0x0, 0x0, 0x0})
    	/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/vendor/github.com/stretchr/testify/require/require.go:1357 +0x96
    github.com/operator-framework/operator-lifecycle-manager/test/e2e.glob..func20.9()
    	/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/operator_groups_e2e_test.go:1746 +0x15b5
------------------------------
@tmshort tmshort added the kind/flake Categorizes issue or PR as related to a flaky test. label Nov 6, 2023
@tmshort tmshort self-assigned this Nov 6, 2023
@tmshort tmshort mentioned this issue Nov 6, 2023
Closed
11 tasks
@tmshort
Copy link
Contributor Author

tmshort commented Nov 6, 2023

Investigating, not sure this is a testing issue, the test is fairly simple and seems OK.

In the success case, the CSV gets updated due to a change in the service account (operatorgroup.go):

Requeuing CSV due to detected service account change

not due to a change in RBAC (operator.go). The failure case (about 25% of the time, or so), does not contain the above log message.

@tmshort
Copy link
Contributor Author

tmshort commented Nov 7, 2023 •
edited
Loading

The occurrence rate is about 25% or less.
EDIT: Success 10/10 times, so less than 10% occurrence.

@tmshort tmshort mentioned this issue Nov 30, 2023
Closed
@tmshort tmshort removed their assignment Dec 7, 2023
@tmshort
Copy link
Contributor Author

tmshort commented Nov 19, 2024

Closing after a year. Our flakes have much improved.

@tmshort tmshort closed this as completed Nov 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/flake Categorizes issue or PR as related to a flaky test.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@tmshort