-
Notifications
You must be signed in to change notification settings - Fork 211
chore(ci): fix error handling & add timeout #3835
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Sometimes the log collection is stuck
|
/approve |
|
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: gustavolira The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
The image is available at: |
|
/test e2e-ocp-helm |
8db5ff1
into
redhat-developer:main



Description
This PR fixes CI/CD pipeline error handling issues that were causing:
kubectl logscommands to hang indefinitely when pods were unresponsiveSee this log, where the issues happened: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/redhat-developer_rhdh/3830/pull-ci-redhat-developer-rhdh-main-e2e-ocp-helm/1998787878628364288/artifacts/e2e-ocp-helm/redhat-developer-rhdh-ocp-helm/build-log.txt
Root Cause Analysis
The CI script was configured with
set -o errexitandtrap cleanup EXIT INT ERR. When combined withpipefail(enabled byconfigure_external_postgres_db()), this caused:yarn playwright test | teewould fail the entire script when tests failed (due topipefail)ERRandEXITtraps fired on failureskubectl logshad no timeout, causing 40+ minute hangs when pods were stuckChanges
openshift-ci-tests.shEXITonly (removesINTandERR)EXITtrap fires exactly once on any termination, preventing duplicate cleanuputils.shretrieve_pod_logs(): Added 30-second timeout tokubectl logscommands to prevent hangingconfigure_external_postgres_db(): Removed unnecessaryset -euo pipefailthat was leaking globallyExpected Behavior
kubectl logshangsWhich issue(s) does this PR fix
PR acceptance criteria
How to test changes / Special notes to the reviewer