Add kola-junit script to generate JUnit XML files
#3667
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jlebon
added a commit
to jlebon/coreos-ci-lib
that referenced
this pull request
Nov 27, 2023
This will allow us to much more easily inspect failing kola tests. Goes together with coreos/coreos-assembler#3667, which has more details. While we're here, use `kola-testiso` as marker since it looks clearer and for consistency with `kola-upgrade`.
jlebon
added a commit
to jlebon/coreos-ci-lib
that referenced
this pull request
Nov 27, 2023
This will allow us to much more easily inspect failing kola tests. Goes together with coreos/coreos-assembler#3667, which has more details. While we're here, use `kola-testiso` as marker since it looks clearer and for consistency with `kola-upgrade`.
|
You can see what it looks like in https://jenkins-coreos-ci.apps.ocp.fedoraproject.org/blue/organizations/jenkins/jlebon-tmp2/detail/jlebon-tmp2/12/tests And screenshot for posterity for when that job will be wiped.
|
dustymabe
reviewed
Nov 28, 2023
|
It would be great in this new page view if we could see the stdout from |
dustymabe
reviewed
Nov 28, 2023
Prep for reading this.
jlebon
added a commit
to jlebon/coreos-ci-lib
that referenced
this pull request
Dec 1, 2023
This will allow us to much more easily inspect failing kola tests. Goes together with coreos/coreos-assembler#3667, which has more details. While we're here, use `kola-testiso` as marker since it looks clearer and for consistency with `kola-upgrade`.
There's a lot of friction associated with debugging pipeline kola test failures. One must scroll a bunch and sift through the output to see which test(s) failed, and if more information is needed, download the archived artifacts, extract it, and look for the journal and console there. Let's make this easier by outputting JUnit XML files. Jenkins can natively display test results in that format so it's much easier to see which tests ran, which ones failed, and their outputs (in the Blue Ocean view, it's the "Tests" tab which we haven't been using so far). For space efficiency and to reduce noise, we default to skipping over passed tests.
While we carry patches, let's freeze the osbuild version so that the patches can be updated atomically with the version freeze, avoiding temporarily breaking builds.
|
Updated for comments and added a freeze on osbuild as discussed in #3671! (Fine to break that out into its own PR if you'd prefer.)
That actually is possible. It was already the case for non-external tests, but now I've tweaked it so we retain the information for external tests too. |
dustymabe
reviewed
Dec 1, 2023
jlebon
added a commit
to coreos/coreos-ci-lib
that referenced
this pull request
Dec 4, 2023
This will allow us to much more easily inspect failing kola tests. Goes together with coreos/coreos-assembler#3667, which has more details. While we're here, use `kola-testiso` as marker since it looks clearer and for consistency with `kola-upgrade`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There's a lot of friction associated with debugging pipeline kola test
failures. One must scroll a bunch and sift through the output to see
which test(s) failed, and if more information is needed, download the
archived artifacts, extract it, and look for the journal and console
there.
Let's make this easier by outputting JUnit XML files. Jenkins can
natively display test results in that format so it's much easier to see
which tests ran, which ones failed, and their outputs (in the Blue Ocean
view, it's the "Tests" tab which we haven't been using so far).
For space efficiency and to reduce noise, we default to skipping over
passed tests.