Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed lcov error #771

Merged
merged 12 commits into from
May 24, 2024
Merged

Fixed lcov error #771

merged 12 commits into from
May 24, 2024

Conversation

ahcorde
Copy link
Contributor

@ahcorde ahcorde commented Apr 22, 2024

There is an error on CI https://ci.ros2.org/view/nightly/job/nightly_linux_coverage/2348/consoleText related with lcov, this should fix the problem

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 22, 2024

  • Linux Build Status

Copy link
Contributor

@clalancette clalancette left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor change, but otherwise looks good to me. Please make sure to run CI on both Noble and Jammy to double check this works for both.

ros2_batch_job/__main__.py Outdated Show resolved Hide resolved
ros2_batch_job/__main__.py Outdated Show resolved Hide resolved
@Crola1702
Copy link
Contributor

Could this solve: #770?

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 22, 2024

@Crola1702 yes, It's a fix for this issue

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 22, 2024

  • Linux Build Status

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 23, 2024

  • Linux Build Status

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 23, 2024

  • Linux Build Status

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 23, 2024

  • Linux Build Status

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 24, 2024

  • Linux Build Status

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 25, 2024

  • Noble Linux Build Status
  • Jammy Linux Build Status

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 25, 2024

@clalancette I think i fixed this build, coverage is here https://ci.ros2.org/view/nightly/job/nightly_linux_coverage/2361/cobertura/.

Do you mind to take another look to the changes?

@ahcorde ahcorde requested a review from clalancette April 25, 2024 20:31
@clalancette
Copy link
Contributor

So I'm not totally sure about this, but this may not be producing correct results.

In particular, if you look at:

https://ci.ros2.org/view/nightly/job/nightly_linux_iron_coverage/408/cobertura/src_ros2_rcutils_src/allocator_c/allocator_c/ , you'll see that everything looks like we expect.

But if you look at https://ci.ros2.org/view/nightly/job/nightly_linux_coverage/2361/cobertura/src_ros2_rcutils_src/allocator_c/allocator_c/ (from the CI here), you'll see some functions with numbers in front of them, like 104,rcutils_allocator_is_valid.

I have no idea if that is what lcov_corbertura_fix was fixing, but it may be. Either way, what this is producing right now doesn't seem right. I guess we should be able to pull lcov_corbertura_fix from pypi, and see what it is doing differently to lcov_corbertura.

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 25, 2024

I have no idea if that is what lcov_corbertura_fix was fixing, but it may be. Either way, what this is producing right now doesn't seem right. I guess we should be able to pull lcov_corbertura_fix from pypi, and see what it is doing differently to lcov_corbertura.

The problem with lcov_corbertura_fix is that is not working, ant the Github repository doesn't exist anymore. I will try to find more details about this

@clalancette
Copy link
Contributor

The problem with lcov_corbertura_fix is that is not working, ant the Github repository doesn't exist anymore. I will try to find more details about this

Yeah, understood. We can at least pull the code from https://pypi.org/project/lcov-cobertura-fix/#files , and compare it to a "pristine" lcov_corbertua to see what the differences are.

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 26, 2024

  • Linux Build Status

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 29, 2024

  • Linux Build Status

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 29, 2024

  • Linux Build Status

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 29, 2024

Coverages are more simular right now

But I can see this error now:

13:30:14 [Cobertura] Publishing Cobertura coverage report...
13:30:14 
13:30:15 [Cobertura] Publishing Cobertura coverage results...
13:30:15 
13:30:16 [Cobertura] Cobertura coverage report found.
13:30:16 
13:30:16 ERROR: ERROR: Failure to paint /home/jenkins-agent/workspace/nightly_linux_coverage/ws/src/ros2/launch/launch/launch/utilities/normalize_to_list_of_substitutions_impl.py to /var/lib/jenkins/jobs/nightly_linux_coverage/cobertura
13:30:16 java.io.IOException: Failed to deserialize response to UserRequest:hudson.FilePath$WritePipe@2dc6ceb6: java.lang.SecurityException: Agent may not access a file path. See the system log for more details about the error ID '62b5d3e7-a5ae-41d2-8a81-d5d411ec39e1' and https://www.jenkins.io/redirect/security-144 for more information.

any thoughts?

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 29, 2024

related PR eriwen/lcov-to-cobertura-xml#55

@ahcorde
Copy link
Contributor Author

ahcorde commented Apr 30, 2024

ERROR: ERROR: Failure to paint /home/jenkins-agent/workspace/nightly_linux_coverage/ws/src/ros2/launch/launch/launch/utilities/normalize_to_list_of_substitutions_impl.py to /var/lib/jenkins/jobs/nightly_linux_coverage/cobertura

@sloretz
Copy link
Contributor

sloretz commented May 10, 2024

@clalancette assigning you as it looks like @ahcorde is awaiting your thoughts 🧇

@@ -144,7 +145,7 @@ RUN if test ${COMPILE_WITH_CLANG} = true; then apt-get update && apt-get install
# Install coverage build dependencies.
RUN apt-get update && apt-get install --no-install-recommends -y lcov

RUN pip3 install -U lcov_cobertura_fix
RUN pip3 install -U lcov_cobertura && rm /usr/local/lib/python3.12/dist-packages/lcov_cobertura/lcov_cobertura.py && wget https://raw.githubusercontent.com/ahcorde/lcov-to-cobertura-xml/master/lcov_cobertura/lcov_cobertura.py -P /usr/local/lib/python3.12/dist-packages/lcov_cobertura/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can move forward with a fork of this fix for now, but we should do that from an repository. We should also add in a comment here explaining exactly why we are doing that.

Let me discuss today with some people to decide where we should put this fork for now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I talked to @nuclearsandwich about this, and I got a much better idea of what we should do here. I think the easiest thing is to just use your fork for now, since you already have a fork. In that case, I think this should be:

Suggested change
RUN pip3 install -U lcov_cobertura && rm /usr/local/lib/python3.12/dist-packages/lcov_cobertura/lcov_cobertura.py && wget https://raw.githubusercontent.com/ahcorde/lcov-to-cobertura-xml/master/lcov_cobertura/lcov_cobertura.py -P /usr/local/lib/python3.12/dist-packages/lcov_cobertura/
# There is a bug in upstream lcov-to-cobertura-xml with newer versions of lcov. There is an
# open PR for it in https://github.com/eriwen/lcov-to-cobertura-xml/pull/55, but until that is
# merged use the fork with the fix.
RUN pip3 install -U git+https://github.com/ahcorde/lcov-to-cobertura-xml@master

Comment on lines +288 to +292
lcov_arguments = ['--ignore-errors', 'inconsistent,inconsistent',
'--ignore-errors', 'mismatch,mismatch',
'--ignore-errors', 'negative,negative',
'--ignore-errors', 'unused,unused',
'--ignore-errors', 'empty,empty']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not against this, but can you explain more why we need to ignore all of these errors? I think it would be best to have a comment here with that, so future readers know why.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just launching the CI and fixing the errors.

@@ -144,7 +145,7 @@ RUN if test ${COMPILE_WITH_CLANG} = true; then apt-get update && apt-get install
# Install coverage build dependencies.
RUN apt-get update && apt-get install --no-install-recommends -y lcov

RUN pip3 install -U lcov_cobertura_fix
RUN pip3 install -U lcov_cobertura && rm /usr/local/lib/python3.12/dist-packages/lcov_cobertura/lcov_cobertura.py && wget https://raw.githubusercontent.com/ahcorde/lcov-to-cobertura-xml/master/lcov_cobertura/lcov_cobertura.py -P /usr/local/lib/python3.12/dist-packages/lcov_cobertura/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I talked to @nuclearsandwich about this, and I got a much better idea of what we should do here. I think the easiest thing is to just use your fork for now, since you already have a fork. In that case, I think this should be:

Suggested change
RUN pip3 install -U lcov_cobertura && rm /usr/local/lib/python3.12/dist-packages/lcov_cobertura/lcov_cobertura.py && wget https://raw.githubusercontent.com/ahcorde/lcov-to-cobertura-xml/master/lcov_cobertura/lcov_cobertura.py -P /usr/local/lib/python3.12/dist-packages/lcov_cobertura/
# There is a bug in upstream lcov-to-cobertura-xml with newer versions of lcov. There is an
# open PR for it in https://github.com/eriwen/lcov-to-cobertura-xml/pull/55, but until that is
# merged use the fork with the fix.
RUN pip3 install -U git+https://github.com/ahcorde/lcov-to-cobertura-xml@master

@ahcorde
Copy link
Contributor Author

ahcorde commented May 21, 2024

  • Linux Build Status

@ahcorde ahcorde requested a review from clalancette May 21, 2024 16:30
Copy link
Contributor

@clalancette clalancette left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more change. Once that is in, I'm going to suggest we run the coverage jobs for all of Humble, Iron, Jazzy, and Rolling to ensure that things are working across all of them.

Comment on lines 128 to 129
yamllint \
wget
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need this change anymore, since we aren't using wget anymore.

@ahcorde
Copy link
Contributor Author

ahcorde commented May 22, 2024

  • Linux rolling Build Status
  • Linux jazzy Build Status
  • Linux iron Build Status
  • Linux humble Build Status

ahcorde added 12 commits May 24, 2024 08:40
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Alejandro Hernández Cordero <[email protected]>
@clalancette
Copy link
Contributor

Jazzy CI failed because this needed to be rebased against the latest code (which I've now done). Here is another round:

  • Linux rolling Build Status
  • Linux jazzy Build Status
  • Linux iron Build Status
  • Linux humble Build Status

@clalancette
Copy link
Contributor

All right, this now looks good to me. Since the PR still hasn't merged upstream, let's go with Alex's fork for now, and then when it is eventually merged and released upstream we can use that.

@clalancette clalancette merged commit 5d3807e into master May 24, 2024
1 check passed
@delete-merged-branch delete-merged-branch bot deleted the ahcorde/fix/lcov branch May 24, 2024 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants