This repository has been archived by the owner on Feb 4, 2021. It is now read-only.
Upgrade Mac and Windows CI hosts #220
Assignees
Labels
Comments
|
I've begun upgrading portable (Windows) and will upgrade mini1 (macOS) as soon as it finishes. I'll post the changes in versions here and test CI when I'm done. I'm following the upgrade notes from here: #74 |
|
Upgrade of
Dump of Specific upgrades:
Custom choco packages were all update with https://github.com/ros2/choco-packages/releases/tag/2019-02-15-1. Python upgrades (some are overloaded by the CI job's virtualenv, but useful for local builds):
DDS Vendors:
CI: (I'll update this comment with more information if this isn't everything I end up doing) |
|
Upgrade of
Dump of
```
mini1:~ osrf$ brew list --versions
aom 1.0.0
apr 1.7.0
apr-util 1.6.1_1
asciidoc 8.6.10_2
asio 1.12.2
assimp 4.1.0
autoconf 2.69
automake 1.16.1_1
boost 1.69.0_2
cairo 1.16.0
ccache HEAD-974c4eb
cmake 3.14.4
console_bridge 0.4.3
cppcheck 1.87
docbook 5.0
docbook-xsl 1.79.1
eigen 3.3.7
ffmpeg 4.1.3_1
flac 1.3.2_1
fontconfig 2.13.1
freetype 2.10.0
frei0r 1.6.1
fribidi 1.0.5
gawk 5.0.0
gcc 9.1.0
gd 2.2.5
gdbm 1.18.1
gettext 0.19.8.1
gflags 2.2.2
giflib 5.1.4_1
glib 2.60.3
glog 0.4.0
gmp 6.1.2_2
gnutls 3.6.7.1
graphite2 1.3.13
graphviz 2.40.1
gtest 1.7.0
harfbuzz 2.4.0_1
icu4c 64.2
ilmbase 2.3.0
isl 0.21
jpeg 9c
lame 3.100
leptonica 1.78.0
libass 0.14.0_1
libbluray 1.1.1
libevent 2.1.8
libffi 3.2.1
libidn2 2.1.1a
libmpc 1.1.0
libogg 1.3.3
libpcl 1.12
libpng 1.6.37
libsamplerate 0.1.9_1
libsndfile 1.0.28
libsoxr 0.1.3
libtasn1 4.13
libtiff 4.0.10_1
libtool 2.4.6_1
libunistring 0.9.10
libvorbis 1.3.6
libvpx 1.8.0
little-cms2 2.9
log4cxx 0.10.0_1
mercurial 5.0
mpfr 4.0.2
nettle 3.4.1
numpy 1.16.3_1
openblas 0.3.6_1
opencore-amr 0.1.5
opencv 4.1.0_2
openexr 2.3.0
openjpeg 2.3.1
openssl 1.0.2r
[email protected] 1.1.1b
opus 1.3.1
p11-kit 0.23.15
pcre 8.43
pixman 0.38.4
pkg-config 0.29.2
poco 1.9.0_1
pyqt 5.10.1_1
python 3.7.3
python@2 2.7.16
qt 5.12.3
readline 8.0.0_1
rtmpdump 2.4+20151223
rubberband 1.8.2_1
sdl2 2.0.9_1
sip 4.19.8_11
snappy 1.1.7_1
source-highlight 3.1.8_11
speex 1.2.0
sqlite 3.28.0
ssh-askpass 1.2.1
tbb 2019_U6
tesseract 4.0.0_1
theora 1.1.1
tinyxml 2.6.2
tinyxml2 7.0.1
[email protected] 6.2.0
unbound 1.9.1_1
uncrustify 0.68.1
webp 1.0.2
wget 1.20.3
x264 r2917
x265 3.0
xvid 1.3.5
xz 5.2.4
```
Upgrading with
CI: (I'll update this comment with more information if this isn't everything I end up doing) |
|
Looks like cppcheck is failing to complete in an appropriate amount of time since upgrading to 1.88. I'm looking for regressions, @dirk-thomas pointed out danmar/cppcheck#2046, which might be related. |
|
@wjwwood - we started seeing cppcheck fail when processing a particular template in rclcpp on the CentOS builds on 7/18. I bisected cppcheck and found two patches necessary for cppcheck to run to completion on that test case. Both have been applied to the cppcheck RPM for Fedora and EPEL, and are in the testing repositories. These are the two patches we applied:
As far as I know, these patches only address the memory consumption problem which results in the kernel killing cppcheck due to OOM. I haven't investigated any problems related to runtime performance regression. The smallest repro of the memory issue that I can find is: As part of ros2/ci#314, I triggered a CI job with a prerelease of cppcheck with those two patches applied: |
|
I ran cppcheck of a few versions on rclcpp to test out things: Basically it breaks down like this:
Regardless, the timeout may need to be extended. |
|
I opened ament/ament_lint#168 as a possible solution to the cppcheck issues that wouldn't require us to modify the installation instructions or anything. |
|
CI on the upgraded build machines with the
|
|
We may have already known this, but Crystal fails to build on the upgraded OpenSplice like so: |
I assume the build tries to build the latest release which doesn't include this backport yet ros2/rosidl_typesupport_opensplice#36 ? |
|
Yes, it's the latest release as opposed to the Crystal "devel" branches. I can rerun it with the other in a little while. |
|
Windows looks good, it has the same three failures as the nightly:
I'll run one more with my changes to the cppcheck branch, just to make sure, but I think it's good to go:
For macos (mini1) I'm looking into new compiler warnings still to see if they're preexisting or not. |
|
I upgraded OpenSplice to the newer version on mini1 too, since there's a new binary now. |
|
The new warnings on macOS are due to new deprecations in Qt. I'm looking into those still, but one is looking to be non-trivial. I suppose we could move forward with that warning though. |
|
Here's |
|
If I have time tonight I may go ahead and upgrade the rest of the mac and Windows machines so we have full capacity for tomorrow. Depend on how the crystal job goes. I may do it anyways since changing the opensplice version is relatively easy on both macOS and Windows. So I could revert the version just for Crystal on one machine of each kind if needed. |
|
I'm not sure what happened in the crystal build (https://ci.ros2.org/job/test_ci_windows/52/console), but it looks like that branch just doesn't build right now (looks like a typo in a C++ test). But I think it got past the OpenSplice issue. |
|
Upgrade of
Dump of Specific upgrades:
Custom choco packages were all update with https://github.com/ros2/choco-packages/releases/tag/2019-02-15-1. Python upgrades (some are overloaded by the CI job's virtualenv, but useful for local builds):
DDS Vendors:
CI: |
|
Upgrade of
Dump of Specific upgrades:
Custom choco packages were all update with https://github.com/ros2/choco-packages/releases/tag/2019-02-15-1. Python upgrades (some are overloaded by the CI job's virtualenv, but useful for local builds):
DDS Vendors:
CI: |
|
Upgrade of
Dump of Upgrading with
CI:
|
|
Upgrade for
Dump of Upgrading with
CI: |
|
I brought windshield back into the worker pool (along with portable), but there are extra test failures on icecube that need resolving. |
|
Ok, there's something wrong with mini2, it has disconnected four times during my test ci, which was the last step before I re-added it to the worker pool. I guess, I'll have to debug that next week sometime. Does anyone have ideas? |
|
Just the usual suspects: Jenkins agent/slave version, Java version... |
|
The VNC has been exceptionally laggy on I tried switching it to WiFi (disabling the Ethernet) and the VNC looks a lot more responsive. I'm going to try another build: Maybe the Ethernet adapter in |
|
Even on wifi However, I tooled around some more and I figured out why it wouldn't update from the java settings panel, it was because there was a JRE and JDK and the JDK was overriding it. Even after manually updating the JRE, the JDK seemed to overshadow it causing |
|
That job also failed due to a disconnect, here's the automatically relaunched one: I'm at a loss for what to do next. I reinstalled the agent as part of my Java updating fiasco. I even re-downloaded the java agent according to these instructions: https://index.ros.org/doc/ros2/Contributing/Set-up-a-new-macOS-CI-node/ Should I leave it off the farm or add it back? (it did pass at least once after requeueing it automatically) Any other ideas what to do next? |
|
I'm bringing |
|
So with that, I think all macOS and Windows machines have been updated for Dashing, and therefore this issue can be closed. What remains is related to Crystal and the updated OpenSplice version. I'll make a new issue for that: #221 |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
In order to produce usable binaries for Dashing Patch Release 2 this needs to be completed ahead of time, ideally with time to stabilize after any potential issues.
I've got the checklist I've used to do this at https://gist.github.com/nuclearsandwich/63ef5a2868ae8a7a6167283789e1a71e
The text was updated successfully, but these errors were encountered: