Skip to content
This repository has been archived by the owner on Feb 4, 2021. It is now read-only.

2017-12-02: Update nodes before release #74

Closed
mikaelarguedas opened this issue Dec 3, 2017 · 4 comments
Closed

2017-12-02: Update nodes before release #74

mikaelarguedas opened this issue Dec 3, 2017 · 4 comments

Comments

@mikaelarguedas
Copy link
Member

Summary of the updates that happened and how I performed them for book-keeping.

Nodes updates:

  • mini1
  • mini2
  • dosa (yeah cmake 3.10 was released after the reinstall a few days ago...)
  • eatable (offline so no point in updating it before reinstall)
  • icecube
  • portable
  • windshield
  • linux2
  • linux3

Updating nodes

For each node:

  • take the node offline
  • update its label to test_<OS> (OS being: linux, linux-aarch64, osx of windows
  • perform update (see sections below for the update steps I followed for each platform)
  • reboot the machine
  • bring node back online
  • run test jobs: test_ci_<OS> and test_packaging_<OS> (for windows I ran debug jobs as we build our own custom python debug interpreter)
  • if all the tests come back gree, reenable the node by reassigning it previous label (linux,linux-aarch64, osx_slave or windows_slave)

Update a Linux node:

We dont need to update much as everything runs in docker.
Perform the update:

sudo apt-get update
sudo apt-get list --upgradable
# if nothing looks risky upgrade everything
sudo apt-get upgrade

Full log at the end of this issue.

Update a MacOS node:

(Steps copied from #38 (comment))

  • vnc in to see if there are updates (open App Store app), do them if there are
  • ssh or vnc in to terminal and do brew update (this updated brew's definitions, like apt update)
  • do brew outdated to see what can be upgraded (note these down for book-keeping)
  • either do brew upgrade to upgrade all or do selective ones with brew upgrade [[[pkg1] pkg2] ...]
  • check if java version is up-to-date and update it if it's not (:warning: looks to be different on High Sierra):
    • click Apple icon top-left corner
    • System Preferences
    • Java
    • Click on update tab
    • Update now
  • reboot

Update a Windows node:

  • vnc in
  • System update
    • hit windows key and type "updates"
    • select "Check for Updates"
    • if any update install them and reboot to apply them
  • Java
    • hit windows key and type "java"
    • select "check for updates"
    • click update now
  • Chocolatey packages updates
    • open a new command prompt with administrator permissions
    • upgrade chocolatey packages one by one:
choco upgrade -y python

Pay extra attention as if Python is updated: if the version changes you'll need to get a matching version of the python debug interpreter (see below for details)

choco upgrade -y cmake
choco upgrade -y curl
choco upgrade -y cppcheck
choco upgrade -y peazip
choco upgrade -y hg
choco upgrade -y svn
  • for each one note down previous and new version for book keeping
  • Python packages:
    • again write down the version changes for book-keeping
pip install -U setuptools pip
pip install -U EmPy pyparsing pyyaml
pip install -U nose coverage mock pytest pytest-cov pytest-runner
pip install -U flake8 flake8-blind-except flake8-builtins flake8-class-newline flake8-comprehensions flake8-deprecated flake8-docstrings flake8-import-order flake8-quotes pep8 pydocstyle pyflakes
pip install -U trollius
pip install -U vcstool
  • Python debug interpreter:
    • If the version of Python change you need to compile the debug version and install it on the system
    • Download the python source code matching your version from https://www.python.org/ftp/python/<VERSION_NUMBER>/Python-<VERSION_NUMBER>.tgz
      • This time the new version was 3.6.3 so I'll use this in the rest of these instructions.
        https://www.python.org/ftp/python/3.6.3/Python-3.6.3.tgz
    • Extract it at C:\dev\C:\dev\Python-3.6.3
    • Build the code:
cd C:\dev\Python-3.6.3\PCbuild
get_externals.bat
build.bat -p x64 -d
  • Copy the produced binary in the python install directory:
    • Note that at the time of the writing chocolatey install Python at C:\Python
cd C:\dev\Python-3.6.3\PCbuild\amd64
copy python_d.exe C:\Python36 /Y
copy python36_d.dll C:\Python36 /Y
copy python3_d.dll C:\Python36 /Y
copy python36_d.lib C:\Python36\libs /Y
copy python3_d.lib C:\Python36\libs /Y
for %I in (*_d.pyd) do copy %I C:\Python36\DLLs /Y
  • Test the python Debug interpreter (in a new shell):
python_d

Confirm that the version number is the one you expect

import _ctypes

To make sure that the built python debug extensions can be imported

  • Update OpenSSL (copied from these instructions):
    • Go to https://slproweb.com/products/Win32OpenSSL.html
    • Download the last Win64 OpenSSL v1.0.2 version (1.0.2m ATM). Don't download the Win32 or Light versions
    • Run the installer with default parameter (just hit next until it stops asking questions)
  • reboot
    Hooray. Time to run a test job. Make sure to run a Debug job to test the python Debug interpreter.

Versions updated this cycle:

Linux

Trusty (linux 2)

The following packages will be upgraded:
  apport curl libcurl3 libcurl3-gnutls libdb5.3 libpython2.7
  libpython2.7-minimal libpython2.7-stdlib libpython3.4-minimal
  libpython3.4-stdlib libssl1.0.0 libxcursor1 openssl perl perl-base
  perl-modules python2.7 python2.7-minimal python3-apport
  python3-problem-report python3.4 python3.4-minimal resolvconf tzdata
  • perl (5.18.2-2ubuntu1.3) over (5.18.2-2ubuntu1.1) ...
  • perl-base (5.18.2-2ubuntu1.3) over (5.18.2-2ubuntu1.1) ...
  • perl-modules (5.18.2-2ubuntu1.3) over (5.18.2-2ubuntu1.1) ...
  • libdb5.3:amd64 (5.3.28-3ubuntu3.1) over (5.3.28-3ubuntu3) ...
  • python3.4 (3.4.3-1ubuntu114.04.6) over (3.4.3-1ubuntu114.04.5) ...
  • python3.4-minimal (3.4.3-1ubuntu114.04.6) over (3.4.3-1ubuntu114.04.5) ...
  • libpython3.4-stdlib:amd64 (3.4.3-1ubuntu114.04.6) over (3.4.3-1ubuntu114.04.5) ...
  • libssl1.0.0:amd64 (1.0.1f-1ubuntu2.23) over (1.0.1f-1ubuntu2.22) ...
  • libpython3.4-minimal:amd64 (3.4.3-1ubuntu114.04.6) over (3.4.3-1ubuntu114.04.5) ...
  • resolvconf (1.69ubuntu1.3) over (1.69ubuntu1.1) ...
  • libcurl3-gnutls:amd64 (7.35.0-1ubuntu2.13) over (7.35.0-1ubuntu2.12) ...
  • curl (7.35.0-1ubuntu2.13) over (7.35.0-1ubuntu2.12) ...
  • libcurl3:amd64 (7.35.0-1ubuntu2.13) over (7.35.0-1ubuntu2.12) ...
  • python2.7 (2.7.6-8ubuntu0.4) over (2.7.6-8ubuntu0.3) ...
  • libpython2.7:amd64 (2.7.6-8ubuntu0.4) over (2.7.6-8ubuntu0.3) ...
  • libpython2.7-stdlib:amd64 (2.7.6-8ubuntu0.4) over (2.7.6-8ubuntu0.3) ...
  • python2.7-minimal (2.7.6-8ubuntu0.4) over (2.7.6-8ubuntu0.3) ...
  • libpython2.7-minimal:amd64 (2.7.6-8ubuntu0.4) over (2.7.6-8ubuntu0.3) ...
  • libxcursor1:amd64 (1:1.1.14-1ubuntu0.14.04.1) over (1:1.1.14-1) ...
  • tzdata (2017c-0ubuntu0.14.04) over (2016j-0ubuntu0.14.04) ...
  • openssl (1.0.1f-1ubuntu2.23) over (1.0.1f-1ubuntu2.22) ...
  • python3-problem-report (2.14.1-0ubuntu3.27) over (2.14.1-0ubuntu3.25) ...
  • python3-apport (2.14.1-0ubuntu3.27) over (2.14.1-0ubuntu3.25) ...
  • apport (2.14.1-0ubuntu3.27) over (2.14.1-0ubuntu3.25) ...

Xenial (linux 3)

apt list --upgradable

  • bind9-host/xenial-updates 1:9.10.3.dfsg.P4-8ubuntu1.9 amd64 [upgradable from: 1:9.10.3.dfsg.P4-8ubuntu1.8]
  • cloud-init/xenial-updates 17.1-27-geb292c18-0ubuntu116.04.1 all [upgradable from: 0.7.9-233-ge586fe35-0ubuntu116.04.2]
  • dnsmasq-base/xenial-updates 2.75-1ubuntu0.16.04.4 amd64 [upgradable from: 2.75-1ubuntu0.16.04.3]
  • dnsutils/xenial-updates 1:9.10.3.dfsg.P4-8ubuntu1.9 amd64 [upgradable from: 1:9.10.3.dfsg.P4-8ubuntu1.8]
  • docker-ce/xenial 17.11.0ce-0ubuntu amd64 [upgradable from: 17.10.0ce-0ubuntu]
  • dpkg/xenial-updates 1.18.4ubuntu1.3 amd64 [upgradable from: 1.18.4ubuntu1.2]
  • grub-legacy-ec2/xenial-updates 17.1-27-geb292c18-0ubuntu116.04.1 all [upgradable from: 0.7.9-233-ge586fe35-0ubuntu116.04.2]
  • iproute2/xenial-updates 4.3.0-1ubuntu3.16.04.2 amd64 [upgradable from: 4.3.0-1ubuntu3.16.04.1]
  • klibc-utils/xenial-updates 2.0.4-8ubuntu1.16.04.4 amd64 [upgradable from: 2.0.4-8ubuntu1.16.04.3]
  • libbind9-140/xenial-updates 1:9.10.3.dfsg.P4-8ubuntu1.9 amd64 [upgradable from: 1:9.10.3.dfsg.P4-8ubuntu1.8]
  • libdns-export162/xenial-updates 1:9.10.3.dfsg.P4-8ubuntu1.9 amd64 [upgradable from: 1:9.10.3.dfsg.P4-8ubuntu1.8]
  • libdns162/xenial-updates 1:9.10.3.dfsg.P4-8ubuntu1.9 amd64 [upgradable from: 1:9.10.3.dfsg.P4-8ubuntu1.8]
  • libisc-export160/xenial-updates 1:9.10.3.dfsg.P4-8ubuntu1.9 amd64 [upgradable from: 1:9.10.3.dfsg.P4-8ubuntu1.8]
  • libisc160/xenial-updates 1:9.10.3.dfsg.P4-8ubuntu1.9 amd64 [upgradable from: 1:9.10.3.dfsg.P4-8ubuntu1.8]
  • libisccc140/xenial-updates 1:9.10.3.dfsg.P4-8ubuntu1.9 amd64 [upgradable from: 1:9.10.3.dfsg.P4-8ubuntu1.8]
  • libisccfg140/xenial-updates 1:9.10.3.dfsg.P4-8ubuntu1.9 amd64 [upgradable from: 1:9.10.3.dfsg.P4-8ubuntu1.8]
  • libklibc/xenial-updates 2.0.4-8ubuntu1.16.04.4 amd64 [upgradable from: 2.0.4-8ubuntu1.16.04.3]
  • liblwres141/xenial-updates 1:9.10.3.dfsg.P4-8ubuntu1.9 amd64 [upgradable from: 1:9.10.3.dfsg.P4-8ubuntu1.8]
  • libpam-systemd/xenial-updates 229-4ubuntu21 amd64 [upgradable from: 229-4ubuntu20]
  • libsystemd0/xenial-updates 229-4ubuntu21 amd64 [upgradable from: 229-4ubuntu20]
  • libudev1/xenial-updates 229-4ubuntu21 amd64 [upgradable from: 229-4ubuntu20]
  • lshw/xenial-updates 02.17-1.1ubuntu3.4 amd64 [upgradable from: 02.17-1.1ubuntu3.2]
  • lxcfs/xenial-updates 2.0.8-0ubuntu116.04.2 amd64 [upgradable from: 2.0.7-0ubuntu116.04.2]
  • mdadm/xenial-updates 3.3-2ubuntu7.6 amd64 [upgradable from: 3.3-2ubuntu7.4]
  • python3-distupgrade/xenial-updates 1:16.04.23 all [upgradable from: 1:16.04.22]
  • python3-update-manager/xenial-updates 1:16.04.10 all [upgradable from: 1:16.04.9]
  • resolvconf/xenial-updates 1.78ubuntu5 all [upgradable from: 1.78ubuntu4]
  • snapd/xenial-updates 2.28.5 amd64 [upgradable from: 2.27.5]
  • systemd/xenial-updates 229-4ubuntu21 amd64 [upgradable from: 229-4ubuntu20]
  • systemd-sysv/xenial-updates 229-4ubuntu21 amd64 [upgradable from: 229-4ubuntu20]
  • ubuntu-core-launcher/xenial-updates 2.28.5 amd64 [upgradable from: 2.27.5]
  • ubuntu-minimal/xenial-updates 1.361.1 amd64 [upgradable from: 1.361]
  • ubuntu-release-upgrader-core/xenial-updates 1:16.04.23 all [upgradable from: 1:16.04.22]
  • ubuntu-server/xenial-updates 1.361.1 amd64 [upgradable from: 1.361]
  • ubuntu-standard/xenial-updates 1.361.1 amd64 [upgradable from: 1.361]
  • udev/xenial-updates 229-4ubuntu21 amd64 [upgradable from: 229-4ubuntu20]
  • update-manager-core/xenial-updates 1:16.04.10 all [upgradable from: 1:16.04.9]

MacOS

  • asciidoc (8.6.9_1) < 8.6.10_1
  • cmake (3.6.2, 3.7.2, 3.8.2, 3.9.1, 3.9.2) < 3.10.0
  • cppcheck (1.75, 1.77, 1.79, 1.80) < 1.81
  • ffmpeg (3.3.3, 3.3.4) < 3.4
  • gawk (4.1.4_1) < 4.2.0
  • gmp (6.1.1, 6.1.2) < 6.1.2_1
  • libpng (1.6.25, 1.6.28, 1.6.29, 1.6.32) < 1.6.34
  • libtiff (4.0.6_2, 4.0.7_2, 4.0.8, 4.0.8_3, 4.0.8_4) < 4.0.8_5
  • mercurial (4.1.1, 4.2.1, 4.3.1) < 4.4.2
  • numpy (1.12.0, 1.13.0, 1.13.1_1) < 1.13.3
  • opencv (3.3.0_3) < 3.3.1_1
  • openssl (1.0.2j, 1.0.2k, 1.0.2l) < 1.0.2m
  • [email protected] (1.1.0f) < 1.1.0g
  • python (2.7.13, 2.7.13_1) < 2.7.14
  • python3 (3.5.2_2, 3.6.0, 3.6.1, 3.6.2) < 3.6.3
  • sqlite (3.14.2_1, 3.16.2, 3.17.0, 3.19.3, 3.20.1) < 3.21.0
  • wget (1.18, 1.19.1, 1.19.1_1) < 1.19.2

Windows

  • git: 2.14.1 -> 2.15.1.2
  • python: 3.6.2 -> 3.6.3
  • CMake: 3.9.1 -> 3.10.0
  • curl: 7.55.1 -> 7.57.0
  • setuptools: 28.8.0 -> 38.2.3
  • flake8-quotes: 0.12.0 -> 0.12.1
  • trollius: 2.1 (apparently was not installed before)
  • cppcheck 1.80.1 -> 1.81.0
  • peazip: 6.4.1 -> 6.5.0
@clalancette clalancette mentioned this issue Dec 11, 2017
Closed
11 tasks
@mikaelarguedas
Copy link
Member Author

closing this as all the nodes were update and the release is out

@mikaelarguedas
Copy link
Member Author

Several dependencies changed since ardent release,
mini1 has been updated and passes CI http://ci.ros2.org/view/All/job/test_ci_osx/43/. I'll wait for feedback, (particularly from @wjwwood) before upgrading the other nodes as some dependencies will impact rviz2 (assimp, qt etc).
Some of these changes prevent the ardent binaries to work out-of-the box for users: ros2/ros2#440 so we may want to consider rebuild ardent binaries on updated machines.

MacOS:

Package name Current versions New version
assimp 4.0.1 4.1.0
boost 1.63.0, 1.64.0_1, 1.65.0, 1.65.1 1.66.0
cmake 3.6.2, 3.7.2, 3.8.2, 3.9.1, 3.9.2, 3.10.0 3.10.1
ffmpeg 3.3.3, 3.3.4, 3.4 3.4.1
libtiff 4.0.6_2, 4.0.7_2, 4.0.8, 4.0.8_3, 4.0.8_4, 4.0.8_5 4.0.9
python3 3.5.2_2, 3.6.0, 3.6.1, 3.6.2, 3.6.3 3.6.4
qt 5.9.3 5.10.0
tbb 2018_U1 2018_U2
tinyxml2 4.0.1, 5.0.1 6.0.0
wget 1.18, 1.19.1, 1.19.1_1, 1.19.2 1.19.2_1

Windows:

Package name Current version New version Pinned
chocolatey-core.extension 1.3.1 1.3.3 false
cmake 3.10.0 3.10.1 false
cmake.install 3.10.0 3.10.1 false
git 2.14.1 2.15.1.2 false
git.install 2.14.1 2.15.1.2 false
hg 4.3.1 4.4.2 false
python 3.6.3 3.6.4 false
python3 3.6.3 3.6.4 false

Only issue so far: Python3.6.4 default build requires Windows SDK 10.0. Currently we don;t have the windows SDK installed. We could either modify the python build options to build without the SDK or install the SDK on the nodes and build with it. Note that other problem may come up once the SDK installed (ros2/common_interfaces#44, this refers to SDK 8.1 but I'd expect ismilar issues on windows 10)

@mikaelarguedas mikaelarguedas mentioned this issue Jan 3, 2018
Closed
7 tasks
@mikaelarguedas mikaelarguedas mentioned this issue Jan 22, 2018
Closed
3 tasks
@dirk-thomas dirk-thomas mentioned this issue Jan 30, 2018
Closed
4 tasks
@wjwwood
Copy link
Member

wjwwood commented Feb 2, 2018

Upgraded the macOS machines again in preparation for the Ardent patch 1:

==> Upgrading 20 outdated packages, with result:
ccache HEAD-b313a68, cmake 3.10.2, cppcheck 1.82, freetype 2.9, gawk 4.2.0_1, gdbm 1.14.1_1, lame 3.100, libtiff 4.0.9_1, llvm 5.0.1, mercurial 4.5, mpfr 4.0.0, numpy 1.14.0, opencv 3.4.0_1, [email protected] 1.1.0g_1, python 2.7.14_2, python3 3.6.4_2, qt 5.10.0_1, sqlite 3.22.0, wget 1.19.4_1, xvid 1.3.5

As well as the Windows machines:

  • python 3.6.3 -> 3.6.4.20180116
    • rebuilt python_d
  • cmake 3.10.1 -> 3.10.2
  • git 2.15.1.2 -> 2.16.1
  • curl 7.57.0 -> 7.58.0

To rebuild Python (3.6.4) I had to use psexec.exe -s -i cmd.exe in an administrative shell to get a SYSTEM user shell because msbuild would not run anywhere else (even an administrator prompt). I also needed to patch the MSVC files that come with that version of python, according to this bug report:

Everything else worked the same.

@wjwwood
Copy link
Member

wjwwood commented Feb 2, 2018 •
edited
Loading

Updating cppcheck also caused a new regression (on macOS):

http://ci.ros2.org/job/ci_osx/3263/testReport/junit/(root)/projectroot/cppcheck_2/

@wjwwood wjwwood mentioned this issue Feb 2, 2018
Merged
@wjwwood wjwwood mentioned this issue Jul 29, 2019
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wjwwood @mikaelarguedas