Update to ubuntu 20.04

.github/workflows/matrix.json

@@ -1,8 +1,8 @@
 {
   "include": [
     {
-      "DOCKER_TAG": "2023.12.0",
-      "OPERATING_SYSTEM_TAG": "18.04",
+      "DOCKER_TAG": "2024.03.0",
+      "OPERATING_SYSTEM_TAG": "20.04",
       "LLVM_VERSION_MAJOR": "14"
     }
   ]

DEVNOTE.md

@@ -11,8 +11,7 @@ needed dependencies such as Git, LLVM, GRPC, GoogleTest and others. **base_env**
 dates. If you are developing the tool, you are most likely to need the most recent version
 from [here](https://github.com/UnitTestBot/UTBotCpp/pkgs/container/utbotcpp%2Fbase_env).
 
-Supported and tested development configuration are Ubuntu 18.04 / 20.04 or Windows Subsystem for Linux (Ubuntu 18.04 /
-20.04).
+Supported and tested development configuration are Ubuntu 20.04 or Windows Subsystem for Linux (Ubuntu 20.04).
 
 1. Install docker for [Ubuntu](https://docs.docker.com/engine/install/ubuntu/)
    or [WSL+Ubuntu](https://docs.docker.com/desktop/windows/wsl/)
@@ -112,4 +111,4 @@ Extensions > UnitTestBot. After UTBot configuration, you can select your source
 toolbar on the left. Then, you can generate tests with the use of Command Palette. Press **F1** and type in "UTBot": You
 will see tests generation options.
 
-## [Troubleshooting](https://github.com/UnitTestBot/UTBotCpp/wiki/troubleshooting)
+## [Troubleshooting](https://github.com/UnitTestBot/UTBotCpp/wiki/troubleshooting)

README.md

@@ -16,7 +16,7 @@ More info on [wiki](https://github.com/UnitTestBot/UTBotCpp/wiki)
 
 ## How to install and use UTBot
 
-For now, you can only use UTBot under Ubuntu 18.04 and above.
+For now, you can only use UTBot under Ubuntu 20.04 and above.
 Navigate to the [**Releases**](https://github.com/UnitTestBot/UTBotCpp/releases) GitHub page and download last version of UTBot.
 
 UTBot is distrbuted as an archive that contains:

docker/Dockerfile_base

@@ -22,7 +22,7 @@ RUN apt update && DEBIAN_FRONTEND=noninteractive apt -y --no-install-recommends
 RUN echo "check_certificate = off" > /etc/wgetrc
 
 # We use C++ 17 for UnitTestBot, it is available in gcc-9; default gcc for ubuntu:18.04 is gcc-7
-RUN add-apt-repository ppa:ubuntu-toolchain-r/test
+RUN if [[ "$OPERATING_SYSTEM_TAG" = "18.04" ]] ; then RUN add-apt-repository ppa:ubuntu-toolchain-r/test ; fi
 RUN apt update && apt install -y --no-install-recommends gcc-9 g++-9 gcc-multilib g++-multilib gcc-9-multilib g++-9-multilib
 
 # Set gcc-9 as default gcc version
@@ -172,11 +172,9 @@ RUN if [[ "$OPERATING_SYSTEM_TAG" = "18.04" ]] ; then apt update && apt install
 RUN apt update && apt install -y --no-install-recommends nodejs npm openssh-server net-tools gdb vim-nox rsync
 RUN pip3 install git+https://chromium.googlesource.com/external/gyp
 
-# Update node and npm since defaults on ubuntu:18.04 have reached end of life
-RUN npm config set strict-ssl false
+# Update node and npm
 RUN npm cache clean -f
 RUN sudo -E npm install -g n
-RUN echo insecure > ~/.curlrc
 RUN sudo -E n 16
 RUN sudo -E apt remove -y --purge nodejs npm
 

docker/Dockerfile_image_without_utbot

@@ -1,4 +1,4 @@
-FROM ubuntu:18.04
+FROM ubuntu:20.04
 # use bash as default shell
 
 SHELL ["/bin/bash", "--login", "-c"]

docker/building_dependencies/runtime_env.sh

@@ -32,7 +32,7 @@ export CC=$UTBOT_INSTALL_DIR/bin/clang
 export CXX=$UTBOT_INSTALL_DIR/bin/clang++
 export CPATH=$CPATH:$UTBOT_ALL/klee/include
 export LD_LIBRARY_PATH=$UTBOT_INSTALL_DIR/lib
-export LDFLAGS='-fuse-ld=gold'
+export LDFLAGS='-fuse-ld=gold -L/usr/lib/x86_64-linux-gnu'
 export CFLAGS="$CFLAGS -gdwarf-4"
 export CXXFLAGS="$CXXFLAGS -gdwarf-4"
 export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$UTBOT_ALL/bear/bin:$UTBOT_ALL/klee/bin:$UTBOT_INSTALL_DIR/bin:$PATH

docker/create_env.sh

@@ -0,0 +1,213 @@
+#!/bin/bash
+
+# install ssh
+apt update && apt install openssh-server sudo -y
+
+useradd -m utbot
+usermod -aG sudo utbot
+passwd utbot
+
+service ssh start
+
+export OPERATING_SYSTEM_TAG=$(eval lsb_release -sr)
+export LLVM_VERSION_MAJOR=14
+
+export UTBOT_ALL=/utbot_distr
+export UTBOT_INSTALL_DIR=$UTBOT_ALL/install
+export UTBOT_CMAKE_BINARY=$UTBOT_INSTALL_DIR/bin/cmake
+
+su - utbot
+chsh -s /bin/bash
+
+#RUN echo "Set disable_coredump false" >> /etc/sudo.conf
+
+sudo apt update
+sudo apt install vim
+sudo apt install sudo file python3-dateutil wget fakeroot libssl-dev build-essential software-properties-common libtcmalloc-minimal4
+
+#echo "check_certificate = off" > /etc/wgetrc
+
+if [[ "$OPERATING_SYSTEM_TAG" = "18.04" ]]; then
+  add-apt-repository ppa:ubuntu-toolchain-r/test
+fi
+
+sudo apt install gcc-9 g++-9 gcc-multilib g++-multilib gcc-9-multilib g++-9-multilib
+sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 100
+sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-9 100
+sudo update-alternatives --install /usr/bin/gcov gcov /usr/bin/gcov-9 100
+
+sudo apt install git libcurl4-openssl-dev
+
+sudo apt install ninja-build python3-setuptools curl libcap-dev libncurses5-dev unzip libtcmalloc-minimal4 libgoogle-perftools-dev libsqlite3-dev doxygen python3-pip autoconf libtool
+
+#sudo apt install libxshmfence1 libglu1 libgconf-2-4 libatk1.0-0 libatk-bridge2.0-0 libgdk-pixbuf2.0-0 libgtk-3-0 libgbm-dev libnss3-dev libxss-dev libasound2 xvfb
+
+sudo mkdir $UTBOT_ALL
+sudo chmod 777 $UTBOT_ALL -R
+sudo chown utbot $UTBOT_ALL -R
+cd $UTBOT_ALL
+
+#Install cmake
+wget https://github.com/Kitware/CMake/releases/download/v3.17.2/cmake-3.17.2.tar.gz -O /tmp/cmake_src.tar.gz
+tar xfz /tmp/cmake_src.tar.gz -C $UTBOT_ALL
+rm /tmp/cmake_src.tar.gz
+cd $UTBOT_ALL/cmake-3.17.2
+./bootstrap --prefix=$UTBOT_INSTALL_DIR --parallel=$(nproc)
+make -j$(nproc)
+make install
+cd $UTBOT_ALL
+rm -rf $UTBOT_ALL/cmake-3.17.2
+
+sudo pip3 install tabulate==0.8.7 \
+  typing==3.7.4.3 \
+  lit==17.0.6 \
+  wllvm==1.3.1
+
+mkdir $UTBOT_ALL/llvm_gold_plugin
+wget -P $UTBOT_ALL/llvm_gold_plugin https://raw.githubusercontent.com/bminor/binutils-gdb/fd67aa1129fd006ad49ed5ecb2b063705211553a/include/plugin-api.h
+git clone --single-branch --branch "release/${LLVM_VERSION_MAJOR}.x" --depth 1 "https://github.com/llvm/llvm-project.git" $UTBOT_ALL/llvm-project
+mkdir -p $UTBOT_ALL/llvm-project/build
+cd $UTBOT_ALL/llvm-project/build
+$UTBOT_CMAKE_BINARY \
+  -DCMAKE_BUILD_TYPE=Release \
+  -DCMAKE_INSTALL_PREFIX=$UTBOT_INSTALL_DIR \
+  -DLLVM_INCLUDE_TESTS=OFF \
+  -DLLVM_BINUTILS_INCDIR=$UTBOT_ALL/llvm_gold_plugin \
+  -DLLVM_ENABLE_RTTI=ON \
+  -DLLVM_ENABLE_EH=ON \
+  -DLLVM_TARGETS_TO_BUILD="host" \
+  -DLLVM_INSTALL_UTILS=ON \
+  -DLLVM_ENABLE_PROJECTS="clang;libclc;lld;lldb;clang-tools-extra" \
+  -DLLVM_ENABLE_RUNTIMES="compiler-rt;libc;libcxx;libcxxabi" \
+  -G "Ninja" ../llvm
+$UTBOT_CMAKE_BINARY --build . --target install
+
+#export CFLAGS="-gdwarf-4"
+#export CXXFLAGS="-gdwarf-4"
+
+export LLVM_COMPILER_PATH=$UTBOT_INSTALL_DIR/bin
+export LLVM_COMPILER=clang
+
+mkdir -p $UTBOT_ALL/llvm-project/libcxx_build
+cd $UTBOT_ALL/llvm-project/libcxx_build
+CC=wllvm CXX=wllvm++ $UTBOT_CMAKE_BINARY \
+  -DLLVM_ENABLE_PROJECTS="libcxx;libcxxabi" \
+  -DLLVM_ENABLE_THREADS:BOOL=OFF \
+  -DLIBCXX_ENABLE_THREADS:BOOL=OFF \
+  -DLIBCXX_ENABLE_SHARED:BOOL=ON \
+  -DLIBCXXABI_ENABLE_THREADS:BOOL=OFF \
+  -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo \
+  -DLLVM_TARGETS_TO_BUILD=host \
+  -DCMAKE_INSTALL_PREFIX=$UTBOT_ALL/libcxx/install \
+  -DLIBCXX_ENABLE_STATIC_ABI_LIBRARY:BOOL=ON ../llvm
+
+CC=wllvm CXX=wllvm++ make cxx -j$(nproc)
+cd projects
+CC=wllvm CXX=wllvm++ make install
+find $UTBOT_ALL/libcxx/install/lib/lib*.so -print0 | xargs -0 --max-args=1 extract-bc
+find $UTBOT_ALL/libcxx/install/lib/lib*.so -print0 | xargs -0 --max-args=1 rm
+find $UTBOT_ALL/libcxx/install/lib/lib*.a -print0 | xargs -0 --max-args=1 extract-bc
+find $UTBOT_ALL/libcxx/install/lib/lib*.a -print0 | xargs -0 --max-args=1 rm
+cp -R $UTBOT_ALL/llvm-project/libcxxabi $UTBOT_ALL/libcxx
+
+cd $UTBOT_ALL
+rm -rf $UTBOT_ALL/llvm-project
+rm -rf $UTBOT_ALL/llvm_gold_plugin
+
+#export CC=$UTBOT_INSTALL_DIR/bin/clang
+#export CXX=$UTBOT_INSTALL_DIR/bin/clang++
+
+#Install GRPC
+git clone --single-branch -b v1.49.0 --depth=1 https://github.com/grpc/grpc --recurse-submodules -j4 $UTBOT_ALL/grpc
+mkdir -p $UTBOT_ALL/grpc/cmake_build
+cd $UTBOT_ALL/grpc/cmake_build
+$UTBOT_CMAKE_BINARY \
+  -DgRPC_INSTALL=ON \
+  -DgRPC_BUILD_TESTS=OFF \
+  -DCMAKE_BUILD_TYPE=Release \
+  -DCMAKE_INSTALL_PREFIX=$UTBOT_INSTALL_DIR \
+  -G "Ninja" ..
+#make -j$(nproc)
+#make install
+$UTBOT_CMAKE_BINARY --build . --target install
+cd $UTBOT_ALL
+rm -rf $UTBOT_ALL/grpc
+
+#Instal gtest
+git clone --single-branch -b release-1.10.0 https://github.com/google/googletest.git $UTBOT_ALL/gtest
+mkdir -p $UTBOT_ALL/gtest/build
+cd $UTBOT_ALL/gtest/build
+$UTBOT_CMAKE_BINARY -G "Ninja" -DCMAKE_INSTALL_PREFIX=$UTBOT_INSTALL_DIR ..
+$UTBOT_CMAKE_BINARY --build . --target install
+cd $UTBOT_ALL
+
+#Install z3
+git clone --single-branch -b z3-4.8.17 --depth=1 https://github.com/Z3Prover/z3.git $UTBOT_ALL/z3-src
+mkdir -p $UTBOT_ALL/z3-src/build
+cd $UTBOT_ALL/z3-src/build
+$UTBOT_CMAKE_BINARY -G "Ninja" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$UTBOT_INSTALL_DIR ..
+$UTBOT_CMAKE_BINARY --build . --target install
+cd $UTBOT_ALL
+rm -rf $UTBOT_ALL/z3-src
+
+#install bitwuzla
+#export PATH=$PATH:/utbot_distr/install/bin
+#sudo pip3 install meson
+#sudo apt install pkg-config libgmp-dev
+#git clone --single-branch -b 0.3.1 --depth=1 https://github.com/bitwuzla/bitwuzla.git $UTBOT_ALL/bitwuzla
+#cd $UTBOT_ALL/bitwuzla
+#./configure.py --build-dir $UTBOT_ALL/bitwuzla/build --prefix $UTBOT_INSTALL_DIR --static --no-unit-testing
+#cd $UTBOT_ALL/bitwuzla/build
+#ninja .
+#ninja install
+#cd $UTBOT_ALL
+#rm -rf $UTBOT_ALL/bitwuzla
+
+if [[ "$OPERATING_SYSTEM_TAG" = "18.04" ]]; then
+  sudo apt install nodejs-dev node-gyp libssl1.0-dev
+fi
+sudo apt install nodejs npm openssh-server net-tools gdb vim-nox rsync
+sudo pip3 install git+https://chromium.googlesource.com/external/gyp
+
+# Update node and npm
+npm cache clean -f
+sudo -E npm install -g n
+sudo -E n 16
+sudo -E apt remove -y --purge nodejs npm
+
+# Install cmake which can generate link_commands.json. Installing cmake the second time in order to build base image faster since this cmake may be changed frequently.
+git clone --single-branch -b utbot-0.1.2b --depth=1 https://github.com/Software-Analysis-Team/CMake.git $UTBOT_ALL/cmake
+cd $UTBOT_ALL/cmake
+$UTBOT_CMAKE_BINARY -G "Ninja" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$UTBOT_INSTALL_DIR .
+$UTBOT_CMAKE_BINARY --build . --target install
+cd $UTBOT_ALL
+rm -rf $UTBOT_ALL/cmake
+
+mkdir $UTBOT_ALL/klee
+mkdir $UTBOT_ALL/server-install
+
+sudo pip3 install lit
+
+wget https://github.com/CLIUtils/CLI11/releases/download/v1.9.1/CLI11.hpp -P $UTBOT_ALL/cli
+wget https://github.com/agauniyal/rang/releases/download/v3.1.0/rang.hpp -P $UTBOT_ALL/cli
+
+#COPY building_dependencies/runtime_env.sh /home/utbot/.bashrc
+#COPY building_dependencies/runtime_env.sh /root/.bashrc
+
+sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 100
+git clone -b klee_uclibc_v1.2 https://github.com/klee/klee-uclibc.git $UTBOT_ALL/klee-uclibc-32
+cp $UTBOT_ALL/klee-uclibc-32 $UTBOT_ALL/klee-uclibc-64 -R
+cd $UTBOT_ALL/klee-uclibc-32
+./configure --make-llvm-lib
+make KLEE_CFLAGS="-m32" -j$(nproc)
+
+cd $UTBOT_ALL/klee-uclibc-64
+./configure --make-llvm-lib
+make -j$(nproc)
+
+# Download library for access private members
+git clone https://github.com/martong/access_private.git $UTBOT_ALL/access_private
+
+
+sudo update-alternatives --install /usr/bin/cc cc $UTBOT_INSTALL_DIR/bin/clang 100
+sudo update-alternatives --install /usr/bin/c++ c++ $UTBOT_INSTALL_DIR/bin/clang++ 100

docker/release_distribution_scripts/utbot_run_system.sh

@@ -8,10 +8,10 @@ os_tag=$(lsb_release -i | cut -f2)
 os_version=$(lsb_release -r | cut -f2)
 major_os_version=${os_version%%.*}
 
-if [ "$os_tag" != "Ubuntu" ] || [ "$major_os_version" -lt "18" ]
+if [ "$os_tag" != "Ubuntu" ] || [ "$major_os_version" -lt "20" ]
 then
     echo "warning: $(lsb_release -d | cut -f2) unsupported system"
-    echo "UTBotCPP support ubuntu 18.04 and above"
+    echo "UTBotCPP support ubuntu 20.04 and above"
     read -r -p "Are want to continue? [y/N] " response
     if ! [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]
     then
@@ -52,9 +52,7 @@ export LD_LIBRARY_PATH=$UTBOT_ALL/install/lib:$UTBOT_ALL/debs-install/lib/x86_64
 export GENERAL_INCLUDES=$UTBOT_ALL/debs-install/usr/lib/gcc/x86_64-linux-gnu/9/include:$UTBOT_ALL/debs-install/usr/local/include:$UTBOT_ALL/debs-install/usr/include/x86_64-linux-gnu:$UTBOT_ALL/debs-install/usr/include
 export C_INCLUDE_PATH=GENERAL_INCLUDES${C_INCLUDE_PATH:+:${C_INCLUDE_PATH}}
 export CPLUS_INCLUDE_PATH=$UTBOT_ALL/debs-install/usr/include/c++/9:$UTBOT_ALL/debs-install/usr/include/x86_64-linux-gnu/c++/9:$UTBOT_ALL/debs-install/usr/include/c++/9/backward:GENERAL_INCLUDES${CPLUS_INCLUDE_PATH:+:${CPLUS_INCLUDE_PATH}}
-export CFLAGS="$CFLAGS -gdwarf-4"
-export CXXFLAGS="$CXXFLAGS -gdwarf-4"
-export LDFLAGS="-fuse-ld=gold $LDFLAGS"
+export LDFLAGS="-fuse-ld=gold -L$UTBOT_ALL/debs-install/lib/x86_64-linux-gnu"
 
 # This function moves dev version of libc into $UTBOT_ALL/debs-install directory
 # Prerequisites: path/to/directory should exist

docker/utbot_docker_dev.sh

@@ -4,7 +4,6 @@
 
 CONTAINER_NAME=$USER-utbot-dev
 MOUNT_NAME=$USER-utbot
-MOUNT_LOCAL_NAME=$MOUNT_NAME-local-mnt
 
 read -e -p "Enter base image tag: " IMAGE_TAG
 IMAGE="ghcr.io/unittestbot/utbotcpp/base_env:$IMAGE_TAG"
@@ -31,21 +30,12 @@ else
 fi
 set -e
 
-# Define local mount folder
-read -e -p "Enter local folder to mount in UTBot: " -i "$PWD" PROJECT_SRC
-if docker volume inspect $MOUNT_LOCAL_NAME > /dev/null 2>&1 ; then
-  docker volume rm $MOUNT_LOCAL_NAME > /dev/null
-fi
-docker volume create --driver lebokus/bindfs:latest -o sourcePath=$PROJECT_SRC -o map=$UID/1000:@$UID/@1000 $MOUNT_LOCAL_NAME > /dev/null
-echo "'$MOUNT_LOCAL_NAME' docker volume created."
-
 echo "Recreating docker container..."
 docker run -d --restart=unless-stopped \
  --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --security-opt apparmor=unconfined \
  --name=$CONTAINER_NAME \
  -p $UTBOT_SSH_PORT:2020 \
  -p $UTBOT_SERVER_PORT:2121 \
- -v $MOUNT_LOCAL_NAME:/home/utbot/mnt \
  -v /etc/localtime:/etc/localtime:ro \
  $IMAGE > /dev/null
 echo "Container '$CONTAINER_NAME' is up and running."

docs/install-docker-windows.md

@@ -2,7 +2,7 @@
 
 Get ready for creating a Docker container:
 
-1. Make sure you have created a WSL container with Ubuntu 18.04:
+1. Make sure you have created a WSL container with Ubuntu 20.04:
 ```bash
 wsl --list --verbose
 ```

docs/install-server-on-ubuntu.md

@@ -1,6 +1,6 @@
-# Install UnitTestBot C/C++ server on Ubuntu 18.04–20.04
+# Install UnitTestBot C/C++ server on Ubuntu 20.04
 
-Whether you have Ubuntu 18.04–20.04 on your computer, Ubuntu 18.04 in WSL or a Docker container, follow the 
+Whether you have Ubuntu 20.04 on your computer, Ubuntu 20.04 in WSL or a Docker container, follow the 
 instructions to install the UnitTestBot C/C++ server.
 
 1. Download a ZIP archive with the release artifact:

docs/install-server.md

@@ -14,10 +14,10 @@ choose one of the UnitTestBot C/C++ installation options and prepare the corresp
 **Local or remote?**
 
 The easiest way to use UnitTestBot C/C++ is to install it _locally_ — this option is available only if you have 
-Windows or Ubuntu 18.04–20.04 on your computer.
+Windows or Ubuntu 20.04 on your computer.
 
 The flip side of local installation is the risk of having dependency conflicts. UnitTestBot C/C++ requires using a specific toolchain: exact LLVM and GCC versions, and a particular solver. The other version of the necessary utility may be installed on a user
 machine — it may cause a version conflict.
 
 _Remote_ installation helps to avoid the conflicts and deploy the necessary toolchain safely. It is also the 
-only available option for macOS or any Linux distribution except Ubuntu 18.04–20.04.
+only available option for macOS or any Linux distribution except Ubuntu 20.04.

docs/install-wsl.md

@@ -9,9 +9,9 @@ Before installing WSL1 or WSL2 on you Windows 10 or Windows 11 machine, make sur
 
 To install WSL2 on you Windows 10 or Windows 11 machine:
 
-1. Create an empty WSL2 container with Ubuntu 18.04:
+1. Create an empty WSL2 container with Ubuntu 20.04:
 ```PowerShell
-wsl --install -d Ubuntu-18.04
+wsl --install -d Ubuntu-20.04
 ```
 2. Perform the initial setup for WSL2 user (in this example, we use "utbot"):
 
@@ -27,9 +27,9 @@ wsl --install -d Ubuntu-18.04
 > To run a command as administrator (user "root"), use "sudo <command>".
 > See "man sudo_root" for details.
 > ```
-3. Check if installation is successful: the new WSL2 container is called _Ubuntu-18.04_ by default:
+3. Check if installation is successful: the new WSL2 container is called _Ubuntu-20.04_ by default:
 ```PowerShell
   C:\Users\user> wsl --list --verbose
   NAME                   STATE           VERSION
-  Ubuntu-18.04           Running         2      <---------- Newly installed WSL2 container   
+  Ubuntu-20.04           Running         2      <---------- Newly installed WSL2 container   
 ```