support disabling sqlite3 and including sqlite3 as source code instead of pkg-config

[sambercovici]

Sqlite should be disabled via directive as its not mandatory.

Boost can be provided to the build and does not mandate pre-installing it OS wide.
Sqlite should also be optionally provided via https://www.sqlite.org/amalgamation.html to be compiled and available locally instead of enforcing installing it OS wide.

[markos]

sqlite3 will be made optional in the next release.

[sambercovici]

@markos , does it make sense to also support sqlite3 as locally compiled library similar to boost?

[markos]

I want to simplify compilation, the only moving target should be the library itself, everything else should be used from stable sources, ie packaged. Unless absolutely necessary, which currently it isn't. As a matter of fact, as a long term future goal, I would like to remove Boost dependency, but that will take a while.

[sambercovici]

Envoy does not precompiled libraries but will build ALL used libraries as part of the build process using bazel.
Clean/simple portable compilation of the library with as little as possible dependency would be greate.
Do you want to keep this open to track "sqlite3 will be made optional in the next release." or should I close it?

[markos]

Keep this open to keep track please, it will be closed when it is actually implemented in Vectorscan.

[markos]

@sambercovici could you please test PR #219? It should work with your usecase, only hsbench requires sqlite3, it doesn't break the build anymore if it's not available.

[sambercovici]

currently it fails. Using ubuntu 20.04
steps:

• sudo apt-get update -y
• sudo apt-get upgrade -y
• sudo apt-get install -y ragel build-essential g++ python3-dev autotools-dev libicu-dev libbz2-dev
• sudo snap install cmake --classic
• cmake -- version
  cmake version 3.28.1

then:

• cd
• mkdir src
• cd src
• git clone https://github.com/VectorCamp/vectorscan.git
• cd vectorscan/
• git checkout bugfix/make-sqlite-optional
• mkdir build
• cd build/
• wget -O boost_1_57_0.tar.gz https://sourceforge.net/projects/boost/files/boost/1.57.0/boost_1_57_0.tar.gz/download
• tar xf boost_1_57_0.tar.gz
• cmake .. -DBOOST_ROOT=/home/ubuntu/src/vectorscan/build/boost_1_57_0/ -DCMAKE_BUILD_TYPE=Release -DFAT_RUNTIME=off -DBUILD_EXAMPLES=off -DBUILD_SHARED_LIBS=off -DCMAKE_INSTALL_LIBDIR="lib"

failed with the following output:
-- Build date: 2024-01-14
CMake Warning (dev) at cmake/boost.cmake:17 (find_package):
Policy CMP0144 is not set: find_package uses upper-case _ROOT
variables. Run "cmake --help-policy CMP0144" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.

CMake variable BOOST_ROOT is set to:

/home/ubuntu/src/vectorscan/build/boost_1_57_0/

For compatibility, find_package is ignoring the variable, but code in a
.cmake module might still use it.
Call Stack (most recent call first):
CMakeLists.txt:27 (include)
This warning is for project developers. Use -Wno-dev to suppress it.

-- Boost version: 1.57.0
CMake Error at /snap/cmake/1361/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)

  Reason given by package: The command
    "/usr/bin/pkg-config" --version
  failed with output:



  stderr:

  result:

no such file or directory

Call Stack (most recent call first):
/snap/cmake/1361/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
/snap/cmake/1361/share/cmake-3.28/Modules/FindPkgConfig.cmake:99 (find_package_handle_standard_args)
CMakeLists.txt:30 (find_package)

-- Configuring incomplete, errors occurred!

[markos]

that is not related to the sqlite fix and has not been changed at all. If it was working before for you it should still work.
But out of curiosity, 20.04 comes with boost 1.71 why do you install a much older version instead?

[sambercovici]

boost - my just copy / paste my original writeup.
If you look at: https://github.com/envoyproxy/envoy/pull/29881/files#diff-f34aba99614f7980c96b61e5105b9b06b55f54a1dc56279c7d189f0ccb6fa2df

I also modified find_package(PkgConfig REQUIRED) to find_package(PkgConfig QUIET)

As noted, in envoy proxy - envoyproxy/envoy-build-tools#303, they DO NOT want to add pkgconfig to any of the build containers

So:
in this branch, I also modified find_package(PkgConfig REQUIRED) to find_package(PkgConfig QUIET) in CMakeLists.txt
after this, the cmake and make complete fine.

Can you add a directive to switch the detection to optionally use find_package(PkgConfig QUIET) instead of find_package(PkgConfig REQUIRED) ?

Without this, I will need to preserve some form of patching in the envoyproxy build

[markos]

ok, but in that case, sqlite detection will be skipped entirely. I don't want to add extra logic to handle sqlite detection which is only needed for a tool, so sqlite will only be checked if pkg-config is available.

[sambercovici]

awesome!