feat crypto: support wolfssl library, help wanted #523

theg4sh · 2024-04-14T14:20:32Z

closes #498

Currently, this PR won't build by the reason described below.

This PR provides minimal changes which replaces internal usage of openssl. It does not checked for other libraries used in userver. Some openssl features does not implemented by wolfssl itself. (has found some typos during investigation, see wolfSSL/wolfssl#7423).

BTW, wolfssl v5.7.0-stable requires few small patches which also included into the PR.

Finally, "short path" to support wolfssl is not available, because of the lib does not implements some of used functions. It also does not provide ENGINE_*, but have not been tried with wolfengine lib yet.
Worst thing is that wolfssl does not provides any CMS_* analogue to soft migrate onto this library, so, need some code branching using available functions in wolfssl.

PR is building with commands:

mkdir -p build_debug
cd build_debug

cmake \
  -Wdev \
  -DCMAKE_CXX_COMPILER=clang++-17 \
  -DUSERVER_FEATURE_WOLFSSL=ON \
  -DUSERVER_DOWNLOAD_PACKAGE_WOLFSSL=ON \
  -DUSERVER_FEATURE_GRPC=OFF \
  -DUSERVER_FEATURE_POSTGRESQL=OFF \
  -DUSERVER_FEATURE_MYSQL=OFF \
  -DUSERVER_FEATURE_STACKTRACE=OFF \
  -DUSERVER_FEATURE_CLICKHOUSE=OFF \
  -DUSERVER_USE_LD=lld \
  ..

Patch might be re-applied with command:
(test -d build_debug/_deps/wolfssl-src && cd build_debug/_deps/wolfssl-src && git checkout -- .)

Current errors will be attached in comments to this PR.
Help needed with re-implementation of current crypto's features using wolfssl.

In my opinion, wolfssl is not production ready yet because of a lot of issues https://github.com/wolfSSL/wolfssl/labels/bug

closes userver-framework#498 This PR provides minimal changes which replaces internal usage of openssl. It does not checked for other libraries used in userver. Some openssl features does not implemented by wolfssl itself. (has found some typos during investigation, see wolfSSL/wolfssl#7423). BTW, wolfssl v5.7.0-stable requires few small patches which also included into the PR. Finally, "short path" to support wolfssl is not available, because of the lib does not implements some of used functions. It also does not provide `ENGINE_*`, but have not been tried with wolfengine lib yet. PR is building with commands: ``` mkdir -p build_debug cd build_debug cmake \ -Wdev \ -DCMAKE_CXX_COMPILER=clang++-17 \ -DUSERVER_FEATURE_WOLFSSL=ON \ -DUSERVER_DOWNLOAD_PACKAGE_WOLFSSL=ON \ -DUSERVER_FEATURE_GRPC=OFF \ -DUSERVER_FEATURE_POSTGRESQL=OFF \ -DUSERVER_FEATURE_MYSQL=OFF \ -DUSERVER_FEATURE_STACKTRACE=OFF \ -DUSERVER_FEATURE_CLICKHOUSE=OFF \ -DUSERVER_USE_LD=lld \ .. ``` Patch might be re-applied with command: ```(test -d build_debug/_deps/wolfssl-src && cd build_debug/_deps/wolfssl-src && git checkout -- .)``` Final error will be attached in comments to this PR. Help needed.

theg4sh · 2024-04-14T14:21:52Z

Latest build log in attach.
build-failed.log

theg4sh · 2024-04-14T14:23:15Z

CMakeLists.txt

@@ -172,6 +173,13 @@ option(USERVER_FEATURE_MYSQL "Provide asynchronous driver for MariaDB/MySQL" "${

 option(USERVER_FEATURE_UBOOST_CORO "Use vendored boost context instead of a system one" ON)

+if (USERVER_FEATURE_WOLFSSL)
+  include(cmake/SetupWolfSSL.cmake)
+  add_compile_definitions("OPENSSL_EXTRA=1")


this compile definitions added directly because it is required by wolfssl headers.

segoon · 2024-05-03T08:35:52Z

CMakeLists.txt

@@ -172,6 +173,13 @@ option(USERVER_FEATURE_MYSQL "Provide asynchronous driver for MariaDB/MySQL" "${

 option(USERVER_FEATURE_UBOOST_CORO "Use vendored boost context instead of a system one" ON)

+if (USERVER_FEATURE_WOLFSSL)
+  include(cmake/SetupWolfSSL.cmake)
+  add_compile_definitions("OPENSSL_EXTRA=1")


target_compile_definitions

segoon · 2024-05-03T08:36:49Z

cmake/SetupWolfSSL.cmake

+    "CMAKE_C_FLAGS -Wall -Wextra -O2 -DOPENSSL_ALL -DOPENSSL_EXTRA"
+)
+
+#add_library(WolfSSL INTERFACE)


garbage? or conditional code?

Had experimented with, mostly a garbage.
The PR still in draft state and I'll clear this when (and if) all job will be done.

segoon · 2024-05-03T08:39:31Z

universal/CMakeLists.txt

 )

+if (USERVER_FEATURE_WOLFSSL)
+  target_link_libraries(${PROJECT_NAME}


the same code snippet is copied multiple times
it's better to declare INTERFACE target, link openssl/wolfssl targets, and define all required macros in this interface library

segoon · 2024-05-03T08:42:00Z

universal/src/crypto/public_key.cpp

@@ -81,6 +91,12 @@ int CurveStringToNid(const std::string_view& curve_str) {
  return it->second;
 }

+#ifdef USERVER_FEATURE_WOLFSSL
+#ifndef EC_KEY_set_public_key_affine_coordinates
+#warning "EC_KEY_set_public_key_affine_coordinates does not provided by wolfSSL"


is it an old wolfssl version limitation?

At the point of PR, 5.7.0 is the latest stable version.
WolfSSL does not provide function with suffix _coordinates, but EC_KEY_set_public_key
See https://github.com/wolfSSL/wolfssl/blob/master/wolfssl/openssl/ec.h

segoon · 2024-05-03T08:43:31Z

universal/src/crypto/private_key.cpp

@@ -41,6 +48,13 @@ std::optional<std::string> GetPemStringImpl(EVP_PKEY* key,

  std::string result;
  result.resize(BIO_pending(membio.get()));
+#ifdef USERVER_FEATURE_WOLFSSL
+#warning "BIO_read_ex does not supported by wolfSSL"
+  if (1 != BIO_read(membio.get(), result.data(), result.size())) {


i guess we may use BIO_read() in openssl too as we don't care about readbytes

segoon · 2024-05-03T08:45:12Z

universal/src/crypto/helpers.cpp

+#define EVP_PKEY_CTX_set_rsa_pss_saltlen(...) 0
+#endif
+#ifndef EVP_PKEY_CTX_set_rsa_mgf1_md
+#warning "EVP_PKEY_CTX_set_rsa_mgf1_md() is undefined by wolfSSL"


why is it possible?

These functions are missed in the header wolfssl/openssl/evp.h. Sadly, buy I have no idea currently how to make workaround for these functions.

segoon · 2024-05-03T08:45:42Z

universal/src/crypto/certificate.cpp

@@ -31,6 +36,13 @@ std::optional<std::string> Certificate::GetPemString() const {

  std::string result;
  result.resize(BIO_pending(membio.get()));
+#ifdef USERVER_FEATURE_WOLFSSL
+#warning "BIO_read_ex does not supported by wolfSSL"
+  if (1 != BIO_read(membio.get(), result.data(), result.size())) {


same here, we can use BIO_read() in both cases

theg4sh · 2024-04-20T17:02:58Z

.gitignore

@@ -13,7 +13,6 @@
 /cmake-build-*/
 /cmake/cmake_generated/
 /docs/
-/third_party/


this line was added eariler by mistake.

theg4sh · 2024-05-04T15:41:26Z

cmake/SetupWolfSSL.cmake

+    "CMAKE_C_FLAGS -Wall -Wextra -O2 -DOPENSSL_ALL -DOPENSSL_EXTRA"
+)
+
+#add_library(WolfSSL INTERFACE)


Had experimented with, mostly a garbage.
The PR still in draft state and I'll clear this when (and if) all job will be done.

theg4sh · 2024-05-04T15:41:41Z

universal/src/crypto/public_key.cpp

@@ -81,6 +91,12 @@ int CurveStringToNid(const std::string_view& curve_str) {
  return it->second;
 }

+#ifdef USERVER_FEATURE_WOLFSSL
+#ifndef EC_KEY_set_public_key_affine_coordinates
+#warning "EC_KEY_set_public_key_affine_coordinates does not provided by wolfSSL"


At the point of PR, 5.7.0 is the latest stable version.
WolfSSL does not provide function with suffix _coordinates, but EC_KEY_set_public_key
See https://github.com/wolfSSL/wolfssl/blob/master/wolfssl/openssl/ec.h

theg4sh · 2024-05-04T15:42:08Z

universal/src/crypto/helpers.cpp

+#define EVP_PKEY_CTX_set_rsa_pss_saltlen(...) 0
+#endif
+#ifndef EVP_PKEY_CTX_set_rsa_mgf1_md
+#warning "EVP_PKEY_CTX_set_rsa_mgf1_md() is undefined by wolfSSL"


These functions are missed in the header wolfssl/openssl/evp.h. Sadly, buy I have no idea currently how to make workaround for these functions.

theg4sh requested a review from segoon as a code owner April 14, 2024 14:20

theg4sh marked this pull request as draft April 14, 2024 14:20

theg4sh commented Apr 14, 2024

View reviewed changes

theg4sh changed the title ~~feat crypto: support wolfssl library~~ feat crypto: support wolfssl library, help wanted Apr 14, 2024

segoon reviewed May 3, 2024

View reviewed changes

using BIO_read instead of BIO_read_ex for openssl too

579bf87

theg4sh commented May 4, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat crypto: support wolfssl library, help wanted #523

feat crypto: support wolfssl library, help wanted #523

theg4sh commented Apr 14, 2024 •

edited

Loading

theg4sh commented Apr 14, 2024

theg4sh Apr 14, 2024 •

edited

Loading

segoon May 3, 2024

segoon May 3, 2024

theg4sh May 4, 2024

segoon May 3, 2024

segoon May 3, 2024

theg4sh May 4, 2024

segoon May 3, 2024

segoon May 3, 2024

theg4sh May 4, 2024

segoon May 3, 2024

theg4sh Apr 20, 2024

theg4sh May 4, 2024

theg4sh May 4, 2024

theg4sh May 4, 2024

feat crypto: support wolfssl library, help wanted #523

Are you sure you want to change the base?

feat crypto: support wolfssl library, help wanted #523

Conversation

theg4sh commented Apr 14, 2024 • edited Loading

theg4sh commented Apr 14, 2024

theg4sh Apr 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

theg4sh commented Apr 14, 2024 •

edited

Loading

theg4sh Apr 14, 2024 •

edited

Loading