[question] CMakeDeps and inter-package dependencies when using shared libraries #16990

bukulin · 2024-09-13T11:54:26Z

What is your question?

Hi!

I ran into a problem with CMakeDeps generator when shared libraries are used and the consuming project has a unit-test-like application. Trying to describe the situation as briefly as I could, but I have to dive deep. This might be due to my misunderstanding of the advanced dependency handling, but I try to do my best.

So let's consider the following things:

conan/2 is used: 2.7
every library is used as shared (*:shared=True)
the dependency graph is the following:
```
graph LR;
rest{{deps. of log4cxx}};
game-->engine;
engine-->log4cxx;
log4cxx-->rest;
```
Loading
Names borrowed from here: https://www.youtube.com/watch?v=kKGglzm5ous . math substituted with log4cxx to have some more dependencies.

Everything works correctly as described in the aforementioned video until I add a unit test to the engine package, just like that:

engine/CMakeLists.txt:

cmake_minimum_required(VERSION 3.15)
project(engine CXX)

include(CTest)

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
set(CMAKE_VERBOSE_MAKEFILE ON)

find_package(log4cxx REQUIRED CONFIG)

add_library(engine src/engine.cpp)
target_include_directories(engine PUBLIC include)

target_link_libraries(engine PRIVATE log4cxx)

set_target_properties(engine PROPERTIES PUBLIC_HEADER "include/engine.h")
install(TARGETS engine)

if (BUILD_TESTING)
  add_subdirectory(test)
endif()

engine/test/CMakeLists.txt:

add_executable(engine_test
  test_main.cpp)

target_link_libraries(engine_test
  PRIVATE engine )

add_test(NAME engine_test
  COMMAND engine_test )

So, engine depends privately on log4cxx and engine_test depends on engine, in turn.
At this point when I try to compile the whole engine package, the link step of engine_test fails:

/usr/bin/c++ -m64 -O3 -DNDEBUG -m64 CMakeFiles/engine_test.dir/test_main.cpp.o -o engine_test  -Wl,-rpath,/home/user/some/path/engine/build/Release ../libengine.so 
/usr/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/bin/ld: warning: libaprutil-1.so.0, needed by /home/user/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so.15, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/bin/ld: warning: libapr-1.so.0, needed by /home/user/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so.15, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/bin/ld: warning: libiconv.so.2, needed by /home/user/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so.15, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/bin/ld: warning: libcharset.so.1, needed by /home/user/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so.15, not found (try using -rpath or -rpath-link)
/usr/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../x86_64-pc-linux-gnu/bin/ld: /home/user/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so.15: undefined reference to `apr_socket_connect'
...

Interestingly liblog4cxx.so.15 is found but its dependencies not. The linker command line lacks here the library paths (-L) (and RPATH definitions (-Wl,--rpath)) of the transitive dependencies of engine, however they were present when engine library was built:

/usr/bin/c++ -fPIC -m64 -O3 -DNDEBUG -m64 -shared -Wl,-soname,libengine.so -o libengine.so CMakeFiles/engine.dir/src/engine.cpp.o   -L/home/nbukuli/.conan2/p/b/log4c3762b28bed3ae/p/lib  -L/home/nbukuli/.conan2/p/apr-u46962cbae52f2/p/lib  -L/home/nbukuli/.conan2/p/apr-u46962cbae52f2/
p/lib/apr-util-1  -L/home/nbukuli/.conan2/p/apr544b7ac059c68/p/lib  -L/home/nbukuli/.conan2/p/libic63a94c7c34484/p/lib  -L/home/nbukuli/.conan2/p/b/expat439bcb6038711/p/lib  -Wl,-rpath,/home/nbukuli/.conan2/p/b/log4c3762b28bed3ae/p/lib:/home/nbukuli/.conan2/p/apr-u46962cbae52f2/p/li
b:/home/nbukuli/.conan2/p/apr-u46962cbae52f2/p/lib/apr-util-1:/home/nbukuli/.conan2/p/apr544b7ac059c68/p/lib:/home/nbukuli/.conan2/p/libic63a94c7c34484/p/lib:/home/nbukuli/.conan2/p/b/expat439bcb6038711/p/lib: /home/nbukuli/.conan2/p/b/log4c3762b28bed3ae/p/lib/liblog4cxx.so -lm

When engine is built its dependencies are all come from the config files generated by CMakeDeps however dependencies of engine_test handled internally by CMake.

Everything works well, if *:shared=False, because in that case every static library is listed in the linker commandline when linking engine_test.

This issue might be related to this one: #16911

What do I wrong? If it is possible, I try to use modern CMake and rely on target_link_libraries with local targets in this case. I suppose a workaround could be written, where I query all the link-time dependencies of engine and apply them to engine_test but I try to avoid that.

Any guidance will be highly appreciated. Unfortunately I cannot find corresponding points in the documentation.
Thank you in advance!

Have you read the CONTRIBUTING guide?

I've read the CONTRIBUTING guide

The text was updated successfully, but these errors were encountered:

bukulin · 2024-09-13T12:17:08Z

As a possible workaround log4cxx could be linked with LINK_ONLY to engine_test:

target_link_libraries(engine_test
  PRIVATE
  engine
  $<LINK_ONLY:log4cxx>
)

That solves the linking problems, but something weird happens at the same time. Despite the LINK_ONLY, include paths of log4cxx appear in the compiler commandline with -isystem:

/usr/bin/c++  -I/home/user/some/path/engine/include -isystem /home/user/.conan2/p/b/log4c3762b28bed3ae/p/include -m64 -O3 -DNDEBUG -std=gnu++14 -MD -MT test/CMakeFiles/engine_test.dir/test_main.cpp.o -MF CMakeFiles/engine_test.dir/test_main.cpp.o.d -o CMakeFiles/engine_test.dir/test_main.cpp.o -c /home/user/some/path/engine/test/test_main.cpp

If I link apr::apr or apr-util::apr-util or even expat::expat with LINK_ONLY the compiler commandline remains intact.

Recipe of log4cxx is a bit suspicious now.

memsharded · 2024-09-13T14:11:07Z

Hi @bukulin

Thanks for your question and thanks specially for your detailed report.

Can you please do a quick check? It would be in your test CMakeLists.txt:

target_link_libraries(engine_test
  PUBLIC engine )

This shouldn't have a bad impact, being the engine_test and executable that is later no further reused by anyone, the way it links engine shouldn't affect.

If this work, this is kind of a known issue, that happens in Linux like shared libraries (it doesn't happen in Windows shared libraries). It is kind of challenging to fix it in CMakeDeps at the moment with the current information we have, but we are working to improve and expand that information and planning an improved CMakeDeps generator (it will take some time, there are still other higher priorities)

#16911

I think this was a very specific issue of the grpc recipe, and the way it works with regards to generated code having direct dependencies to other transitive dependencies, so it doesn't seem related.

bukulin · 2024-09-13T17:58:05Z

Hi @memsharded,

Thank you for the quick reply!

target_link_libraries(engine_test
  PUBLIC engine )

Unfortunately the public dependency did not help here, I have got the same linker error and the same linker commandline.

However, if engine depends publicly on log4cxx:

target_link_libraries(engine PUBLIC log4cxx)

, then the link step of engine_test works, but that is unfortunately not acceptable. Private dependency required there.

it will take some time, there are still other higher priorities

Thank you for the effort. We can live with a workaround that does not harm the required dependency chain.

#16911
Understood, thank you.

memsharded · 2024-09-13T18:29:02Z

Maybe this is related to the package_type of the different packages involved.
Could you please share your engine conanfile.py? Does it declare package_type or shared option?

Is the log4cxx package the one from ConanCenter as-is? Or is it your own custom package?

bukulin · 2024-09-16T05:04:50Z

engine is created with the following commands:
conan new -d name=engine -d version=1.0.0 cmake_lib
thus, the package_type is library and a shared option is defined.

The conanfile.py of engine:

from conan import ConanFile
from conan.tools.cmake import CMakeToolchain, CMake, cmake_layout, CMakeDeps


class engineRecipe(ConanFile):
    name = "engine"
    version = "1.0.0"
    package_type = "library"

    # Optional metadata
    license = "MIT"
    author = "Norbert Bukuli"
    url = "-"
    description = "middle engine library"

    # Binary configuration
    settings = "os", "compiler", "build_type", "arch"
    options = {
        "shared": [True, False],
        "fPIC": [True, False]
    }
    default_options = {
        "shared": False,
        "fPIC": True
    }

    # Sources are located in the same place as this recipe, copy them to the recipe
    exports_sources = "CMakeLists.txt", "src/*", "include/*"

    def requirements(self):
        self.requires("log4cxx/1.2.0")

    def config_options(self):
        if self.settings.os == "Windows":
            self.options.rm_safe("fPIC")

    def configure(self):
        if self.options.shared:
            self.options.rm_safe("fPIC")

    def layout(self):
        cmake_layout(self)

    def generate(self):
        deps = CMakeDeps(self)
        deps.generate()
        tc = CMakeToolchain(self)
        tc.generate()

    def build(self):
        cmake = CMake(self)
        cmake.configure(cli_args=["--graphviz", "deps.dot"])
        cmake.build()
        cmake.ctest(cli_args=["--output-on-failure"])

    def package(self):
        cmake = CMake(self)
        cmake.install()

    def package_info(self):
        self.cpp_info.libs = ["engine"]

log4cxx is used from ConanCenter:
log4cxx/1.2.0#95be70abf50d3c18e43b56697a2acdd5:60d416a34bb3a6d305ac643316beb9dea8dcd7be#ab45281aded77ea3bbe27b7af61f0f3b

memsharded · 2024-09-24T13:51:43Z

Hi @bukulin

I am trying to fully reproduce your case, this is what I have done: engine.zip

I am building it with either:

cd engine
conan build . -o "*:shared=True"
# or
conan create . -o "*:shared=True"

In both cases it works fine, because my actual test is empty.
Can you please check this and provide a fully reproducible example with your real test_main.cpp?
Actually putting it in a github repo is usually even better, as it is easier to collaborate. Many thanks!

bukulin · 2024-09-25T05:19:52Z

Hi @memsharded

Thank you for spending the time with this.

Here is the repository: https://github.com/bukulin/conan_cmakedeps
I have also invited you.

In the meantime I also investigate your implementation. For the first try it also failed to link the test executable.

bukulin · 2024-09-25T05:53:59Z

The problem might be with the environment. On a more recently updated machine everything works well. The production machine and the new machine differs in many ways. The compiler, linker, cmake, make, everything is newer, only conan is the same: 2.7.1.

As a quick check I've built engine on the new machine with oldish cmake-3.24.3, but that also works.

So the differences are:

|               | oldest |    old |      new | notes                           |
|---------------+--------+--------+----------+---------------------------------|
| linux-headers |   5.15 |   5.15 |      6.6 |                                 |
| binutils      | 2.33.1 |   2.38 |     2.42 |                                 |
| gcc           |  7.5.0 |   11.3 |   13.3.1 |                                 |
| cmake         | 3.22.4 | 3.24.3 |   3.30.2 | works on 'new' even with 3.24.3 |
| make          |    4.3 |    4.3 |    4.4.1 |                                 |
|---------------+--------+--------+----------+---------------------------------|
| result        |  error |  error | compiles |                                 |

If there are other suspicious system packages please let me know.

Comparing the build directories from the afformentioned old and the new environment dependencies of libengine.so differ, however notable differences cannot be found in the corresponding link.txts or Makefiles. Please skip the resolved paths, ldd is called from an unprepared shell environment, conanrun.sh is not sourced.

bukulin@bukulin /tmp $ ldd build-old/Release/libengine.so 
        linux-vdso.so.1 (0x00007ffc90924000)
        liblog4cxx.so.15 => not found
        libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/13/libstdc++.so.6 (0x00007f3654800000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f3654b33000)
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/13/libgcc_s.so.1 (0x00007f3654b0e000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f3654621000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3654c32000)
bukulin@bukulin /tmp $ ldd build-new/Release/libengine.so 
        linux-vdso.so.1 (0x00007ffc6fffe000)
        liblog4cxx.so.15 => /home/bukulin/.conan2/p/b/log4c8090bfdb2d469/p/lib/liblog4cxx.so.15 (0x00007fe00c800000)
        libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/13/libstdc++.so.6 (0x00007fe00c400000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fe00c71f000)
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/13/libgcc_s.so.1 (0x00007fe00cb01000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fe00c221000)
        libaprutil-1.so.0 => /usr/lib64/libaprutil-1.so.0 (0x00007fe00cad1000)
        libapr-1.so.0 => /usr/lib64/libapr-1.so.0 (0x00007fe00ca90000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fe00cb44000)
        libiconv.so.2 => not found
        libcharset.so.1 => not found
        libexpat.so.1 => /usr/lib64/libexpat.so.1 (0x00007fe00ca65000)
        libcrypt.so.2 => /usr/lib64/libcrypt.so.2 (0x00007fe00c6e4000)
        libuuid.so.1 => /usr/lib64/libuuid.so.1 (0x00007fe00ca5b000)

memsharded · 2024-09-25T08:20:24Z

My environment:

make 4.3
cmake 3.22
gcc 11
binutils 2.38
linux-headers (5.15.146.1-microsoft-standard-WSL2 from uname -r)

This is an Ubuntu 22.04 on WSL

When trying to build your project, the first thing I get is:

/usr/bin/ld: /home/memsharded/.conan2/p/log4c067173661e23f/p/lib/liblog4cxx.so.15: undefined reference to `apr_socket_connect'

Note this is what was happening, the problem with your working scenario is that it is finding and using the apr dependency n the system, not the conan one:

libaprutil-1.so.0 => /usr/lib64/libaprutil-1.so.0 (0x00007fe00cad1000)

This can be fixed changing the CMakeLists to:

-target_link_libraries(engine PRIVATE log4cxx)
+target_link_libraries(engine PUBLIC log4cxx)

Note this shouldn't be an issue, because engine, when used by other packages like game doesn't necessarily bring log4cxx as public, as Conan is the one managing that transitivity, not CMake (game could be even using other build system)

When changing that, I am able to get:

(conan2) /mnt/c/Users/memsharded/conanws/kk/conan_cmakedeps/engine$ ldd build/Release/libengine.so
        linux-vdso.so.1 (0x00007ffddd9a0000)
        liblog4cxx.so.15 => /home/memsharded/.conan2/p/log4c067173661e23f/p/lib/liblog4cxx.so.15 (0x00007f52476c6000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f524748c000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f524746a000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5247241000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f5247911000)
        libaprutil-1.so.0 => not found
        libapr-1.so.0 => not found
        libiconv.so.2 => not found
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5247158000)

Which looks better, because the dependencies apr, apr-util, iconv are not found int he system, but provided by Conan.

memsharded · 2024-09-25T09:21:40Z

I have reviewed the CMake targets properties, but it seems correct, might be a CMake issue?

Lets call the dependencies engine -> libb -> liba for simplicity:

liba::liba target defines

set_property(TARGET liba::liba APPEND PROPERTY INTERFACE_LINK_DIRECTORIES $<$<CONFIG:Release>:${liba_LIB_DIRS_RELEASE}>)

libb::libb is depending on liba::liba with INTERFACE_LINK_LIBRARIES:

if(NOT TARGET libb_DEPS_TARGET)
  add_library(libb_DEPS_TARGET INTERFACE IMPORTED)
endif()

set_property(TARGET libb_DEPS_TARGET
           APPEND PROPERTY INTERFACE_LINK_LIBRARIES
           $<$<CONFIG:Release>:${libb_FRAMEWORKS_FOUND_RELEASE}>
           $<$<CONFIG:Release>:${libb_SYSTEM_LIBS_RELEASE}>
           $<$<CONFIG:Release>:liba::liba>)

With target_link_libraries(engine PRIVATE libb::libb) it fails, with target_link_libraries(engine PUBLIC libb::libb) it works.

I would expect that the liba INTERFACE_LINK_DIRECTORIES should be passed down to consumers of engine, because linkage is not really PRIVATE, but it seems that unless the targets are explicitly defined as SHARED and not INTERFACE, they are not propagated. I'll try to think if it is possible to improve something here, but might be difficult.

bukulin · 2024-09-25T09:31:53Z

I've checked with a clean virtual machine (lxd container), that does not have apr-utils installed in the system. The build failed there, as expected. If engine publicly depends on log4cxx, then the link step of engine_test succeeds. If apr-util is installed in the system, then the link step also succeeds.

I would expect that the liba INTERFACE_LINK_DIRECTORIES should be passed down to consumers of engine, because linkage is not really PRIVATE, but it seems that unless the targets are explicitly defined as SHARED and not INTERFACE, they are not propagated. I'll try to think if it is possible to improve something here, but might be difficult.

I experienced the same behavior. Assuming your dependency chain I did the following workaround:

target_link_libraries(engine PRIVATE libb::libb $<LINK_ONLY:liba::liba>)

But that could go crazy if libb::libb has a lot of dependencies, just like log4cxx.

Note this shouldn't be an issue, because engine, when used by other packages like game doesn't necessarily bring log4cxx as public, as Conan is the one managing that transitivity, not CMake (game could be even using other build system)

We are right now in a transition from conan/1 to conan/2. With the former one we used cmake_paths generator and relied on the CMake configuration files generated mostly by CMake itself :configure_package_config_file, find_dependencies, check_required_components, write_basic_package_version_file ans so on. Until this transition is finished the PRIVATE dependency helps us a lot.

In the long run, we would like to keep these CMake configuration files, not necessarily in the conan package but as part of the build. However this brings in some sort of duplication. All the required information must be set in our CMake system and in the conanfile.py, as well. Formerly cmake_paths was a right fit for this, but CMakeDeps went to a different direction. It must be said, that for non-CMake projects CMakeDeps is a very powerful tool.

Is there any chance to have best of both worlds? Is it feasible to have a generator, that uses the packaged CMake config files if they are present but behaves like CMakeDeps if the requirement package does not have a proper CMake config file?

memsharded · 2024-09-25T20:29:54Z

We are right now in a transition from conan/1 to conan/2. With the former one we used cmake_paths generator and relied on the CMake configuration files generated mostly by CMake itself :configure_package_config_file, find_dependencies, check_required_components, write_basic_package_version_file ans so on. Until this transition is finished the PRIVATE dependency helps us a lot.

I am not sure, are you trying to move to CMakeDeps while moving to Conan 2? Why not moving to Conan 2 while keeping your usage of in-package xxx-config.cmake files? This is still possible and documented in Conan 2: https://docs.conan.io/2/examples/tools/cmake/cmake_toolchain/use_package_config_cmake.html

Is there any chance to have best of both worlds? Is it feasible to have a generator, that uses the packaged CMake config files if they are present but behaves like CMakeDeps if the requirement package does not have a proper CMake config file?

Yes, if the package that generates the xxx-config.cmake inside the package defines the .set_property("cmake_find_mode", "none") then the consumer CMakeDeps will not create the files for it, and use the xxx-config.cmake inside the package.
Isn't this what you are looking for?

memsharded · 2024-09-25T20:31:52Z

In the long run, we would like to keep these CMake configuration files, not necessarily in the conan package but as part of the build.

In the long run, the goal would actually to use CPS information, we are working on this in the C++ tooling evolution group to try to standardize it, we already did some progress recently presented in CppCon: https://cppcon2024.sched.com/event/1gZew/common-package-specification-cps-in-practice-a-full-round-trip-implementation-in-conan-c-package-manager

bukulin · 2024-09-26T09:12:15Z

I am not sure, are you trying to move to CMakeDeps while moving to Conan 2? Why not moving to Conan 2 while keeping your usage of in-package xxx-config.cmake files? This is still possible and documented in Conan 2: https://docs.conan.io/2/examples/tools/cmake/cmake_toolchain/use_package_config_cmake.html

Yes, actually exactly that happened. Somehow I passed over this section of the documentation. Please let me point out, the documentation is invaluable.

I have tried it and mostly works. The problem is again with log4cxx as a transitive dependency. The INTERFACE_LINK_DIRECTORIES property of log4cxx is not incarnated in the linker commandline with -L. But that is the same what you have mentioned here: #16990 (comment)

Yes, if the package that generates the xxx-config.cmake inside the package defines the .set_property("cmake_find_mode", "none") then the consumer CMakeDeps will not create the files for it, and use the xxx-config.cmake inside the package.
Isn't this what you are looking for?

Exactly, thank you.

In the long run, the goal would actually to use CPS information, we are working on this in the C++ tooling evolution group to try to standardize it, we already did some progress recently presented in CppCon: https://cppcon2024.sched.com/event/1gZew/common-package-specification-cps-in-practice-a-full-round-trip-implementation-in-conan-c-package-manager

Great news, I will check it out. Thanks again.

memsharded · 2024-10-29T09:59:27Z

We are releasing in Conan 2.9 a completely new CMakeDeps generator in #16964 that has closed this ticket, with many pending features and fixes:

Allow defining cpp_info.default_components and using that information to generate CMake targets by default for packages
Allow defining cpp_info.exe to model executable imported targed generation
Creating always actual SHARED/STATIC/INTERFACE imported targets, with their IMPORTED_LOCATION, the IMPORTED_IMPLIB, the IMPORTED_CONFIGURATIONS
No artificial library or package targets
Allow using the package languages attribute or a new cpp_info.languages information to define IMPORTED_LINK_INTERFACE_LANGUAGES
Define config files for executable targets for the "build" context automatically, without any special configuration
Use the correct CONFIGURATION from the package instead of the consumer one
Generate a new conan_cmakedeps_paths.cmake file, that can be used for integrations that cannot use a CMakeToolchain, like cmake-conan
Allow using executable targets from the host context if not cross-building (and not tool-requires that would prioritize those executable targets)

Current known pending functionality (to be added soon):

Not managing Apple frameworks
Not generating find modules, only CMake Config files

The new CMakeDeps generator is intended for testing and validation only, being a transparent replacement of the old one, so it is behind a new conf. To use it, use the -c tools.cmake.cmakedeps:new=will_break_next, and that will use the new generator instead of the old one. Note the will_break_next value means exactly that, that value will change in next release to force a break, so no one can depend on this generator in production yet.

Your feedback is very important

As this is a major change, we will only remove the conf gate when we get confirmation from users that it works and solve the issues. Please try the new generator for your project, and let us know if it works. If it doesn't, please re-open this ticket and let us know what failed. Thanks very much!

bukulin · 2024-10-31T07:31:37Z

Great news! Thank you very much for all of your efforts.

I have tested the new CMakeDeps2 generator and an error emerged:

engine/1.0.0: WARN: Using the new CMakeDeps generator, behind the 'tools.cmake.cmakedeps:new' gate conf. This conf will changenext release, breaking, so use it only for testing and dev
ERROR: Traceback (most recent call last):
  File "/home/user/conan2-venv/lib/python3.9/site-packages/conan/internal/errors.py", line 35, in conanfile_exception_formatter
    yield
  File "/home/user/conan2-venv/lib/python3.9/site-packages/conan/internal/api/install/generators.py", line 126, in write_generators
    conanfile.generate()
  File "/home/user/.conan2/p/engin4f0278febf2d5/e/conanfile.py", line 46, in generate
    deps.generate()
  File "/home/user/conan2-venv/lib/python3.9/site-packages/conan/tools/cmake/cmakedeps2/cmakedeps.py", line 41, in generate
    generator_files = self._content()
  File "/home/user/conan2-venv/lib/python3.9/site-packages/conan/tools/cmake/cmakedeps2/cmakedeps.py", line 68, in _content
    ret[target_configuration.filename] = target_configuration.content()
  File "/home/user/conan2-venv/lib/python3.9/site-packages/conan/tools/cmake/cmakedeps2/target_configuration.py", line 24, in content
    return t.render(self._context)
  File "/home/user/conan2-venv/lib/python3.9/site-packages/conan/tools/cmake/cmakedeps2/target_configuration.py", line 74, in _context
    cpp_info = self._conanfile.cpp_info.deduce_full_cpp_info(self._conanfile)
  File "/home/user/conan2-venv/lib/python3.9/site-packages/conans/model/build_info.py", line 750, in deduce_full_cpp_info
    result._package.deduce_locations(pkg_type)
  File "/home/user/conan2-venv/lib/python3.9/site-packages/conans/model/build_info.py", line 549, in deduce_locations
    if self._type != pkg_type:
  File "/home/user/conan2-venv/lib/python3.9/site-packages/conans/model/pkg_type.py", line 22, in __eq__
    return super().__eq__(PackageType(other))
  File "/usr/lib/python3.9/enum.py", line 384, in __call__
    return cls.__new__(cls, value)
  File "/usr/lib/python3.9/enum.py", line 702, in __new__
    raise ve_exc
ValueError: None is not a valid PackageType

engine/1.0.0: Error in generate() method, line 46
        deps.generate()
        ValueError: None is not a valid PackageType

I suppose that the problem lies in this function: https://github.com/conan-io/conan/blob/develop2/conans/model/build_info.py#L511-L517
when we try to find shared libraries on linux here: https://github.com/conan-io/conan/blob/develop2/conans/model/build_info.py#L520

In case of *nix, in the guts of _find_matching() we will find one "real" file and two symbolic links with glob.glob(), so len(matches) == 3. In that case the _find_matching() function does not return (or returns None). Now, if we return the whole matches set, some string substition will fail later, so I suppose we should return the linkname here, i.e. the one without version numbers:

def _find_matching(patterns, dirs):
    matches = set()
    for pattern in patterns:
        for d in dirs:
            matches.update(glob.glob(f"{d}/{pattern}"))
    if len(matches) == 1:
        return next(iter(matches))
    if len(matches) > 1:
        return min(matches, key=len)

After that modification, conan successfully installed the requirements and generated everything, but CMake configuration fails with the following error message:

engine/1.0.0: RUN: cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE="generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="/home/user/.conan2/p/b/engine18a6ef5d1050/p" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" -DCMAKE_BUILD_TYPE="Release" "/home/user/.conan2/p/b/engine18a6ef5d1
050/b"
engine/1.0.0: Full command: . "/home/user/.conan2/p/b/engine18a6ef5d1050/b/build/Release/generators/conanbuild.sh" && cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE="generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="/home/user/.conan2/p/b/engine18a6ef5d1050/p" -DCMAK
E_POLICY_DEFAULT_CMP0091="NEW" -DCMAKE_BUILD_TYPE="Release" "/home/user/.conan2/p/b/engine18a6ef5d1050/b"
-- Using Conan toolchain: /home/user/.conan2/p/b/engine18a6ef5d1050/b/build/Release/generators/conan_toolchain.cmake
-- Conan toolchain: Defining architecture flag: -m64
-- Conan toolchain: Including CMakeDeps generated conan_find_paths.cmake
-- Conan toolchain: Setting BUILD_SHARED_LIBS = ON
-- The CXX compiler identification is GNU 7.5.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring Targets for log4cxx/1.2.0
-- Configuring Targets for apr-util/1.6.1
-- Configuring Targets for apr/1.7.4
-- Conan: Target declared imported SHARED library 'apr::apr'
-- Configuring Targets for libiconv/1.17
-- Conan: Target declared imported SHARED library 'libiconv::_iconv'
-- Conan: Target declared imported SHARED library 'libiconv::_charset' 
-- Conan: Target declared imported INTERFACE library 'libiconv::_common'
-- Conan: Target declared imported INTERFACE library 'Iconv::Iconv'
-- Configuring Targets for expat/2.6.3
-- Conan: Target declared imported SHARED library 'expat::expat'
-- Conan: Target declared imported SHARED library 'apr-util::apr-util' 
-- Conan: Target declared imported SHARED library 'log4cxx'
-- Configuring done
CMake Error at CMakeLists.txt:10 (add_library):
  Target "engine" links to target "libiconv::libiconv" but the target was not
  found.  Perhaps a find_package() call is missing for an IMPORTED target, or
  an ALIAS target is missing?


-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

The dependency graph is the following:

Please let me note here, that I cannot express how thankful I am for the simplified graph with the meaningful names compared to the old CMakeDeps with the auto-generated names.

I suppose that the problem here might be that libiconv package exports the following target in Iconv-Targets-Release.cmake:

#################### Iconv::Iconv ####################
if(NOT TARGET Iconv::Iconv)
    message(STATUS "Conan: Target declared imported INTERFACE library 'Iconv::Iconv'")
    add_library(Iconv::Iconv INTERFACE IMPORTED)
endif()

target_link_libraries(Iconv::Iconv INTERFACE libiconv::_iconv libiconv::_charset libiconv::_common)

but apr-util wants the lowercase iconv::inconv in apr-util-Targets-release.cmake:

target_link_libraries(apr-util::apr-util INTERFACE apr::apr libiconv::libiconv expat::expat)

I cannot figure out right now where this uppercase - lowercase problem comes from, but libiconv exports Iconv::Iconv for clients:
https://github.com/conan-io/conan-center-index/blob/master/recipes/libiconv/all/conanfile.py#L164

@memsharded what do you think, shall we reopen this ticket?

memsharded · 2024-10-31T09:49:33Z

Hi @bukulin

Thanks for the extensive feedback!

We are already aware of that error, the PR #17257 tries to fix it.
It seems it is derived mostly from the failure to find the location of a library, we knew that this function is not very robust yet, and we planned to improve it.

Please let me note here, that I cannot express how thankful I am for the simplified graph with the meaningful names compared to the old CMakeDeps with the auto-generated names.

yes, definitely, we were also really looking forward this! :) It is just a matter of available time and resources, I am very happy that we are finally moving this forward.

Let's re-open this and iterate it with that feedback, try to solve those issues for next release :)

memsharded · 2024-10-31T12:42:57Z

I have added a check for the "Iconv" failure in PR #17257, if you want to try things from that branch, otherwise I will keep you updated in this thread when it is merged

I will be working with @franramirez688 to improve also the find_location detection of libraries next week.

Thanks for your feedback, super useful, exactly the kind of feedback we need to improve the new CMakeDeps generator quickly.

bukulin · 2024-10-31T13:11:08Z

Great, thank you in advance! I'm looking forward for the new version!

Thanks for your feedback, super useful, exactly the kind of feedback we need to improve the new CMakeDeps generator quickly.
I'm happy to help.

memsharded · 2024-11-25T10:21:48Z

Hi @bukulin

Better find-locations has been implemented in #17296, for next Conan 2.10. Looking forward your feedback, please create new tickets as necessary.

memsharded self-assigned this Sep 13, 2024

memsharded mentioned this issue Oct 15, 2024

New CMakeDeps #16964

Merged

czoido closed this as completed in #16964 Oct 29, 2024

memsharded reopened this Oct 31, 2024

memsharded added this to the 2.10.0 milestone Oct 31, 2024

franramirez688 mentioned this issue Nov 12, 2024

[cpp_info][BazelDeps] Improve auto deduce locations #17296

Merged

franramirez688 self-assigned this Nov 12, 2024

czoido closed this as completed in #17296 Nov 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[question] CMakeDeps and inter-package dependencies when using shared libraries #16990

[question] CMakeDeps and inter-package dependencies when using shared libraries #16990

bukulin commented Sep 13, 2024

bukulin commented Sep 13, 2024 •

edited

Loading

memsharded commented Sep 13, 2024

bukulin commented Sep 13, 2024

memsharded commented Sep 13, 2024

bukulin commented Sep 16, 2024 •

edited

Loading

memsharded commented Sep 24, 2024

bukulin commented Sep 25, 2024

bukulin commented Sep 25, 2024 •

edited

Loading

memsharded commented Sep 25, 2024

memsharded commented Sep 25, 2024

bukulin commented Sep 25, 2024 •

edited

Loading

memsharded commented Sep 25, 2024

memsharded commented Sep 25, 2024

bukulin commented Sep 26, 2024

memsharded commented Oct 29, 2024

bukulin commented Oct 31, 2024 •

edited

Loading

memsharded commented Oct 31, 2024

memsharded commented Oct 31, 2024

bukulin commented Oct 31, 2024

memsharded commented Nov 25, 2024

[question] CMakeDeps and inter-package dependencies when using shared libraries #16990

[question] CMakeDeps and inter-package dependencies when using shared libraries #16990

Comments

bukulin commented Sep 13, 2024

What is your question?

Have you read the CONTRIBUTING guide?

bukulin commented Sep 13, 2024 • edited Loading

memsharded commented Sep 13, 2024

bukulin commented Sep 13, 2024

memsharded commented Sep 13, 2024

bukulin commented Sep 16, 2024 • edited Loading

memsharded commented Sep 24, 2024

bukulin commented Sep 25, 2024

bukulin commented Sep 25, 2024 • edited Loading

memsharded commented Sep 25, 2024

memsharded commented Sep 25, 2024

bukulin commented Sep 25, 2024 • edited Loading

memsharded commented Sep 25, 2024

memsharded commented Sep 25, 2024

bukulin commented Sep 26, 2024

memsharded commented Oct 29, 2024

Your feedback is very important

bukulin commented Oct 31, 2024 • edited Loading

memsharded commented Oct 31, 2024

memsharded commented Oct 31, 2024

bukulin commented Oct 31, 2024

memsharded commented Nov 25, 2024

bukulin commented Sep 13, 2024 •

edited

Loading

bukulin commented Sep 16, 2024 •

edited

Loading

bukulin commented Sep 25, 2024 •

edited

Loading

bukulin commented Sep 25, 2024 •

edited

Loading

bukulin commented Oct 31, 2024 •

edited

Loading