Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search RUNPATH to find library on Linux #44

Closed
wants to merge 11 commits into from
25 changes: 24 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ if(WIN32)
target_compile_definitions(${PROJECT_NAME}
PRIVATE "RCPPUTILS_BUILDING_LIBRARY")
endif()
set_target_properties(${PROJECT_NAME} PROPERTIES
BUILD_RPATH ${CMAKE_CURRENT_BINARY_DIR}
INSTALL_RPATH_USE_LINK_PATH TRUE
)
Comment on lines +37 to +40
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The overall thing that concerns me about this is this bit. As far as I know, we don't currently set RPATH/RUNPATH on any of our shared objects. In order for this to truly work in general, we would have to start setting that, correct?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for taking so long to reply. I think you are right, for this to work in general, you would need to have set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE) for all projects.

What we are currently doing in our project is using set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE) together with set(CMAKE_EXE_LINKER_FLAGS "-Wl,--disable-new-dtags"). This makes use of RPATH instead of RUNPATH. RPATH works with indirect shared library dependencies, whereas RUNPATH does not. This, together with the changes from this pull request, allows us to load all shared libraries without having to rely on LD_LIBRARY_PATH. However, RPATH is deprecated, so I don't think this is a good general solution.

Would it be an option to set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE) globally for ament_cmake? I am not sure about any downsides of this.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There has been no activity here in quite a while, so I just wanted to ask if there is any update on this. As a workaround, we are currently running our node in a root shell, but of course we would prefer not to.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also wanted to check on the status of this. I'm working on a linuxdeploy plugin for building AppImages that bundle a ROS2 app and its dependencies (similar to the linuxdeploy qt plugin).

Certain dependencies, like RMW implementations, are searched for at runtime with RPATH/RUNPATH excluded, so even if the executable's RPATH points to the place where these libraries are located, find_library_path() fails to pick them up. Would be nice to not have to rely on setting environment variables like LD_LIBRARY_PATH or having to source the setup.bash scripts as a custom init step in the AppImage's entrypoint (they are already slow enough to start, due to the squashfs mounting step 🙂)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I think there are two different things going on in this PR (please correct me if I'm wrong):

  1. We are adding code to search the RPATH/RUNPATH for symbols (these are the changes in find_library.cpp).
  2. We are switching the code to force RPATH instead of RUNPATH (this is the initial changes in CMakeLists.txt).

I'm fine with the first one; in our current system, that will really have no effect, but gets us closer to what @wieset and others want. I'm much less sure about the implications about the second one, which I think is what is mostly holding this up.

So my suggestion here is that we split out the second part from this PR, and just concentrate on getting the "search in RPATH" changes in. Once that is in, we can consider the "force RUNPATH -> RPATH" separately. Does that make sense?

(also, this PR needs a rebase since things have changed in the meantime)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, there are two things going on. I think the first one is obsolete now with ros2/rcutils#320 which is great! The second one is actually a bit different. It does not force RPATH instead of RUNPATH at all. Even though CMake uses RPATH in their variable names, it refers to RUNPATH since RPATH is deprecated.

  • BUILD_RPATH ${CMAKE_CURRENT_BINARY_DIR} is necessary so that RUNPATH in the build binary gets populated with the path to the test library
  • INSTALL_RPATH_USE_LINK_PATH TRUE is necessary so that RUNPATH in the installed binary gets populated properly

However, thinking about it again, INSTALL_RPATH_USE_LINK_PATH TRUE needs to be set for ALL packages so that RUNPATH can be used as an alternative to LD_LIBRARY_PATH. The reason for this is that RUNPATH only works for direct shared object dependencies. See also the discussion here: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1737608

And this is what's necessary for us to be able to run a node with setcap since it drops LD_LIBRARY_PATH. A solution I see would be to have set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE) in the ament_package() macro.

I hope this explanation is clear. What do you think about closing this PR and continuing the discussion in https://github.com/ament/ament_cmake/?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope this explanation is clear. What do you think about closing this PR and continuing the discussion in https://github.com/ament/ament_cmake/?

That sounds reasonable to me, thanks. I'll go ahead and close this; please feel free to open an issue over there.

ament_target_dependencies(${PROJECT_NAME} rcutils)
ament_export_libraries(${PROJECT_NAME})

Expand Down Expand Up @@ -71,12 +75,31 @@ if(BUILD_TESTING)

ament_add_gtest(test_endian test/test_endian.cpp)

if(WIN32)
set(LIBRARY_PATH_VAR "PATH")
elseif(APPLE)
set(LIBRARY_PATH_VAR "DYLD_LIBRARY_PATH")
else()
set(LIBRARY_PATH_VAR "LD_LIBRARY_PATH")
endif()

add_library(test_library SHARED test/test_library.cpp)

ament_add_gtest(test_find_library test/test_find_library.cpp)
target_link_libraries(test_find_library ${PROJECT_NAME} test_library)
set_tests_properties(test_find_library PROPERTIES
ENVIRONMENT
"_TEST_LIBRARY_DIR=$<TARGET_FILE_DIR:test_library>;_TEST_LIBRARY=$<TARGET_FILE:test_library>")
"${LIBRARY_PATH_VAR}=$<TARGET_FILE_DIR:test_library>;_TEST_LIBRARY=$<TARGET_FILE:test_library>")

# Test with empty LD_LIBRARY_PATH to emulate setcap / setuid executable.
# Using RUNPATH to find library instead.
if(NOT (WIN32 OR APPLE))
ament_add_gtest(test_find_library_runpath test/test_find_library.cpp)
target_link_libraries(test_find_library_runpath ${PROJECT_NAME} test_library)
set_tests_properties(test_find_library_runpath PROPERTIES
ENVIRONMENT
"${LIBRARY_PATH_VAR}=;_TEST_LIBRARY=$<TARGET_FILE:test_library>")
endif()
endif()

ament_package()
Expand Down
4 changes: 2 additions & 2 deletions include/rcpputils/find_library.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ namespace rcpputils
{

/// Finds a library located in the OS's specified environment variable for
/// library paths and returns the absolute filesystem path, including the
/// appropriate prefix and extension.
/// library paths or in RUNPATH on Linux and returns the absolute filesystem
/// path, including the appropriate prefix and extension.
/**
* The environment variable and file format per platform:
* * Linux: `${LD_LIBRARY_PATH}`, `lib{}.so`
Expand Down
40 changes: 38 additions & 2 deletions src/find_library.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@

#include "rcpputils/find_library.hpp"

#if !defined(_WIN32) && !defined(__APPLE__)
#include <link.h>
wieset marked this conversation as resolved.
Show resolved Hide resolved
#endif

#include <cassert>
#include <cstddef>

Expand Down Expand Up @@ -69,12 +73,44 @@ std::list<std::string> split(const std::string & value, const char delimiter)
return list;
}

// Retrieves a list of paths from the RUNPATH header on Linux.
// Useful when LD_LIBRARY_PATH is ignored for setcap / setuid executables.
std::list<std::string> retrieve_runpath_paths()
{
#if defined(_WIN32) || defined(__APPLE__)
// Return empty list for win / macos
return std::list<std::string>();
#else
const ElfW(Dyn) * dyn = _DYNAMIC;
const ElfW(Dyn) * runpath = NULL;
const char * strtab = NULL;

for (; dyn->d_tag != DT_NULL; ++dyn) {
if (dyn->d_tag == DT_RUNPATH) {
runpath = dyn;
} else if (dyn->d_tag == DT_STRTAB) {
strtab = (const char *)dyn->d_un.d_val;
}
}

std::string search_path;
if (strtab != NULL && runpath != NULL) {
search_path = std::string(strtab + runpath->d_un.d_val);
}

return split(search_path, kPathSeparator);
#endif
}

} // namespace

std::string find_library_path(const std::string & library_name)
{
std::string search_path = get_env_var(kPathVar);
std::list<std::string> search_paths = split(search_path, kPathSeparator);
std::string search_path_env = get_env_var(kPathVar);
wieset marked this conversation as resolved.
Show resolved Hide resolved
std::list<std::string> search_paths = split(search_path_env, kPathSeparator);
wieset marked this conversation as resolved.
Show resolved Hide resolved

std::list<std::string> search_paths_runpath = retrieve_runpath_paths();
wieset marked this conversation as resolved.
Show resolved Hide resolved
search_paths.splice(search_paths.cend(), search_paths_runpath);

std::string filename = kSolibPrefix;
filename += library_name + kSolibExtension;
Expand Down
23 changes: 1 addition & 22 deletions test/test_find_library.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -37,29 +37,8 @@ TEST(test_find_library, find_library)
expected_library_path = _expected_library_path;
}

const char * test_lib_dir{};
EXPECT_EQ(rcutils_get_env("_TEST_LIBRARY_DIR", &test_lib_dir), nullptr);
EXPECT_NE(test_lib_dir, nullptr);

// Set our relevant path variable.
const char * env_var{};
#ifdef _WIN32
env_var = "PATH";
#elif __APPLE__
env_var = "DYLD_LIBRARY_PATH";
#else
env_var = "LD_LIBRARY_PATH";
#endif

#ifdef _WIN32
EXPECT_EQ(_putenv_s(env_var, test_lib_dir), 0);
#else
const int override = 1;
EXPECT_EQ(setenv(env_var, test_lib_dir, override), 0);
#endif

// Positive test.
const std::string test_lib_actual = find_library_path("test_library");
std::string test_lib_actual = find_library_path("test_library");
wieset marked this conversation as resolved.
Show resolved Hide resolved
EXPECT_EQ(test_lib_actual, expected_library_path);

// (Hopefully) Negative test.
Expand Down