stdgpu target fails the CMake configure step with CUDA 12.4 due to changes in Thrust version header

[patrikhuber]

Checklist

• I have searched for similar issues.
• For Python issues, I have tested with the latest development wheel.
• I have checked the release documentation and the latest documentation (for main branch).

Steps to reproduce the issue

I first cloned Open3D by:

git clone https://github.com/isl-org/Open3D.git
cd Open3D

Then, I build Open3D (on Windows 11 with VS2022, latest MSVC, with CUDA 12.4) with:

mkdir build
cd build
cmake -DBUILD_SHARED_LIBS=OFF -DBUILD_CUDA_MODULE=ON -DBUILD_WEBRTC=OFF ..
# Then Open Visual Studio 2022 and build Open3D (or just the stdgpu target).

Error message

[13/148] Performing configure step for 'ext_stdgpu'
  CMake Error at cmake/Findthrust.cmake:17 (math):
  math cannot parse the expression: "200301 // macro expansion with ##
  requires this to be a single value / 100000": syntax error, unexpected
  exp_DIVIDE (9).
Call Stack (most recent call first):
  src/stdgpu/CMakeLists.txt:7 (find_package)
CMake Error at C:/Program Files/Microsoft Visual Studio/2022/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
    Could NOT find thrust: Found unsuitable version "ERROR.ERROR.ERROR", but
    required is at least "1.9.2" (found C:/Program Files/NVIDIA GPU Computing
    Toolkit/CUDA/v12.4/include)

Open3D, Python and System information

- Operating system: Windows 11 64-bit
- Python version: not applicable
- Open3D version: latest main (8 Apr 2024)
- System architecture: x64
- Is this a remote workstation?: no
- How did you install Open3D?: build from source
- Compiler version (if built from source): Microsoft (R) C/C++ Optimizing Compiler Version 19.39.33523 for x64

Additional information

This error occurs because cmake/Findthrust.cmake extracts the Thrust version from Thrust's header file. In CUDA 12.3 (and presumably earlier), the line in the Thrust header looks like:

#define THRUST_VERSION 200200

And in CUDA 12.4 it is:

#define THRUST_VERSION 200301 // macro expansion with ## requires this to be a single value

Thus, CMake's regexp in cmake/Findthrust.cmake fails as it encounters the unexpected trailing comment on that line.

[stotko]

Author of stdgpu here. This has been fixed upstream in stotko/stdgpu#408, so updating the dependency in Open3D should fix it there as well.

[manhha1402]

Hi @stotko, do you mean we need to update this file:
https://github.com/isl-org/Open3D/blob/main/3rdparty/stdgpu/stdgpu.cmake

[patrikhuber]

@stotko Hi and thank you for the reply here, that's great!

@manhha1402 Yes I believe that is exactly what would need to be done in the Open3D repo - update the commit hash in that cmake file.

[TheExDeus]

Sadly with the last stdgpu version Open3D seems to be incompatible. It shows errors like:

#13 247.4 /opt/Open3D/build/stdgpu/include/stdgpu/impl/unordered_base_detail.cuh(1230): error: a value of type "cuda::std::__4::pair<const open3d::utility::MiniVec<int64_t, 1>, open3d::core::buf_index_t> *" cannot be assigned to an entity of type "stdgpu::pair<const open3d::utility::MiniVec<int64_t, 1>, open3d::core::buf_index_t> *"
#13 247.4       result._values = allocator_traits<allocator_type>::allocate(result._allocator, total_count);

[stotko]

Sadly with the last stdgpu version Open3D seems to be incompatible. It shows errors like:

#13 247.4 /opt/Open3D/build/stdgpu/include/stdgpu/impl/unordered_base_detail.cuh(1230): error: a value of type "cuda::std::__4::pair<const open3d::utility::MiniVec<int64_t, 1>, open3d::core::buf_index_t> *" cannot be assigned to an entity of type "stdgpu::pair<const open3d::utility::MiniVec<int64_t, 1>, open3d::core::buf_index_t> *"
#13 247.4       result._values = allocator_traits<allocator_type>::allocate(result._allocator, total_count);

The incompatibility stems from the recent change in stdgpu to move to a custom implementation of pair instead of pointing to thrust::pair which resolves some namespace issues, see stotko/stdgpu#425.

Adapting Open3D to that change should mostly reduce to replace thrust::pair by the intended stdgpu::pair here:

Open3D/cpp/open3d/core/hashmap/CUDA/StdGPUHashBackend.h

Line 93 in f4e1fa9

StdGPUAllocator<thrust::pair<const Key, buf_index_t>>;

Open3D/cpp/open3d/core/hashmap/CUDA/StdGPUHashBackend.h

Line 255 in f4e1fa9

operator()(const thrust::pair<Key, buf_index_t>& x) const {

[asdiujvaosidujdposiadjapoij]

@stotko I applied your changes and now I can compile and install Open3d with stdgpu. However, the OnlineSLAMRGBD example does not work. It shows the initial windows but when it fails when touching the resume/pause button and starts integrating.

I ran the example, compiled in RelWithDebInfo inside WinDbg, and it shows this stack trace:

[0x0]   KERNELBASE!RaiseException+0x6c   0x570f9fdf00   0x7ffcf5e95267   
[0x1]   VCRUNTIME140!_CxxThrowException+0x97   0x570f9fdfe0   0x7ff721acabff   
[0x2]   OnlineSLAMRGBD!thrust::THRUST_200500_860_NS::cuda_cub::throw_on_error+0x121   (Inline Function)   (Inline Function)   
[0x3]   OnlineSLAMRGBD!thrust::THRUST_200500_860_NS::cuda_cub::detail::transform_reduce_n_impl<thrust::THRUST_200500_860_NS::cuda_cub::par_t,thrust::THRUST_200500_860_NS::counting_iterator<int,thrust::THRUST_200500_860_NS::use_default,thrust::THRUST_200500_860_NS::use_default,thrust::THRUST_200500_860_NS::use_default>,__int64,stdgpu::detail::count_block_bits<unsigned int>,int,stdgpu::plus<int> >+0x27f   0x570f9fe040   0x7ff721aca926   
[0x4]   OnlineSLAMRGBD!thrust::THRUST_200500_860_NS::cuda_cub::transform_reduce<thrust::THRUST_200500_860_NS::cuda_cub::par_t,thrust::THRUST_200500_860_NS::counting_iterator<int,thrust::THRUST_200500_860_NS::use_default,thrust::THRUST_200500_860_NS::use_default,thrust::THRUST_200500_860_NS::use_default>,stdgpu::detail::count_block_bits<unsigned int>,int,stdgpu::plus<int> >+0x46   0x570f9fe1a0   0x7ff721aca8d8   
[0x5]   OnlineSLAMRGBD!thrust::THRUST_200500_860_NS::transform_reduce<thrust::THRUST_200500_860_NS::cuda_cub::par_t,thrust::THRUST_200500_860_NS::counting_iterator<int,thrust::THRUST_200500_860_NS::use_default,thrust::THRUST_200500_860_NS::use_default,thrust::THRUST_200500_860_NS::use_default>,stdgpu::detail::count_block_bits<unsigned int>,int,stdgpu::plus<int> >+0x28   0x570f9fe1f0   0x7ff721aca96d   
[0x6]   OnlineSLAMRGBD!stdgpu::transform_reduce_index<int,thrust::THRUST_200500_860_NS::cuda_cub::par_t const &,int,stdgpu::plus<int>,stdgpu::detail::count_block_bits<unsigned int>,void,0>+0x2d   0x570f9fe240   0x7ff721acb962   
[0x7]   OnlineSLAMRGBD!stdgpu::bitset<unsigned int,open3d::core::StdGPUAllocator<unsigned int> >::count+0x45   (Inline Function)   (Inline Function)   
[0x8]   OnlineSLAMRGBD!stdgpu::bitset<unsigned int,open3d::core::StdGPUAllocator<unsigned int> >::set+0x7f   (Inline Function)   (Inline Function)   
[0x9]   OnlineSLAMRGBD!stdgpu::detail::vector_clear_iota<thrust::THRUST_200500_860_NS::cuda_cub::par_t const &,int,open3d::core::StdGPUAllocator<int> >+0xc2   0x570f9fe290   0x7ff721aa6158   
[0xa]   OnlineSLAMRGBD!stdgpu::detail::unordered_base<open3d::utility::MiniVec<int,3>,stdgpu::pair<open3d::utility::MiniVec<int,3> const ,unsigned int>,stdgpu::detail::select1st<stdgpu::pair<open3d::utility::MiniVec<int,3> const ,unsigned int> >,open3d::utility::MiniVecHash<int,3>,open3d::utility::MiniVecEq<int,3>,open3d::core::StdGPUAllocator<stdgpu::pair<open3d::utility::MiniVec<int,3> const ,unsigned int> > >::createDeviceObject<thrust::THRUST_200500_860_NS::cuda_cub::par_t const &,void,0>+0x248   0x570f9fe2f0   0x7ff721aa7e5f   
[0xb]   OnlineSLAMRGBD!stdgpu::unordered_map<open3d::utility::MiniVec<int,3>,unsigned int,open3d::utility::MiniVecHash<int,3>,open3d::utility::MiniVecEq<int,3>,open3d::core::StdGPUAllocator<stdgpu::pair<open3d::utility::MiniVec<int,3> const ,unsigned int> > >::createDeviceObject<thrust::THRUST_200500_860_NS::cuda_cub::par_t const &,void,0>+0x4f   0x570f9fe3f0   0x7ff721b4ddab   
[0xc]   OnlineSLAMRGBD!stdgpu::unordered_map<open3d::utility::MiniVec<int,3>,unsigned int,open3d::utility::MiniVecHash<int,3>,open3d::utility::MiniVecEq<int,3>,open3d::core::StdGPUAllocator<stdgpu::pair<open3d::utility::MiniVec<int,3> const ,unsigned int> > >::createDeviceObject+0x1b   0x570f9fe4e0   0x7ff721b07470   
[0xd]   OnlineSLAMRGBD!open3d::core::StdGPUHashBackend<open3d::utility::MiniVec<int,3>,open3d::utility::MiniVecHash<int,3>,open3d::utility::MiniVecEq<int,3> >::Allocate+0xe0   0x570f9fe510   0x7ff721ad268a   
[0xe]   OnlineSLAMRGBD!open3d::core::StdGPUHashBackend<open3d::utility::MiniVec<int,3>,open3d::utility::MiniVecHash<int,3>,open3d::utility::MiniVecEq<int,3> >::StdGPUHashBackend<open3d::utility::MiniVec<int,3>,open3d::utility::MiniVecHash<int,3>,open3d::utility::MiniVecEq<int,3> >+0x12a   0x570f9fe630   0x7ff721afb63e   
[0xf]   OnlineSLAMRGBD!std::_Construct_in_place+0x13   (Inline Function)   (Inline Function)   
[0x10]   OnlineSLAMRGBD!std::_Ref_count_obj2<open3d::core::StdGPUHashBackend<open3d::utility::MiniVec<int,3>,open3d::utility::MiniVecHash<int,3>,open3d::utility::MiniVecEq<int,3> > >::{ctor}+0x32   (Inline Function)   (Inline Function)   
[0x11]   OnlineSLAMRGBD!std::make_shared+0x4c   (Inline Function)   (Inline Function)   
[0x12]   OnlineSLAMRGBD!open3d::core::CreateCUDAHashBackend::__l10::<lambda_d37eec2021cbaee434354b90e0025a4f>::()::__l53::<lambda_a3bdcf899d229b726082f9305d608735>::operator()+0x4c   (Inline Function)   (Inline Function)   
[0x13]   OnlineSLAMRGBD!<lambda_d37eec2021cbaee434354b90e0025a4f>::operator()+0x4ae   0x570f9fe670   0x7ff721b0bddc   
[0x14]   OnlineSLAMRGBD!open3d::core::CreateCUDAHashBackend+0x14c   0x570f9fe6f0   0x7ff7216878ad   
[0x15]   OnlineSLAMRGBD!open3d::core::CreateDeviceHashBackend+0x8d   0x570f9fe7c0   0x7ff7215c2560   
[0x16]   OnlineSLAMRGBD!open3d::core::HashMap::Init+0x110   0x570f9fe820   0x7ff7215c069d   
[0x17]   OnlineSLAMRGBD!open3d::core::HashMap::HashMap+0x7d   0x570f9fe8b0   0x7ff721737fa6   
[0x18]   OnlineSLAMRGBD!std::_Construct_in_place+0x37   (Inline Function)   (Inline Function)   
[0x19]   OnlineSLAMRGBD!std::_Ref_count_obj2<open3d::core::HashMap>::{ctor}+0x56   (Inline Function)   (Inline Function)   
[0x1a]   OnlineSLAMRGBD!std::make_shared+0x70   (Inline Function)   (Inline Function)   
[0x1b]   OnlineSLAMRGBD!open3d::t::geometry::VoxelBlockGrid::VoxelBlockGrid+0x306   0x570f9fe8e0   0x7ff7215eab88   
[0x1c]   OnlineSLAMRGBD!open3d::t::pipelines::slam::Model::Model+0x2b8   0x570f9feaf0   0x7ff7214ea9cb   
[0x1d]   OnlineSLAMRGBD!std::_Construct_in_place+0x21   (Inline Function)   (Inline Function)   
[0x1e]   OnlineSLAMRGBD!std::_Ref_count_obj2<open3d::t::pipelines::slam::Model>::{ctor}+0x3a   (Inline Function)   (Inline Function)   
[0x1f]   OnlineSLAMRGBD!std::make_shared+0x54   (Inline Function)   (Inline Function)   
[0x20]   OnlineSLAMRGBD!<lambda_a3b64392188269a5207f98bdf7d4c150>::operator()+0x80b   0x570f9fedc0   0x7ff7215f3ef4   
[0x21]   OnlineSLAMRGBD!std::_Func_class<void>::operator()+0x1a   (Inline Function)   (Inline Function)   
[0x22]   OnlineSLAMRGBD!open3d::visualization::gui::Application::ProcessQueuedEvents+0x304   0x570f9ff4d0   0x7ff7215f49f7   
[0x23]   OnlineSLAMRGBD!open3d::visualization::gui::Application::RunOneTick+0x3a7   0x570f9ff590   0x7ff7215f44f0   
[0x24]   OnlineSLAMRGBD!open3d::visualization::gui::Application::Run+0x40   0x570f9ff730   0x7ff721502049   
[0x25]   OnlineSLAMRGBD!main+0xd99   0x570f9ff770   0x7ff723f3b71c   
[0x26]   OnlineSLAMRGBD!invoke_main+0x22   (Inline Function)   (Inline Function)   
[0x27]   OnlineSLAMRGBD!__scrt_common_main_seh+0x10c   0x570f9ffc00   0x7ffd5118257d   
[0x28]   KERNEL32!BaseThreadInitThunk+0x1d   0x570f9ffc40   0x7ffd51e6af28   
[0x29]   ntdll!RtlUserThreadStart+0x28   0x570f9ffc70   0x0   

I am working with Windows 11, CUDA 12.6, CUDNN 9.4, compiled with Cmake + Visual Studio 2022.

[stotko]

@asdiujvaosidujdposiadjapoij I tested this on Ubuntu 22.04 with CUDA 12.6 + latest stdgpu + pair fix and the OnlineSLAMRGBD example ran without errors on my setup. So, the error you observed is quite strange. Is your NVIDIA driver up-to-date? CUDA typically requires the most recent driver available at its release time. For CUDA 12.6, this means at least driver version 560 is needed (see https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html)

[patrikhuber]

With the above help, I've now also been able to compile a static version of Open3D with CUDA 12.6 on Windows 11, latest MSVC (14.42.34433), and Visual Studio's File->Open CMake. I've used the latest stdgpu master + pair fix.

Out of interest, I just ran the OnlineSLAMRGBD.exe example, and it runs fine, without a crash.

[stotko]

Since the bug seems to become increasing problematic, I've went ahead and submitted the respective fix in #7083.