Still failed to detect OP_CXX_ABI with the latest devel version

[haideeyuan]

I have already installed cuda toolkit 11.6 and tensorflow 2.9.0.
Then I cloned the repository by git clone https://github.com/deepmodeling/deepmd-kit.git
Next, I changed the branch by git checkout -b dev origin/devel and set the parameters by deepmd_source_dir=path_to_deepmd and export DP_VARIANT=cuda
Finally, I ran the pip install .
I got the following errors: "Failed to detect OP_CXX_ABI, please set it manually" may be the most critical seen from my perspective.

I have seen that there is a bug in TF version >= 2.9.0, and which can be solved with the latest devel version.
However, in the latest devel version, I still met the OP_CXX_ABI problem.
By the way, I tried TF version 2.8.0, which is good.

The whole output of pip install .:

  (dp-zyy) [zhouyy@centos1271 deepmd-kit]$ pip install .
  Processing /home/zhouyy/deepmd-kit
    Installing build dependencies ... done
    Getting requirements to build wheel ... done
    Preparing metadata (pyproject.toml) ... done
  Requirement already satisfied: h5py in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages (from deepmd-kit==2.1.6.dev26+ge2a3823) (3.7.0)
  Requirement already satisfied: python-hostlist>=1.21 in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages (from deepmd-kit==2.1.6.dev26+ge2a3823) (1.22)
  Requirement already satisfied: wcmatch in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages (from deepmd-kit==2.1.6.dev26+ge2a3823) (8.4.1)
  Requirement already satisfied: scipy in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages (from deepmd-kit==2.1.6.dev26+ge2a3823) (1.9.3)
  Requirement already satisfied: pyyaml in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages (from deepmd-kit==2.1.6.dev26+ge2a3823) (6.0)
  Requirement already satisfied: dargs>=0.2.6 in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages (from deepmd-kit==2.1.6.dev26+ge2a3823) (0.3.3)
  Requirement already satisfied: numpy in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages (from deepmd-kit==2.1.6.dev26+ge2a3823) (1.23.4)
  Requirement already satisfied: packaging in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages (from deepmd-kit==2.1.6.dev26+ge2a3823) (21.3)
  Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages (from packaging->deepmd-kit==2.1.6.dev26+ge2a3823) (3.0.9)
  Requirement already satisfied: bracex>=2.1.1 in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages (from wcmatch->deepmd-kit==2.1.6.dev26+ge2a3823) (2.3.post1)
  Building wheels for collected packages: deepmd-kit
    Building wheel for deepmd-kit (pyproject.toml) ... error
    error: subprocess-exited-with-error
    
    × Building wheel for deepmd-kit (pyproject.toml) did not run successfully.
    │ exit code: 1
    ╰─> [98 lines of output]
        Not searching for unused variables given on the command line.
        -- The C compiler identification is GNU 7.3.1
        -- Detecting C compiler ABI info
        -- Detecting C compiler ABI info - done
        -- Check for working C compiler: /opt/rh/devtoolset-7/root/usr/bin/cc - skipped
        -- Detecting C compile features
        -- Detecting C compile features - done
        -- The CXX compiler identification is GNU 7.3.1
        -- Detecting CXX compiler ABI info
        -- Detecting CXX compiler ABI info - done
        -- Check for working CXX compiler: /opt/rh/devtoolset-7/root/usr/bin/c++ - skipped
        -- Detecting CXX compile features
        -- Detecting CXX compile features - done
        -- Configuring done
        -- Generating done
        -- Build files have been written to: /home/zhouyy/deepmd-kit/_cmake_test_compile/build
        -- The C compiler identification is GNU 7.3.1
        -- The CXX compiler identification is GNU 7.3.1
        -- Detecting C compiler ABI info
        -- Detecting C compiler ABI info - done
        -- Check for working C compiler: /opt/rh/devtoolset-7/root/usr/bin/cc - skipped
        -- Detecting C compile features
        -- Detecting C compile features - done
        -- Detecting CXX compiler ABI info
        -- Detecting CXX compiler ABI info - done
        -- Check for working CXX compiler: /opt/rh/devtoolset-7/root/usr/bin/c++ - skipped
        -- Detecting CXX compile features
        -- Detecting CXX compile features - done
        -- Found Git: /usr/bin/git (found version "1.8.3.1")
        -- Supported model version: 1.1
        -- Found CUDA: /usr/local/cuda (found version "11.6")
        -- Found CUDA in /usr/local/cuda, build nv GPU support
        -- Will not build AMD GPU support
        -- Disabled cpp interface build, looking for tensorflow_framework
        CMake Warning (dev) at cmake/Findtensorflow.cmake:158 (file):
          You have used file(GET_RUNTIME_DEPENDENCIES) in project mode.  This is
          probably not what you intended to do.  Instead, please consider using it in
          an install(CODE) or install(SCRIPT) command.  For example:
        
            install(CODE [[
              file(GET_RUNTIME_DEPENDENCIES
                # ...
                )
              ]])
        Call Stack (most recent call first):
          CMakeLists.txt:95 (find_package)
        This warning is for project developers.  Use -Wno-dev to suppress it.
        
        -- Found TensorFlow: /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages/tensorflow/include;/home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages/tensorflow/include, , /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages/tensorflow/libtensorflow_framework.so.2  in /home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages/tensorflow;/home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages/tensorflow/../tensorflow_core (found version "2.9.0")
        CMake Error at cmake/Findtensorflow.cmake:269 (message):
          Failed to detect OP_CXX_ABI, please set it manually
        Call Stack (most recent call first):
          CMakeLists.txt:95 (find_package)
        
        
        -- Configuring incomplete, errors occurred!
        See also "/home/zhouyy/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-build/CMakeFiles/CMakeOutput.log".
          File "/tmp/pip-build-env-2gnqc4zc/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 637, in setup
            env = cmkr.configure(
          File "/tmp/pip-build-env-2gnqc4zc/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 328, in configure
            raise SKBuildError(
        
        
        --------------------------------------------------------------------------------
        -- Trying "Ninja" generator
        --------------------------------
        ---------------------------
        ----------------------
        -----------------
        ------------
        -------
        --
        --
        -------
        ------------
        -----------------
        ----------------------
        ---------------------------
        --------------------------------
        -- Trying "Ninja" generator - success
        --------------------------------------------------------------------------------
        
        Configuring Project
          Working directory:
            /home/zhouyy/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-build
          Command:
            cmake /home/zhouyy/deepmd-kit/source -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/home/zhouyy/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.4 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-2gnqc4zc/overlay/lib/python3.10/site-packages/skbuild/resources/cmake -DPython3_EXECUTABLE:FILEPATH=/home/zhouyy/miniconda2/envs/dp-zyy/bin/python -DPython3_INCLUDE_DIR:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/include/python3.10 -DPython3_LIBRARY:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/lib/libpython3.10.so -DPython_EXECUTABLE:FILEPATH=/home/zhouyy/miniconda2/envs/dp-zyy/bin/python -DPython_INCLUDE_DIR:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/include/python3.10 -DPython_LIBRARY:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/lib/libpython3.10.so -DPYTHON_EXECUTABLE:FILEPATH=/home/zhouyy/miniconda2/envs/dp-zyy/bin/python -DPYTHON_INCLUDE_DIR:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/include/python3.10 -DPYTHON_LIBRARY:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/lib/libpython3.10.so -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-2gnqc4zc/overlay/lib/python3.10/site-packages/ninja/data/bin/ninja -DTENSORFLOW_ROOT:STRING=/home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages/tensorflow -DBUILD_PY_IF:BOOL=TRUE -DBUILD_CPP_IF:BOOL=FALSE -DUSE_CUDA_TOOLKIT:BOOL=TRUE -DCMAKE_BUILD_TYPE:STRING=Release
        
        Traceback (most recent call last):
        
        An error occurred while configuring with CMake.
          Command:
            cmake /home/zhouyy/deepmd-kit/source -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/home/zhouyy/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.4 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-2gnqc4zc/overlay/lib/python3.10/site-packages/skbuild/resources/cmake -DPython3_EXECUTABLE:FILEPATH=/home/zhouyy/miniconda2/envs/dp-zyy/bin/python -DPython3_INCLUDE_DIR:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/include/python3.10 -DPython3_LIBRARY:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/lib/libpython3.10.so -DPython_EXECUTABLE:FILEPATH=/home/zhouyy/miniconda2/envs/dp-zyy/bin/python -DPython_INCLUDE_DIR:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/include/python3.10 -DPython_LIBRARY:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/lib/libpython3.10.so -DPYTHON_EXECUTABLE:FILEPATH=/home/zhouyy/miniconda2/envs/dp-zyy/bin/python -DPYTHON_INCLUDE_DIR:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/include/python3.10 -DPYTHON_LIBRARY:PATH=/home/zhouyy/miniconda2/envs/dp-zyy/lib/libpython3.10.so -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-2gnqc4zc/overlay/lib/python3.10/site-packages/ninja/data/bin/ninja -DTENSORFLOW_ROOT:STRING=/home/zhouyy/miniconda2/envs/dp-zyy/lib/python3.10/site-packages/tensorflow -DBUILD_PY_IF:BOOL=TRUE -DBUILD_CPP_IF:BOOL=FALSE -DUSE_CUDA_TOOLKIT:BOOL=TRUE -DCMAKE_BUILD_TYPE:STRING=Release
          Source directory:
            /home/zhouyy/deepmd-kit/source
          Working directory:
            /home/zhouyy/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-build
        Please see CMake's output for more information.
        [end of output]
    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for deepmd-kit
  Failed to build deepmd-kit
  ERROR: Could not build wheels for deepmd-kit, which is required to install pyproject.toml-based projects”

[njzjz]

devtoolset on RHEL6 and RHEL7 does not support _GLIBCXX_USE_CXX11_ABI=1. See https://bugzilla.redhat.com/show_bug.cgi?id=1546704

[haideeyuan]

Thank you so much for your reply.
Do you mean that I should change my OS to someone except RHEL6 and RHL7, for example, Ubuntu?

[haideeyuan]

I changed my OS to Ubuntu 20.04 and it works. Thanks again!

[njzjz]

Indeed the issue comes from the compiler, so it is enough to just change the compiler.

[yulongliuno1]

I also encountered the same problem during the source code installation process. I reported the same error in centos7 and even centos stream 8, but successfully installed in ubuntu18.04 and 20.04. Is this a problem that can only be solved by changing the system? However, most server versions are still based on centos 7, which is very inconvenient. Thank you very much!