Installing sf from compiled GDAL, PROJ and GEOS in Red Hat Enterprise Linux RHEL 8.9 (no admin user)

[alexyshr]

LSB Version:	:core-4.1-amd64:core-4.1-noarch
Distributor ID:	RedHatEnterprise
Description:	Red Hat Enterprise Linux release 8.9 (Ootpa)
Release:	8.9
Codename:	Ootpa
----
R version 4.3.3 (2024-02-29) -- "Angel Food Cake"

I need to install sf from GitHub to allow proper reading of HDF5 files. I don't have an admin user. IT support can help me to install from repositories but not to compile from source.

IT support updated all the dependencies. Library names: udunits2-devel, libcurl-devel, libxml2-devel, libgdal-dev, proj-dev, sqlite-devel, geos-config. The versions of the libraries were at that point as shown bellow:

library(sf)
#> Linking to GEOS 3.7.2, GDAL 3.0.4, PROJ 6.3.2; sf_use_s2() is TRUE
library(stars)
#> Loading required package: abind
library(terra)
#> terra 1.7.74

Nonetheless, the sf package was unable to read the coordinates from HDF5 files.

So, I started the process of complying from sources. I managed to compile everything from 1 to 11. I got error only on 12 (installing sf from GitHub)

1. Install the dependencies (udunits2-devel, libcurl-devel, libxml2-devel) (Admin IT user)

2. Compile SQLite3
   Download: https://www.sqlite.org/2024/sqlite-autoconf-3450300.tar.gz

mkdir sqlite
tar xvfz sqlite-autoconf-3450300.tar.gz
cd sqlite-autoconf-3450300
./configure
make
make install DESTDIR=~/sqlite

3. Edit .bashrc file to add or edit environment variables

export PATH="/home/myuser/sqlite:$PATH"
export SQLite3_INCLUDE_DIR=/home/myuser/sqlite/usr/local/include
export SQLite3_LIBRARY=/home/myuser/sqlite/usr/local/lib/libsqlite3.so
export LD_LIBRARY_PATH="/home/myuser/sqlite/usr/local/lib:$LD_LIBRARY_PATH"

4. Compile GTest

Procedure: here

mkdir ~/work-tmp
cd work-tmp/
git clone https://github.com/google/googletest.git
cd googletest/
ls -la
git tag -l
git checkout v1.14.0
mkdir build
cd build
cmake ..
make
mkdir ~/googletest
make install DESTDIR=~/googletest

5. Edit .bashrc

export PATH="~/googletest/usr/local/include:$PATH"
export PATH="~/googletest/usr/local/lib64:$PATH"

6. Compiling PROJ

Download: here
Procedure: here

mkdir proj940
tar xvf proj-9.4.0.tar.gz
cd proj-9.4.0
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$HOME/proj940 .. -DCMAKE_CXX_STANDARD=14 -DCMAKE_CXX_STANDARD_REQUIRED=ON
cmake --build .
cmake --build . --target install

7. Edit .bashrc

export PROJ_INCLUDE_DIR=/home/myuser/proj940/include
export PROJ_LIBRARY_RELEASE=/home/myuser/proj940/lib64/libproj.so

8. Compiling GEOS

The installer: here
The procedure: here

mkdir geos3121
tar xvfj geos-3.12.1.tar.bz2
cd geos-3.12.1
mkdir _build
cd _build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/geos3121 .. -DCMAKE_CXX_STANDARD=14 -DCMAKE_CXX_STANDARD_REQUIRED=ON
make
ctest
make install

9)Edit .bashrc

export GEOS_INCLUDE_DIR=/home/myuser/geos3121/include
export GEOS_LIBRARY=/home/myuser/geos3121/lib64/libgeos_c.so
export GEOS_DIR=/home/myuser/geos3121

10. Compiling GDAL

Download: here
Procedure: here

mkdir gdal385
tar xvf gdal-3.8.5.tar.gz
cd gdal-3.8.5
mkdir build
cd build

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/gdal385 .. -DCMAKE_CXX_STANDARD=14 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DPROJ_INCLUDE_DIR=$PROJ_INCLUDE_DIR -DPROJ_LIBRARY_RELEASE=$PROJ_LIBRARY_RELEASE -DGEOS_INCLUDE_DIR=$GEOS_INCLUDE_DIR -DGEOS_LIBRARY=$GEOS_LIBRARY -DSQLite3_INCLUDE_DIR=$SQLite3_INCLUDE_DIR -DSQLite3_LIBRARY=$SQLite3_LIBRARY -DGDAL_USE_SQLITE3=ON

cmake --build .
cmake --build . --target install

The outcome of the first cmake is
cmake_gdal_outcome.txt. There is a warning:

CMake Warning:
  Manually-specified variables were not used by the project:

    GEOS_INCLUDE_DIR
    GEOS_LIBRARY

11. edit .bashrc
    export GDAL_LIBRARY_PATH=/home/myuser/gdal385/lib64/libgdal.so

12. Install sf from GitHub using non-standard locations (compiled folders)

remotes::install_github("r-spatial/sf", configure.args = "--with-gdal-config=/home/myuser/gdal385/bin/gdal-config --with-geos-config=/home/myuser/geos3121/bin/geos-config --with-proj-include=/home/myuser/proj940/include/ --with-proj-lib=/home/myuser/proj940/lib64/")

The outcome of sf GitHub install is sf_install_github.txt. There are some issues related to:

...
libgdal.so: undefined reference to 
...
libgeos_c.so: undefined reference to 
...
collect2: error: ld returned 1 exit status
configure: Install failure: compilation and/or linkage problems.
configure: error: GDALAllRegister not found in libgdal.
ERROR: configuration failed for package ‘sf’
...

Any advice will be appreciated.

Best

[alexyshr]

I complemented the LD_LIBRARY_PATH inside .bashrc:

export LD_LIBRARY_PATH="/home/myuser/sqlite/usr/local/lib:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="~/googletest/usr/local/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="/home/myuser/proj940/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="/home/myuser/geos3121/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="/home/myuser/gdal385/lib64:$LD_LIBRARY_PATH"

After running

remotes::install_github("r-spatial/sf", configure.args = "--with-gdal-config=/home/myuser/gdal385/bin/gdal-config --with-geos-config=/home/myuser/geos3121/bin/geos-config --with-proj-include=/home/myuser/proj940/include/ --with-proj-lib=/home/myuser/proj940/lib64/")

the errors are shown below

configure: error: libproj or sqlite3 not found in standard or given locations.

Then I found this:

https://github.com/r-spatial/sf#multiple-gdal-geos-andor-proj-versions-on-your-system

Let's see if I can solve it!

[alexyshr]

This person tried setting the variable inside R with some success: https://github.com/r-spatial/sf/issues/844#issuecomment-654477840

I had to add some additional details, because after implementing his solution, I got the next error:

** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘sf’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/myuser/R/x86_64-pc-linux-gnu-library/4.4/00LOCK-sf/00new/sf/libs/sf.so':
  /home/myuser/gdal385/lib64/libgdal.so.34: undefined symbol: sqlite3_total_changes64

These are the steps to solve my issue:

1. IT (admin user) support removed proj (binaries or headers/devel) from the system.
2. According to IT support (admin user): The SQLite library (not the devel package) was not removed because it was required for Rstudio.
3. Libraries were loaded inside R by adding next lines to .Rprofile

Important: The order of loading libraries matters!
Note the use of the variable CPATH and the dynamic load of sqlite

Sys.setenv("LD_LIBRARY_PATH" = "/home/muyser/gdal385/lib64:/home/muyser/geos3121/lib64:/home/myuser/proj940/lib64:~/googletest/usr/local/lib64:/home/myuser/sqlite/usr/local/lib")
dyn.load("/home/myuser/geos3121/lib64/libgeos.so.3.12.1")
dyn.load("/home/myuser/geos3121/lib64/libgeos_c.so")
Sys.setenv("CPATH" = "/home/myuser/proj940/include:/home/myuser/sqlite/usr/local/include")
dyn.load("/home/myuser/sqlite/usr/local/lib/libsqlite3.so")
dyn.load("/home/myuser/sqlite/usr/local/lib/libsqlite3.so.0.8.6")
dyn.load("/home/myuser/proj940/lib64/libproj.so.25.9.4.0")
dyn.load("/home/myuser/proj940/lib64/libproj.so")

Then I was able to install from GitHub using:

remotes::install_github("r-spatial/sf", configure.args = "--with-gdal-config=/home/myuser/gdal385/bin/gdal-config --with-geos-config=/home/myuser/geos3121/bin/geos-config --with-proj-include=/home/myuser/proj940/include/ --with-proj-lib=/home/myuser/proj940/lib64/")

Extra Note: If you get the next error when compiling gdal, it was related to extra characters (:) in the environment variables. It was solved by removing the extra ":" at the end of some environment variables defined inside .bashrc.

[93%] Built target gcore_mdreader
CMakeFiles/GDAL.dir/build.make:2457: *** target pattern contains no '%'.  Stop.
make[1]: *** [CMakeFiles/Makefile2:4697: CMakeFiles/GDAL.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

[edzer]

Congrats on getting this done, and I'd agree that this is a very hard and painful procedure.

[alexyshr]

Thank you!. Please let me know if you have any additional advice or different procedures for making this easier!