Error checking a Rocky Linux installation

[pv-rse]

Hello,
after installing on a Rockey Linux 9.2 distribution with all dependencies, a simple check returned:
[user@frontend ~]$ firedrake-check
Traceback (most recent call last):
File "/home/bicocca/oil_leak/bin/firedrake-check", line 5, in
from firedrake._check import main
File "/home/bicocca/oil_leak/lib/python3.10/site-packages/firedrake/init.py", line 16, in
petsc4py.init(sys.argv)
File "/home/bicocca/oil_leak/lib/python3.10/site-packages/petsc4py/init.py", line 42, in init
PETSc = petsc4py.lib.ImportPETSc(arch)
File "/home/bicocca/oil_leak/lib/python3.10/site-packages/petsc4py/lib/init.py", line 33, in ImportPETSc
return Import('petsc4py', 'PETSc', path, arch)
File "/home/bicocca/oil_leak/lib/python3.10/site-packages/petsc4py/lib/init.py", line 100, in Import
module = import_module(pkg, name, path, arch)
File "/home/bicocca/oil_leak/lib/python3.10/site-packages/petsc4py/lib/init.py", line 77, in import_module
module = importlib.util.module_from_spec(spec)
ImportError: /lib64/libldap.so.2: undefined symbol: EVP_md2, version OPENSSL_3.0.0

we installed it in a python virtual environment. We tried it and returned what you can see in this screenshot:

[connorjward]

I think this sort of error occurs when you have multiple libraries in your system and are loading the wrong one at runtime.

• Does your system have any non-standard networking setup? The error is getting thrown by openssl which I have never seen before.
• What does make check inside $PETSC_DIR show?

You may be able to fix this by either:

• Setting LD_PRELOAD to the correct .so
• Reinstalling and setting LDFLAGS appropriately.

[pv-rse]

Hello, thanks for your wuick answer. We tried to install it on a single node of a HPC cluster. This node is a virtual machine but it has a "normal" network configuration. All the others nodes have infiniband, not this one. It has a single ethernet interface.

Here is make check result:

[root@node101 petsc]# make check
Running PETSc check examples to verify correct installation
Using PETSC_DIR=/home/user/petsc and PETSC_ARCH=arch-firedrake-default
make[3]: [/home/user/petsc/lib/petsc/conf/rules:46: ex19.PETSc] Error 2 (ignored)
*******************Error detected during compile or link!*******************
See https://petsc.org/release/faq/
/home/user/petsc/src/snes/tutorials ex19
*********************************************************************************
mpicc -fPIC -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-stringop-overflow -fstack-protector -fvisibility=hidden -O3 -march=native -mtune=native  -I/home/user/petsc/include -I/home/user/petsc/arch-firedrake-default/include -I/home/user/petsc/arch-firedrake-default/include/suitesparse     -Wl,-export-dynamic ex19.c   -Wl,-rpath,/home/user/petsc/arch-firedrake-default/lib -L/home/user/petsc/arch-firedrake-default/lib -Wl,-rpath,/usr/lib64/openmpi/lib -L/usr/lib64/openmpi/lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/11 -L/usr/lib/gcc/x86_64-redhat-linux/11 -lpetsc -lHYPRE -lspqr -lumfpack -lklu -lcholmod -lamd -lsuperlu_dist -ldmumps -lmumps_common -lpord -lpthread -lscalapack -lfftw3_mpi -lfftw3 -lflexiblas -lptesmumps -lptscotchparmetisv3 -lptscotch -lptscotcherr -lesmumps -lscotch -lscotcherr -lmetis -lnetcdf -lpnetcdf -lhdf5_hl -lhdf5 -lm -lz -lhwloc -lX11 -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lstdc++ -lquadmath  -o ex19
/usr/bin/bash: line 1: mpicc: command not found
make[4]: *** [/home/user/petsc/lib/petsc/conf/rules:217: ex19] Error 127
Possible error running C/C++ src/snes/tutorials/ex19 with 1 MPI process
See https://petsc.org/release/faq/
--------------------------------------------------------------------------
mpiexec has detected an attempt to run as root.

Running as root is *strongly* discouraged as any mistake (e.g., in
defining TMPDIR) or bug can result in catastrophic damage to the OS
file system, leaving your system in an unusable state.

We strongly suggest that you run mpiexec as a non-root user.

You can override this protection by adding the --allow-run-as-root option
to the cmd line or by setting two environment variables in the following way:
the variable OMPI_ALLOW_RUN_AS_ROOT=1 to indicate the desire to override this
protection, and OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 to confirm the choice and
add one more layer of certainty that you want to do so.
We reiterate our advice against doing so - please proceed at your own risk.
--------------------------------------------------------------------------
Possible error running C/C++ src/snes/tutorials/ex19 with 2 MPI processes
See https://petsc.org/release/faq/
--------------------------------------------------------------------------
mpiexec has detected an attempt to run as root.

Running as root is *strongly* discouraged as any mistake (e.g., in
defining TMPDIR) or bug can result in catastrophic damage to the OS
file system, leaving your system in an unusable state.

We strongly suggest that you run mpiexec as a non-root user.

You can override this protection by adding the --allow-run-as-root option
to the cmd line or by setting two environment variables in the following way:
the variable OMPI_ALLOW_RUN_AS_ROOT=1 to indicate the desire to override this
protection, and OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 to confirm the choice and
add one more layer of certainty that you want to do so.
We reiterate our advice against doing so - please proceed at your own risk.
--------------------------------------------------------------------------
1,5c1,16
< lid velocity = 0.0016, prandtl # = 1., grashof # = 1.
<   0 SNES Function norm 0.0406612
<   1 SNES Function norm 4.12227e-06
<   2 SNES Function norm 6.098e-11
< Number of SNES iterations = 2
---
> --------------------------------------------------------------------------
> mpiexec has detected an attempt to run as root.
>
> Running as root is *strongly* discouraged as any mistake (e.g., in
> defining TMPDIR) or bug can result in catastrophic damage to the OS
> file system, leaving your system in an unusable state.
>
> We strongly suggest that you run mpiexec as a non-root user.
>
> You can override this protection by adding the --allow-run-as-root option
> to the cmd line or by setting two environment variables in the following way:
> the variable OMPI_ALLOW_RUN_AS_ROOT=1 to indicate the desire to override this
> protection, and OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 to confirm the choice and
> add one more layer of certainty that you want to do so.
> We reiterate our advice against doing so - please proceed at your own risk.
> --------------------------------------------------------------------------
/home/user/petsc/src/snes/tutorials
Possible problem with ex19 running with HYPRE, diffs above
=========================================
1,9c1,16
< lid velocity = 0.0625, prandtl # = 1., grashof # = 1.
<   0 SNES Function norm 0.239155
<     0 KSP Residual norm 0.235858
<     1 KSP Residual norm < 1.e-11
<   1 SNES Function norm 6.81968e-05
<     0 KSP Residual norm 2.30906e-05
<     1 KSP Residual norm < 1.e-11
<   2 SNES Function norm < 1.e-11
< Number of SNES iterations = 2
---
> --------------------------------------------------------------------------
> mpiexec has detected an attempt to run as root.
>
> Running as root is *strongly* discouraged as any mistake (e.g., in
> defining TMPDIR) or bug can result in catastrophic damage to the OS
> file system, leaving your system in an unusable state.
>
> We strongly suggest that you run mpiexec as a non-root user.
>
> You can override this protection by adding the --allow-run-as-root option
> to the cmd line or by setting two environment variables in the following way:
> the variable OMPI_ALLOW_RUN_AS_ROOT=1 to indicate the desire to override this
> protection, and OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 to confirm the choice and
> add one more layer of certainty that you want to do so.
> We reiterate our advice against doing so - please proceed at your own risk.
> --------------------------------------------------------------------------
/home/user/petsc/src/snes/tutorials
Possible problem with ex19 running with MUMPS, diffs above
=========================================
1,5c1,16
< lid velocity = 0.0016, prandtl # = 1., grashof # = 1.
<   0 SNES Function norm 0.0406612
<   1 SNES Function norm 3.32845e-06
<   2 SNES Function norm < 1.e-11
< Number of SNES iterations = 2
---
> --------------------------------------------------------------------------
> mpiexec has detected an attempt to run as root.
>
> Running as root is *strongly* discouraged as any mistake (e.g., in
> defining TMPDIR) or bug can result in catastrophic damage to the OS
> file system, leaving your system in an unusable state.
>
> We strongly suggest that you run mpiexec as a non-root user.
>
> You can override this protection by adding the --allow-run-as-root option
> to the cmd line or by setting two environment variables in the following way:
> the variable OMPI_ALLOW_RUN_AS_ROOT=1 to indicate the desire to override this
> protection, and OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 to confirm the choice and
> add one more layer of certainty that you want to do so.
> We reiterate our advice against doing so - please proceed at your own risk.
> --------------------------------------------------------------------------
/home/user/petsc/src/snes/tutorials
Possible problem with ex19 running with SuiteSparse, diffs above
=========================================
1,2c1,16
< lid velocity = 0.0025, prandtl # = 1., grashof # = 1.
< Number of SNES iterations = 2
---
> --------------------------------------------------------------------------
> mpiexec has detected an attempt to run as root.
>
> Running as root is *strongly* discouraged as any mistake (e.g., in
> defining TMPDIR) or bug can result in catastrophic damage to the OS
> file system, leaving your system in an unusable state.
>
> We strongly suggest that you run mpiexec as a non-root user.
>
> You can override this protection by adding the --allow-run-as-root option
> to the cmd line or by setting two environment variables in the following way:
> the variable OMPI_ALLOW_RUN_AS_ROOT=1 to indicate the desire to override this
> protection, and OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 to confirm the choice and
> add one more layer of certainty that you want to do so.
> We reiterate our advice against doing so - please proceed at your own risk.
> --------------------------------------------------------------------------
/home/user/petsc/src/snes/tutorials
Possible problem with ex19 running with SuperLU_DIST, diffs above
=========================================
make[3]: [/home/user/petsc/lib/petsc/conf/rules:46: ex47.PETSc] Error 2 (ignored)
*******************Error detected during compile or link!*******************
See https://petsc.org/release/faq/
/home/user/petsc/src/vec/vec/tests ex47
*********************************************************************************
mpicc -fPIC -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-stringop-overflow -fstack-protector -fvisibility=hidden -O3 -march=native -mtune=native  -I/home/user/petsc/include -I/home/user/petsc/arch-firedrake-default/include -I/home/user/petsc/arch-firedrake-default/include/suitesparse     -Wl,-export-dynamic ex47.c   -Wl,-rpath,/home/user/petsc/arch-firedrake-default/lib -L/home/user/petsc/arch-firedrake-default/lib -Wl,-rpath,/usr/lib64/openmpi/lib -L/usr/lib64/openmpi/lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/11 -L/usr/lib/gcc/x86_64-redhat-linux/11 -lpetsc -lHYPRE -lspqr -lumfpack -lklu -lcholmod -lamd -lsuperlu_dist -ldmumps -lmumps_common -lpord -lpthread -lscalapack -lfftw3_mpi -lfftw3 -lflexiblas -lptesmumps -lptscotchparmetisv3 -lptscotch -lptscotcherr -lesmumps -lscotch -lscotcherr -lmetis -lnetcdf -lpnetcdf -lhdf5_hl -lhdf5 -lm -lz -lhwloc -lX11 -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lstdc++ -lquadmath  -o ex47
/usr/bin/bash: line 1: mpicc: command not found
make[4]: *** [/home/user/petsc/lib/petsc/conf/rules:217: ex47] Error 127
0a1,16
> --------------------------------------------------------------------------
> mpiexec has detected an attempt to run as root.
>
> Running as root is *strongly* discouraged as any mistake (e.g., in
> defining TMPDIR) or bug can result in catastrophic damage to the OS
> file system, leaving your system in an unusable state.
>
> We strongly suggest that you run mpiexec as a non-root user.
>
> You can override this protection by adding the --allow-run-as-root option
> to the cmd line or by setting two environment variables in the following way:
> the variable OMPI_ALLOW_RUN_AS_ROOT=1 to indicate the desire to override this
> protection, and OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 to confirm the choice and
> add one more layer of certainty that you want to do so.
> We reiterate our advice against doing so - please proceed at your own risk.
> --------------------------------------------------------------------------
/home/user/petsc/src/vec/vec/tests
Possible problem with ex47 running with HDF5, diffs above
=========================================
Completed PETSc check examples
Error while running make check
make[1]: *** [makefile:125: check] Error 1
make: *** [GNUmakefile:17: check] Error 2

[connorjward]

This looks really close now. I have seen this issue before but don't quite remember how to fix it.

1. What does pip list show you?
2. If you cd to a different directory and run the command again does anything change?

[pv-rse]

Hello, thanks! Now we tried alongside another solution with a physical machine and a 24.04 Ubuntu. Hopefully everything will be straightforward with the installation.
In the other system, here is pip list result:

Package              Version
-------------------- --------------
annotated-types      0.7.0
cachetools           5.5.2
certifi              2025.4.26
cgen                 2020.1
charset-normalizer   3.4.2
checkpoint_schedules 1.0.4
click                8.2.0
codepy               2025.1
colorama             0.4.6
constantdict         2025.1.1
decorator            4.4.2
exceptiongroup       1.3.0
fenics-ufl           2025.1.0
firedrake            2025.4.0.post0
firedrake-fiat       2025.4.0
genpy                2022.1
gmsh                 4.13.1
h5py                 3.13.0
idna                 3.10
immutabledict        4.2.1
iniconfig            2.1.0
islpy                2025.1.5
libsupermesh         2025.3.0
loopy                2025.1
Mako                 1.3.10
mannrs               1.2.0
MarkupSafe           3.0.2
mpi-pytest           2025.4.0
mpi4py               4.0.3
mpmath               1.3.0
numpy                2.2.5
packaging            25.0
petsc                3.23.2
petsc4py             3.23.0
pip                  23.0.1
pkgconfig            1.5.5
platformdirs         4.3.8
pluggy               1.6.0
progress             1.6
pyadjoint-ad         2025.4.0
pycparser            2.22
pydantic             2.11.4
pydantic_core        2.33.2
pyevtk               1.6.0
pymbolic             2024.2.2
pytest               8.3.5
pytools              2025.1.5
recursivenodes       0.2.0
requests             2.32.3
rtree                1.4.0
scipy                1.15.3
setuptools           65.5.0
siphash              0.0.1
siphash24            1.7
six                  1.17.0
symengine            0.14.1
sympy                1.14.0
toml                 0.10.2
tomli                2.2.1
tqdm                 4.67.1
typing_extensions    4.13.2
typing-inspection    0.4.0
UFL                  2017.1.0
urllib3              2.4.0

[notice] A new release of pip is available: 23.0.1 -> 25.1.1
[notice] To update, run: pip install --upgrade pip

Unfortunately, outside the PETSC dir, the command returns the same results:

(environment) [user@node101 ~]$ firedrake-check
Traceback (most recent call last):
  File "/home/user/environment/bin/firedrake-check", line 5, in <module>
    from firedrake._check import main
  File "/home/user/environment/lib/python3.10/site-packages/firedrake/__init__.py", line 20, in <module>
    from firedrake import petsc
  File "/home/user/environment/lib/python3.10/site-packages/firedrake/petsc.py", line 432, in <module>
    _DEFAULT_KSP_PARAMETERS = flatten_parameters({
  File "/home/user/environment/lib/python3.10/site-packages/firedrake/petsc.py", line 55, in flatten_parameters
    from firedrake.logging import warning
  File "/home/user/environment/lib/python3.10/site-packages/firedrake/logging.py", line 5, in <module>
    import tsfc.logging             # noqa: F401
  File "/home/user/environment/lib/python3.10/site-packages/tsfc/__init__.py", line 1, in <module>
    from tsfc.driver import compile_form, compile_expression_dual_evaluation  # noqa: F401
  File "/home/user/environment/lib/python3.10/site-packages/tsfc/driver.py", line 5, in <module>
    from finat.physically_mapped import DirectlyDefinedElement, PhysicallyMappedElement
  File "/home/user/environment/lib/python3.10/site-packages/finat/__init__.py", line 43, in <module>
    from . import element_factory                                      # noqa: F401
  File "/home/user/environment/lib/python3.10/site-packages/finat/element_factory.py", line 26, in <module>
    import finat.ufl
  File "/home/user/environment/lib/python3.10/site-packages/finat/ufl/__init__.py", line 14, in <module>
    from finat.ufl.brokenelement import BrokenElement  # noqa: F401
  File "/home/user/environment/lib/python3.10/site-packages/finat/ufl/brokenelement.py", line 12, in <module>
    from finat.ufl.finiteelementbase import FiniteElementBase
  File "/home/user/environment/lib/python3.10/site-packages/finat/ufl/finiteelementbase.py", line 17, in <module>
    from ufl import pullback
  File "/home/user/environment/lib/python3.10/site-packages/ufl/__init__.py", line 270, in <module>
    from ufl.finiteelement import AbstractFiniteElement
ImportError: cannot import name 'AbstractFiniteElement' from 'ufl.finiteelement' (/home/user/environment/lib/python3.10/site-packages/ufl/finiteelement/__init__.py)

Thanks

[connorjward]

Your pip list output includes a package called UFL. How did that get there? You likely want to uninstall that and perhaps reinstall fenics-ufl.

Check that you can run python3 -c "import ufl" and python3 -c "import firedrake"

[pv-rse]

Thanks! now i reached a good step forward, we have only to link mpiexec to our environment, but i will leave the final user on the new environment we set up (a new physical machine with a regular ubuntu installed).

Now this is the situation:

(environment) [user@node101 ~]$ firedrake-check
firedrake:WARNING OMP_NUM_THREADS is not set or is set to a value greater than 1, we suggest setting OMP_NUM_THREADS=1 to improve performance
make: Entering directory '/home/user/environment/lib/python3.10/site-packages/firedrake/_check'
    Running serial smoke tests
...........................                                                                                      [100%]
27 passed in 34.43s
    Serial tests passed
    Running parallel smoke tests
/bin/sh: line 1: mpiexec: command not found
make: *** [Makefile:108: check] Error 127
make: Leaving directory '/home/user/environment/lib/python3.10/site-packages/firedrake/_check'

[connorjward]

Ah yes this is a known issue that I will eventually get around to fixing. You can test that things work in parallel by either:

1. Creating an alias/shim script for mpiexec that calls srun or whatever it is you need (this is awkward so I would probably not bother)
2. Running (but replacing mpiexec)

$ mpiexec -n 3 python3 -m pytest tests/firedrake/regression/test_stokes_hdiv_parallel.py -v

if the tests pass then things are working in parallel.