Problem building AMUSE from source

[Spijkerberg]

Hello,

I am trying to build AMUSE from source on a sterrewacht computer.
I am running into issues with certain packages not being built.

Here is the buildlog of the building process.

Thanks in advance,
Menno

[LourensVeen]

Could you paste the output of module list for the shell in which you're building?

[LourensVeen]

Looks like Steven is onto something:

make: Entering directory '/data2/vandereerden/amuse/src/amuse/community/capreole'
make -C build_mpi amuse_interface_mpi  VPATH=../src F90FLAGS1="-g -O2 -DNOMPI -fPIC  " FC="/usr/bin/gfortran -fallow-argument-mismatch" MPIFC="/usr/bin/gfortran -fallow-argument-mismatch"
make[1]: Entering directory '/data2/vandereerden/amuse/src/amuse/community/capreole/build_mpi'
/usr/bin/gfortran -fallow-argument-mismatch -c -g -O2 -DNOMPI -fPIC   -DMPI  -I/data2/vandereerden/amuse/lib/stopcond ../src/amuse_mpi.F90
../src/amuse_mpi.F90:54:0:

   54 |   include 'mpif.h'
      | 
Fatal Error: Cannot open included file ‘mpif.h’
compilation terminated.

That's definitely not right. The question is why this is happening. I should be getting access to these machines, so then I can experiment a little.

[rieder]

It seems to use gfortran, I would expect mpifortran needs to be used here instead.
The argument MPIFC="/usr/bin/gfortran -fallow-argument-mismatch" is probably causing this.

[rieder]

MPIFC should not be set to gfortran, keep it at mpifortran or undefined.
You can set FC to gfortran -fallow-argument-mismatch if this is important, but redefining an MPI compiler to a non-MPI one is probably what's causing the issue.

[rieder]

... thinking about it further, I guess this may follow from AMUSE being configured without MPI in the first place. So a first step would be to re-run configure in the AMUSE dir, and checking config.mk for the definition of MPIFC there.

[Spijkerberg]

The output of `module load` is: Currently Loaded Modules: 1) localhosts 2) GCCcore/12.3.0 3) zlib/1.2.13-GCCcore-12.3.0 4) binutils/2.40-GCCcore-12.3.0 5) GCC/12.3.0 6) numactl/2.0.16-GCCcore-12.3.0 7) XZ/5.4.2-GCCcore-12.3.0 8) libxml2/2.11.4-GCCcore-12.3.0 9) libpciaccess/0.17-GCCcore-12.3.0 10) hwloc/2.9.1-GCCcore-12.3.0 11) OpenSSL/1.1 12) libevent/2.1.12-GCCcore-12.3.0 13) UCX/1.14.1-GCCcore-12.3.0 14) libfabric/1.18.0-GCCcore-12.3.0 15) PMIx/4.2.4-GCCcore-12.3.0 16) UCC/1.2.0-GCCcore-12.3.0 17) OpenMPI/4.1.5-GCC-12.3.0 This is the result of loading the modules 'AMUSE' (AMUSE/2023.5.1) and 'OpenMPI'. In that order.

[rieder]

I'd like to see the config.mk file (in the AMUSE root dir) of the sterrewacht installation of AMUSE, can you find this @Spijkerberg?

[Spijkerberg]

I found the file in the AMUSE root dir on the sterrewacht machine. I turned it into a txt file to share it.

[rieder]

Thanks.
This shows AMUSE was configured without MPI support, so no wonder stuff that requires MPI breaks...
This will require fixing on the module level.

[Spijkerberg]

Looking at the config.mk that was produced from my own installation from source, I see that MPI_ENABLED=no, while I did load the MPI module before installing. I could try to install AMUSE from source again, but I do not know if that would help.

[rieder]

You will also need to install mpi4py (`pip install mpi4py`). If this is not detected, configure will set MPI_ENABLED to no.

[Spijkerberg]

Looking at the packages enabled in my environment I see that mpi4py is already installed.
Here is the output of pip freeze:
-e git+https://github.com/amusecode/amuse.git@72c4a3c32c21e48f3a823af9f742c7de2684138b#egg=amuse_devel
docutils==0.20.1
h5py==3.10.0
iniconfig==2.0.0
mpi4py==3.1.5
numpy==1.26.2
packaging==23.2
pluggy==1.3.0
pytest==7.4.3
setuptools-scm==8.0.4
typing_extensions==4.8.0

[rieder]

If you re-run configure, does that change the config.mk file?

[Spijkerberg]

This did change the config.mk file. I can see that MPI_ENABLED=yes is set correctly now.
I will try rebuilding AMUSE to see what the result is.

[Spijkerberg]

It seems that some more of the community codes have been built, but there are still some errors when building.
I have provided the buildlogs again for you to inspect.

Testing to see if the community codes work results in UNPACK-OPAL-VALUE: UNSUPPORTED TYPE 33 FOR KEY .

[LourensVeen]

OPAL is OpenMPI's utility library. This error sounds like some kind of data format mismatch, which suggests there are different versions of MPI in use. Possibly mpi4py got compiled against a different version of MPI than AMUSE? Or the version you have active when running your script doesn't match the one that was loaded when mpi4py was installed and/or when AMUSE was built?

[LourensVeen]

I had a go at building AMUSE on a Sterrenwacht machine. Progress so far:

(amuse-env) <user>@<host>:~/amuse$ python setup.py develop_build
Illegal instruction (core dumped)

Looks like some kind of numpy build issue. Time to start digging...

[rieder]

If you do module load AMUSE you should now get the correct prerequisites, there was an issue with mpi4py installing the wrong mpi...

[rieder]

OPAL is OpenMPI's utility library. This error sounds like some kind of data format mismatch, which suggests there are different versions of MPI in use. Possibly mpi4py got compiled against a different version of MPI than AMUSE? Or the version you have active when running your script doesn't match the one that was loaded when mpi4py was installed and/or when AMUSE was built?

this was exactly the issue. mpi4py was installed in an incorrect way, built against the wrong (conda) openmpi library - which then clashed with the correct one. This also caused the wrong configuration of the AMUSE module.

[LourensVeen]

What a mess. Actually, when I try to module load AMUSE I get this:

Lmod has detected the following error:  Unable to load module because of error when evaluating modulefile:
    /easybuild/easybuild/el8_8/modules/all/AMUSE/2023.10.0.lua: Empty or non-existent file
    Please check the modulefile and especially if there is a line number specified in the above message  
While processing the following module(s):
   Module fullname  Module Filename
   ---------------  ---------------
   AMUSE/2023.10.0  /easybuild/easybuild/el8_8/modules/all/AMUSE/2023.10.0.lua

That lua script exists, but has permissions 600, so it can't read it...

Seems like I should put a working EasyBuild configuration for AMUSE on my to-do list, after Conda packages and a new build system.

[rieder]

I think the script may be getting updated at the moment, which could account for that weirdness. But yes, an “official” easybuild module might be a good idea.