Some MPI Functions can be Macros

[besnardjb]

Problem

The ABI is all about being able to retarget MPI binaries to another MPI implementation. It means all MPI libraries should provides the same interface and therefore there should be no "hidden" expectation from the binary, for example through macros coming from mpi.h. In the standard, there are of course constants as handled in #1 but some functions could also be manipulated at compile time through macros as per 2.6.4:

An implementation is allowed to implement MPI_WTIME, PMPI_WTIME, MPI_WTICK, PMPI_WTICK, MPI_AINT_ADD, PMPI_AINT_ADD, MPI_AINT_DIFF, PMPI_AINT_DIFF, and the handle-conversion functions (MPI_Group_f2c, etc.) in Section 19.3.4, and no others, as macros in C.

Hidden Function Call

Therefore, if you consider an MPI_WTIME unfolded as:

#define MPI_Wtime() _mpi_get_timer()

Changing the MPI library would lead to an undefined symbol as it would not be caught as part of the regular interface.

Handle Magics

The C2F interface is also concerned and therefore if there is some bit-manipulation magic in place in the conversion functions, this conversion would not be relevant for another implementation. This second issue is probably going to mitigate itself when working on the C ABI (as there will be less handle freedom).

Proposal

• Force to use standard libc or assembly for Wtime / Wticks or do not make a macro
• ADD & DIFF should solve themselves as part of the ABI typing (and unfolds in portable code)
• The C2F interface should be macro only if the code is portable from the new ABI point of view (should be the case once F77 is out of the way ;-))

Changes to the Text

TBD

Impact on Implementations

• It has to be checked but I recall wrapping Wtime in most implementations
• These are relatively simple functions, changing (if needed) should be acceptable

Impact on Users

• If WTIME is not inlined anymore it can change perf (but it is not already BTW)
• Make sure to be able to retarget a given MPI program without recompiling it (=save 🌲)

References and Pull Requests

TBD

• Look at C2F implementation in main MPI runtimes (approach & current cost)
• Look at the "Macro" status of all these functions in main MPI runtimes & document
• Conclude from previous tasks

[jeffhammond]

The F2C/C2F functions aren't on the critical path of most code. They really only matter if somebody writes a Fortran interface that talks to C without assuming implementation details (as I've been doing).

If we standardize a C ABI and change Fortran MPI_VAL to match, conversions can be macros for F08. Then we can make the others real functions and just tell the F90 users that it isn't 1992 anymore.

Wtime should be a symbol. Nobody should call it so much that it matters. Most implementations call enough stuff that a macro implementation is pointless (Blue Gene is the only exception in recent times and it's dead).

The AINT arithmetic functions were created because Fortran doesn't have unsigned integers. It is a stupid problem and deserves a stupid solution (function call).

Modern links can do inclining now anyways.

[besnardjb]

The F2C/C2F functions aren't on the critical path of most code. They really only matter if somebody writes a Fortran interface that talks to C without assuming implementation details (as I've been doing).

Yes, just for the story, once wrote a toy generator from MPI bindings description, for exactly this too: https://github.com/besnardjb/mpi_meta#cfbindpy

Thanks for all these interesting details that I overlooked on the Fortran side ! Yes if it impacts you -- you probably looked for it. Advice would then be close to: make it a Macro if you can write it in plain C / ASM without function call, this way we do not close to door to optimized C2F (or others).

[jeffhammond]

It looks like we will get symbols for timers and handle conversions in 4.1. The AINT macros might remain, because they're just too stupid to mess with. The primary reason they're there is segmented addressing in C, which isn't a thing anymore. Another reason is specific to dynamic windows in RMA.

[gonzalobg]

My preference is to expose the MPI ABI in <mpi_abi.h> instead of mpi.h and require these to be functions there.

Whether an implementation's mpi_abi.h and mpi.h header are then two different headers, or the same header, is something that each implementation can control.

[jeffhammond]

We should be able to get them into mpi.h in MPI 4.1 since this isn't a serious ABI issue.