coll MCA variables READONLY scope prevents their use via MPI_T tools interface

[burlen]

Several of the coll component's MCA variables are restrictively scoped as MCA_BASE_VAR_SCOPE_READONLY. A variable scoped READONLY can be set on the command line but not with the MPI_T tools interface. A change to MCA_BASE_VAR_SCOPE_ALL would enable these to be set by the MPI_T tools interface.

Resolves #12902

[burlen]

updated copyright, and rebased from main

[gkatev]

Is there any reason not to have a variable registered in a way that allows it to be set from the MPI_T tools interface? Maybe just to 'protect' important variables from being altered from tools? I basically want to make sure that I should also do the same thing for coll/xhc's variables. But also asking out of curiosity, since I think I'm seeing many/most variables in OMPI being registered READONLY rather than READ_ALL with SETTABLE.

[bosilca]

It will depend on how they are read and used.If they are cached by a framework being able to set them might not lead to what you expect. To be blunt, many of the MCA variables shall never be altered after MPI_Init as this might have unexpected outcomes.

Talking specifically about the collective MCA, I somehow had the assumption that changing these MCA variables will only affect future communicators. But it turns out that is incorrect: once changed an MCA variable impacts all collectives, including in existing communicators.

[burlen]

once changed an MCA variable impacts all collectives, including in existing communicators.

It's not quite right for the coll tuned component. In the tuned component, MCA variables are stored in global variables. When a communicator is created the value of the global variable is read and stored in the communicator itself(here). Future calls reference the value cached in the communicator, so changing MCA variable only impacts collectives on future communicators. The settings stick to the communicator.