Use MPI_Allreduce for error code checking

[wkliao]

This is to fix #520

[jayeshkrishna]

@dqwu : Let us discuss this change before merging it into master (performance impact, should this be tied to the user error handler etc)

[wkliao]

Actually, when PnetCDF is used, all the MPI communications, bcast or allreduce.
used for inq APIs' error checking are not necessary, because PnetCDF
guarantees the metadata consistency among all MPI processes.

I can make more changes to this PR.

[wkliao]

I think using MPI_Bcast for inquired arguments is necessary when
the number of I/O tasks is less than the total number of MPI tasks.

However, when all MPI tasks are I/O tasks, those MPI_Bcast calls used
for error codes and inquiring arguments are not necessary for PnetCDF.
The cost of MPI_Bcast can be expensive when the number of variables
to be "inquired" is large.

When other I/O methods is used, I think Allreduce is the right way
to check the error codes, especially for those libraries that do not
guarantee the metadata consistency.

[jayeshkrishna]

@wkliao : Is it possible to get meta-data consistency with PnetCDF across all MPI processes while using a subset of these processes for all read/write operations (meta-data/file_header is available for all processes in MPI_COMM_WORLD, or a parent mpi comm, for ex while you use a subset MPI Comm for read/write)?

[wkliao]

The metadata consistency applies to the MPI processes in the
MPI communicator used in the file create/open call. File create/open
call returns a file handler, ncid, which is used by a process
to do I/O and retrieve metadata.

If Scorpio created a sub-communicator and used it to call file create/open,
then those MPI processes in MPI_COMM_WORLD but not in the
sub-communicator will not obtain an ncid and therefore not be able to
retrieve the metadata.