- split/duplicate functionality
- rework inter-node/intra-node handling to allow selection of
splitting based on 'shared' or hostname (default).
- always creates node communicators at startup:
* commInterNode() - between nodes
* commLocalNode() - within a node
- world-comm is now always a duplicate of MPI_COMM_WORLD to provide
better separation from other processes.
NB:
the inter-node comm is a slight exception to other communicators
in that we always retain its list of (global) ranks, even if the
local process is not in that communicator.
This can help when constructing topology-aware patterns.
FIX: non-participating ranks still had knowledge of their potential siblings
- after create by group, the procIDs_ of non-participating ranks
should be empty (except for the inter-node exception)
- previously had an additional stack for freedRequests_,
which were used to 'remember' locations into the list of
outstandingRequests_ that were handled by 'waitRequest()'.
This was principally done for sanity checks on shutdown,
but we now just test for any outstanding requests that
are *not* MPI_REQUEST_NULL instead (much simpler).
The framework with freedRequests_ also had a provision to 'recycle'
them by popping from that stack, but this is rather fragile since it
would only triggered by some collectives
(MPI_Iallreduce, MPI_Ialltoall, MPI_Igather, MPI_Iscatter)
with no guarantee that these will all be properly removed again.
There was also no pruning of extraneous indices.
ENH: consolidate internal reset/push of requests
- replace duplicate code with inline functions
reset_request(), push_request()
ENH: null out trailing requests
- extra safety (paranoia) for the UPstream::Request versions
of finishedRequests(), waitAnyRequest()
CONFIG: document nPollProcInterfaces in etc/controlDict
- still experimental, but at least make the keyword known
- mechanism has been unused for at least a decade or more
(or was never used). Message tags are assigned on an ad hoc basis
locally when collision avoidance is necessary.
- permits distinction between communicators/groups that were
user-created (eg, MPI_Comm_create) versus those queried from MPI.
Previously simply relied on non-null values, but that is too fragile
ENH: support List<Request> version of UPstream::finishedRequests
- allows more independent algorithms
ENH: added UPstream::probeMessage(...). Blocking or non-blocking
- now simply a no-op for out-of-range values (instead of an error),
which simplifies the calling code.
Previously
==========
if (request_ >= 0 && request_ < UPstream::nRequests())
{
UPstream::waitRequest(request_);
}
Updated
=======
UPstream::waitRequest(request_);
- when 'recycling' freed request indices, ensure they are actually
within the currently addressable range
- MPI finalization now checks outstanding requests against
MPI_REQUEST_NULL to verify that they have been waited or tested on.
Previously simply checked against freed request indices
ENH: consistent initialisation of send/receive bookkeeping
