- additional startup guard for inter-node/local-node queries (UPstream)
- impose linear communication tree for inter-node/local-node
communicators. Was previously defaulted to a basic tree, but more
consistent to have flat addressing for these types of connections.
- demand-driven UPstream::interNode_offsets() for walking
inter-node ranges instead of creating it manually in various places.
- (style): List<int> instead of labelList for internal commsStruct
since the communication structures are tied to MPI sizes
and not to the OpenFOAM label sizes
- reduce the number of intermediate buffer allocations within
gatherList, scatterList.
- split/duplicate functionality
- rework inter-node/intra-node handling to allow selection of
splitting based on 'shared' or hostname (default).
- always creates node communicators at startup:
* commInterNode() - between nodes
* commLocalNode() - within a node
- world-comm is now always a duplicate of MPI_COMM_WORLD to provide
better separation from other processes.
NB:
the inter-node comm is a slight exception to other communicators
in that we always retain its list of (global) ranks, even if the
local process is not in that communicator.
This can help when constructing topology-aware patterns.
FIX: non-participating ranks still had knowledge of their potential siblings
- after create by group, the procIDs_ of non-participating ranks
should be empty (except for the inter-node exception)
