- UPstream::Communicator is similar to UPstream::Request to
wrap/unwrap MPI_Comm. Provides a 'lookup' method to transcribe
the internal OpenFOAM communicator tracking to the opaque wrapped
version.
- provide an 'openfoam_mpi.H' interfacing file, which includes
the <mpi.h> as well as casting routines.
Example (caution: ugly!)
MPI_Comm myComm =
PstreamUtils::Cast::to_mpi
(
UPstream::Communicator::lookup(UPstream::worldComm)
);
- MPI_THREAD_MULTIPLE is usually undesirable for performance reasons,
but in some cases may be necessary if a linked library expects it.
Provide a '-mpi-threads' option to explicitly request it.
ENH: consolidate some looping logic within argList
- not currently used, but it is possible that communicator allocation
modifies the list of sub-ranks. Ensure that the correct size is used
when (re)initialising the linear/tree structures.
STYLE: adjust MPI test applications
- remove some clutter and unneeded grouping.
Some ideas for host-only communicators
- UPstream::globalComm constant always refers to MPI_COMM_WORLD but
UPstream::worldComm could be MPI_COMM_WORLD (single world)
or a dedicated local communicator (for multi-world).
- provide a Pstream wrapped version of MPI_COMM_SELF,
references as UPstream::selfComm
- UPstream::isUserComm(label)
test for additional user-defined communicators
- prefix FOAM_MPI and library directories with 'sys-' for system
versions for uniform identication.
WM_MPLIB | libdir (FOAM_MPI) | old naming |
SYSTEMMPI | sys-mpi | mpi |
SYSTEMOPENMPI | sys-openmpi | openmpi-system |
- prefix preferences with 'prefs.' to make them more easily
identifiable, and update bin/tools/create-mpi-config accordingly
Old name: config.{csh,sh}/openmpi
New name: config.{csh,sh}/prefs.openmpi
- additional mpi preferences now available:
* prefs.intelmpi
* prefs.mpich
...
CONFIG: added hook for EASYBUILDMPI (eb-mpi), somewhat like USERMPI
- EasyBuild uses mpicc when compiling, so no explicit wmake rules are
used
ENH: support different major versions for system openmpi
- for example, with
WM_MPLIB=SYSTEMOPENMPI2
defines FOAM_MPI=sys-openmpi2 and thus creates lib/sys-openmpi2
ENH: centralize handling of mpi as 'mpi-rules'
Before:
sinclude $(GENERAL_RULES)/mplib$(WM_MPLIB)
sinclude $(DEFAULT_RULES)/mplib$(WM_MPLIB)
ifeq (,$(FOAM_MPI_LIBBIN))
FOAM_MPI_LIBBIN := $(FOAM_LIBBIN)/$(FOAM_MPI)
endif
After:
include $(GENERAL_RULES)/mpi-rules
- also allows variants such as SYSTEMOPENMPI2 to be handled separately
