songtianlun/openfoam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
d27457d41d
openfoam/applications/test
History
Andrew Heather d8d6030ab6 INT: Integration of Mattijs' collocated parallel IO additions 
Original commit message:
------------------------

Parallel IO: New collated file format

When an OpenFOAM simulation runs in parallel, the data for decomposed fields and
mesh(es) has historically been stored in multiple files within separate
directories for each processor.  Processor directories are named 'processorN',
where N is the processor number.

This commit introduces an alternative "collated" file format where the data for
each decomposed field (and mesh) is collated into a single file, which is
written and read on the master processor.  The files are stored in a single
directory named 'processors'.

The new format produces significantly fewer files - one per field, instead of N
per field.  For large parallel cases, this avoids the restriction on the number
of open files imposed by the operating system limits.

The file writing can be threaded allowing the simulation to continue running
while the data is being written to file.  NFS (Network File System) is not
needed when using the the collated format and additionally, there is an option
to run without NFS with the original uncollated approach, known as
"masterUncollated".

The controls for the file handling are in the OptimisationSwitches of
etc/controlDict:

OptimisationSwitches
{
    ...

    //- Parallel IO file handler
    //  uncollated (default), collated or masterUncollated
    fileHandler uncollated;

    //- collated: thread buffer size for queued file writes.
    //  If set to 0 or not sufficient for the file size threading is not used.
    //  Default: 2e9
    maxThreadFileBufferSize 2e9;

    //- masterUncollated: non-blocking buffer size.
    //  If the file exceeds this buffer size scheduled transfer is used.
    //  Default: 2e9
    maxMasterFileBufferSize 2e9;
}

When using the collated file handling, memory is allocated for the data in the
thread.  maxThreadFileBufferSize sets the maximum size of memory in bytes that
is allocated.  If the data exceeds this size, the write does not use threading.

When using the masterUncollated file handling, non-blocking MPI communication
requires a sufficiently large memory buffer on the master node.
maxMasterFileBufferSize sets the maximum size in bytes of the buffer.  If the
data exceeds this size, the system uses scheduled communication.

The installation defaults for the fileHandler choice, maxThreadFileBufferSize
and maxMasterFileBufferSize (set in etc/controlDict) can be over-ridden within
the case controlDict file, like other parameters.  Additionally the fileHandler
can be set by:
- the "-fileHandler" command line argument;
- a FOAM_FILEHANDLER environment variable.

A foamFormatConvert utility allows users to convert files between the collated
and uncollated formats, e.g.
    mpirun -np 2 foamFormatConvert -parallel -fileHandler uncollated

An example case demonstrating the file handling methods is provided in:
$FOAM_TUTORIALS/IO/fileHandling

The work was undertaken by Mattijs Janssens, in collaboration with Henry Weller.
2017-07-07 11:39:56 +01:00
..
alloc STYLE: uniform 'Test-' prefix for all applications/test 2010-11-23 16:26:04 +01:00
base64 Add base64 encoding layer (issue #272) 2016-10-17 12:05:29 +02:00
BinSum ENH: Random numbers - updated dependent code from change cachedRandom->Random class 2017-04-28 09:15:52 +01:00
boundBox ENH: additional methods/operators for boundBox (related to #196) 2017-01-25 19:26:50 +01:00
callback applications/test: Updated include and link options 2015-07-16 14:37:19 +01:00
Circulator Rename circulators to be consistent with the standard OpenFOAM class naming convention 2015-08-06 16:54:47 +01:00
codeStream STYLE: missing include in codeStream test dictionary 2016-07-15 09:05:44 +02:00
CompactIOList INT: Integration of Mattijs' collocated parallel IO additions 2017-07-07 11:39:56 +01:00
CompactListList ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
cplusplus1 ENH: improve HashTable iterator access and management 2017-05-04 10:17:18 +02:00
cpuInfo ENH: provide basic cpu-information 2016-06-15 12:38:46 +02:00
cstring ENH: adapter for a list of C++ strings <-> a list of C-style strings 2016-06-16 08:22:53 +02:00
cubicEqn Added robust primitive cubic/quadratic/linear equation solutions. 2017-03-22 15:11:54 +00:00
cyclic applications/test: Updated include and link options 2015-07-16 14:37:19 +01:00
decomposedBlockData INT: Integration of Mattijs' collocated parallel IO additions 2017-07-07 11:39:56 +01:00
delete Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
dictionary STYLE: manage dictionary inputMode directly within entry class 2017-08-03 07:14:17 +02:00
Dictionary ENH: add some standard templates and macros into stdFoam.H 2017-04-29 22:28:16 +02:00
dimensionedType ENH: provide a 'dimensioned' null-like constructor for dimensionedType 2016-12-02 08:48:01 +01:00
Distribution ENH: Random numbers - updated dependent code from change cachedRandom->Random class 2017-04-28 09:15:52 +01:00
DLList STYLE: adjust const access for linked-list iterators 'operator*' 2017-05-12 12:26:28 +02:00
DynamicField Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
dynamicIndexedOctree ENH: relocate triSurface classes into surfMesh library (issue #294) 2017-05-18 10:42:05 +02:00
DynamicList ENH: Support more C++11 initializer lists (issue #261) 2016-10-18 20:08:37 +02:00
edges STYLE: add edge(labelPair) constructor, debug info etc. 2017-05-04 02:42:50 +02:00
ensightFile ENH: relocate triSurface classes into surfMesh library (issue #294) 2017-05-18 10:42:05 +02:00
ensightFoamReader Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
error ENH: added OStringStream reset method (closes #534) 2017-07-17 18:32:42 +02:00
etcFiles ENH: various general improvments to the POSIX functions 2017-03-30 11:45:25 +02:00
extendedStencil Standardized cell, patch and face loop index names 2016-04-25 10:28:32 +01:00
externalFileCoupler ENH: refactor and combine externalFileCoupler (issue #529) 2017-07-17 12:54:02 +02:00
faces ENH: additional method for switching error throwing on/off (issue #552) 2017-07-29 12:25:30 +02:00
Field STYLE: uniform 'Test-' prefix for all applications/test 2010-11-23 16:26:04 +01:00
fieldDependency Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
fieldMapping TUT: various blockMeshDict with convertToMeters instead of scale 2017-08-07 20:57:05 +02:00
fileName ENH: cleanup wordRe interfaces etc. 2017-05-16 23:54:43 +02:00
fileNameClean Rationalized the keyword to specify a file name in a dictionary to 'file' 2017-01-07 09:38:54 +00:00
findCell-octree ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
findSphereFeatureEdges-octree ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
findTimes Formatting: Rationalized the indentation of #include 2015-02-10 20:35:50 +00:00
FixedList MRG: merged develop line back into integration branch 2017-05-18 11:11:12 +01:00
foamVersion Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
Function1 MRG: Integrated Foundation code to commit 7d6845d 2017-03-23 14:33:33 +00:00
fvc applications/test: Updated include and link options 2015-07-16 14:37:19 +01:00
fvc2D GIT: Resolved conflicts arising from merge with develop branch 2016-09-26 10:57:34 +01:00
fvSolutionCombine COMP: Test apps: make compile 2013-01-17 11:06:38 +00:00
GAMGAgglomeration GeometricField: Renamed internalField() -> primitiveField() and dimensionedInternalField() -> internalField() 2016-04-30 21:40:09 +01:00
globalIndex ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
globalMeshData Further standardization of loop index naming: pointI -> pointi, patchI -> patchi 2016-05-18 21:20:42 +01:00
graph Removed trailing blank lines 2017-01-19 20:17:47 +00:00
graphXi Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
Hashing ENH: Enum class as drop-in alternative for NamedEnum 2017-05-29 13:13:53 +02:00
HashingSpeed C++11: Replaced the C NULL with the safer C++11 nullptr 2016-08-05 17:19:38 +01:00
HashPtrTable ENH: HashPtrTable remove/erase now include safeguard against end-iterator 2017-05-15 09:57:25 +02:00
HashSet ENH: added HashTable 'lookup' and 'retain' methods 2017-05-11 12:25:35 +02:00
HashTable ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
HashTable2 HashTable: Added C++11 initializer_list constructor 2016-08-05 22:30:26 +01:00
HashTable3 Update code to use the simpler C++11 template syntax removing spaces between closing ">"s 2016-01-10 22:41:16 +00:00
hexRef8 TUT: various blockMeshDict with convertToMeters instead of scale 2017-08-07 20:57:05 +02:00
IndirectList Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
io ENH: consolidate, cleanup some string methods 2017-08-02 12:33:35 +02:00
IOField INT: Integration of Mattijs' collocated parallel IO additions 2017-07-07 11:39:56 +01:00
IOobjectList ENH: added classes() method to objectRegistry/IOobjectList 2017-05-17 10:43:24 +02:00
ISLList ENH: add some standard templates and macros into stdFoam.H 2017-04-29 22:28:16 +02:00
IStringStream ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
labelRanges ENH: improvements to labelRange const_iterator 2017-05-14 14:39:17 +02:00
liquid thermophysicalProperties: New base-class for liquidProperties and in the future gasProperties 2017-02-18 21:53:20 +00:00
List ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
ListOps ENH: add inplaceUniqueSort function 2017-03-08 11:10:16 +01:00
Map ENH: provide HashTable::iterator::found() method 2017-04-11 09:55:54 +02:00
mapDistributePolyMesh TUT: various blockMeshDict with convertToMeters instead of scale 2017-08-07 20:57:05 +02:00
mappedPatch STYLE: eliminate duplicate includes (issue #293) 2017-07-17 14:44:05 +02:00
Matrix LUscalarMatrix: Added processor-local matrix inverse function 2016-07-17 14:44:50 +01:00
maxMem COMP: Applications/test: clean build 2011-06-08 15:37:11 +01:00
memInfo Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
mesh ENH: optionally eliminate duplicates on hashedWordList construction (issue #375) 2017-01-10 12:42:40 +01:00
mkdir Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
momentOfInertia primitiveShapes: Generalised tetrahedron and triangle cutting. Cuts are 2017-05-22 11:40:37 +01:00
mvBak Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
NamedEnum ENH: Enum class as drop-in alternative for NamedEnum 2017-05-29 13:13:53 +02:00
nullObject ENH: ensure nullObject is large enough for reinterpret 2017-05-01 22:39:36 +02:00
objectRegistry ENH: improve objectRegistry functionality (issue #322) 2016-12-01 13:04:07 +01:00
ODE Matrix: Added (i, j) addressing to allow support for addressing blocks of the matrix 2016-03-20 19:44:29 +00:00
OStringStream ENH: added OStringStream reset method (closes #534) 2017-07-17 18:32:42 +02:00
PackedList Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
PackedList1 Test-PackedList1: Removed chained assigments 2016-02-08 16:30:00 +00:00
PackedList2 STYLE: hash constructors with power-of-two 2017-07-05 10:46:50 +02:00
PackedList3 Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
PackedList4 ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
parallel ENH: add mpiBufferSize optimisationSwitch (issue #517) 2017-07-05 15:52:44 +02:00
parallel-communicators Updated UPstream::commsTypes to use the C++11 enum class 2017-03-10 19:54:55 +00:00
parallel-nonBlocking Updated UPstream::commsTypes to use the C++11 enum class 2017-03-10 19:54:55 +00:00
parallelOverset TUT: various blockMeshDict with convertToMeters instead of scale 2017-08-07 20:57:05 +02:00
passiveParticle STYLE: Consistency updates to change input of <var>Name to <var>. Fixes #306 2016-11-22 14:50:33 +00:00
PatchEdgeFaceWave Standardized cell, patch and face loop index names 2016-04-25 10:28:32 +01:00
patchRegion TUT: various blockMeshDict with convertToMeters instead of scale 2017-08-07 20:57:05 +02:00
PatchTools Further standardization of loop index naming: pointI -> pointi, patchI -> patchi 2016-05-18 21:20:42 +01:00
PointEdgeWave Further standardization of loop index naming: pointI -> pointi, patchI -> patchi 2016-05-18 21:20:42 +01:00
pointField vector::zero -> Zero 2016-04-16 18:34:41 +01:00
Polynomial ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
POSIX Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
predicates ENH: added constant predicates 2017-05-17 10:18:14 +02:00
prefixOSstream ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
primitivePatch Further standardization of loop index naming: pointI -> pointi, patchI -> patchi 2016-05-18 21:20:42 +01:00
pTraits COMP: hash-table size overflow with 64-bit labels (fixes #498) 2017-06-14 12:54:38 +02:00
PtrList ENH: Update tests for PtrList and sort 2013-09-25 10:11:36 +01:00
PtrListDictionary PtrListDictionary: New form of Dictionary in which the list type is PtrList rather than a linked-list 2015-09-16 21:26:26 +01:00
quaternion quaternion/septernion: Added multi- quaternion/septernion averaging 2016-04-16 15:59:05 +01:00
reconstruct applications/test: Updated include and link options 2015-07-16 14:37:19 +01:00
regex GIT: Initial state after latest Foundation merge 2016-09-20 14:49:08 +01:00
rigidBodyDynamics applications/test/rigidBodyDynamics/pendulumAndSpring: Another slightly more complex test 2016-04-17 15:47:03 +01:00
router ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
sha1 ENH: added OStringStream reset method (closes #534) 2017-07-17 18:32:42 +02:00
simpleMatrix Matrix: Added (i, j) addressing to allow support for addressing blocks of the matrix 2016-03-20 19:44:29 +00:00
sizeof ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
slicedField applications/test: Updated include and link options 2015-07-16 14:37:19 +01:00
SLList STYLE: add edge(labelPair) constructor, debug info etc. 2017-05-04 02:42:50 +02:00
sort ENH: Update tests for PtrList and sort 2013-09-25 10:11:36 +01:00
speed Removed use of the deprecated "register" keyword 2015-06-30 10:26:44 +01:00
sphericalTensorField Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
spline STYLE: Updated version to 'plus' 2015-12-22 23:14:17 +00:00
StaticHashTable ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
string ENH: consolidate, cleanup some string methods 2017-08-02 12:33:35 +02:00
stringList ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00
stringSplit ENH: add stringOps::splitAny, stringOps::splitSpace 2017-08-14 10:36:12 +02:00
surfaceIntersection ENH: add input surface scaling (issue #514) 2017-08-14 09:18:15 +02:00
surfaceMeshConvert INT: Integration of Mattijs' collocated parallel IO additions 2017-07-07 11:39:56 +01:00
symmTensorField Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
syncTools Further standardization of loop index naming: pointI -> pointi, patchI -> patchi 2016-05-18 21:20:42 +01:00
sysInfo Cleanup endian support (closes #271) 2016-10-17 12:02:01 +02:00
tensor messageStream: Added DebugInfo and DebugInFunction: 2016-03-04 11:30:13 +00:00
tensor2D tensor2D: Added specialisation for the outerProduct of Vector2D to enable outer product operation 2012-02-01 12:30:16 +00:00
tetTetOverlap primitiveShapes: Generalised tetrahedron and triangle cutting. Cuts are 2017-05-22 11:40:37 +01:00
thermoMixture thermophysicalModels: Changed specie thermodynamics from mole to mass basis 2017-02-17 11:22:14 +00:00
tmpField Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
tokenize ENH: improved behaviour of input stream rewind (issue #534) 2017-07-18 11:10:51 +02:00
triTet STYLE: uniform 'Test-' prefix for all applications/test 2010-11-23 16:26:04 +01:00
Tuple2 ENH: add Tuple2 comparison operators, typedefs 2017-04-30 13:05:49 +02:00
UDictionary Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
UIndirectList Copyright transfered to the OpenFOAM Foundation 2011-08-14 12:17:30 +01:00
UniformField UniformField: New field type 2013-09-27 22:47:59 +01:00
unitConversion ENH: add (C++11) user-literal for degrees to radians conversion 2017-05-31 10:16:19 +02:00
vector STYLE: uniform 'Test-' prefix for all applications/test 2010-11-23 16:26:04 +01:00
vectorTools STYLE: use FOAM_UTILITIES in Make/options 2017-04-07 08:17:43 +02:00
volField TUT: various blockMeshDict with convertToMeters instead of scale 2017-08-07 20:57:05 +02:00
volPointInterpolation boundaryField() -> boundaryFieldRef() 2016-04-24 22:07:37 +01:00
wallDist ENH: additional methods/operators for boundBox (related to #196) 2017-01-25 19:26:50 +01:00
wallDistDyM ENH: overset: Initial release of overset capability. 2017-06-14 09:51:02 +01:00
wordRe ENH: added constant predicates 2017-05-17 10:18:14 +02:00
xfer ENH: make treatment of stream allocators more uniform (issue #532) 2017-07-17 15:14:38 +02:00