Commit Graph

31 Commits

Author SHA1 Message Date
mattijs
c880efb4a6 ENH: improve fileHandler support for redistributePar
- use file handlers to manage where meshes/fields are to be read and
  written.
2023-11-21 11:14:28 +00:00
Mark Olesen
d5cdc60a54 BUG: processorMeshes removeFiles does not remove collated (fixes #2607)
ENH: extend rmDir to handle removal of empty directories only

- recursively remove directories that only contain other directories
  but no other contents. Treats dead links as non-content.
2022-10-11 17:58:22 +02:00
Mark Olesen
cc1e6c12bb ENH: redistributePar support for finiteArea (#2436) 2022-05-25 13:12:38 +00:00
Mark Olesen
68b692fdfc ENH: combine loadOrCreateMesh from redistributePar into fvMeshTools.
- similar functionality as newMesh etc.
  Relocated to finiteVolume since there are no dynamicMesh dependencies.

- use simpler procAddressing (with updated mapDistributeBase).
  separated from redistributePar
2022-05-17 17:35:51 +02:00
Mark Olesen
f3674eee36 ENH: simpler use of broadcast (via Pstream::scatter)
- avoids worrying about forgetting a (Pstream::parRun()) guard
  and reduces code. The Pstream::scatter does the same thing under the
  hood.
2022-03-04 17:49:23 +00:00
Mark Olesen
1348cd7e7b ENH: use broadcasting Pstreams for one-to-all sends 2022-03-04 17:49:23 +00:00
Mark Olesen
62ec2f2ddf COMP: deprecate domainName and full hostName (#2280)
- unused in regular OpenFOAM code
- POSIX version uses deprecated gethostbyname()
- Windows version never worked

COMP: localize, noexcept on internal OSspecific methods

STYLE: support fileName::Type SYMLINK and LINK as synonyms
2022-02-10 16:46:12 +01:00
mattijs
b157614e00 ENH: redistributePar: write collated format. #806
- supports redistributePar -decompose -fileHandler collated
- supports redistributePar -reconstruct
- does not support redistributePar with collated in redistribution mode
2021-10-19 12:18:07 +01:00
Mark Olesen
9a2a22a03a ENH: provide setter methods for IOobject read/write options etc.
- simplifies local toggling.

- centralize fileModification static variables into IOobject.
  They were previously scattered between IOobject and regIOobject
2021-03-17 15:10:00 +01:00
Mattijs Janssens
46dbfabd9d ENH: primitiveMesh: make geometry calculation runtime selectable
This adds a 'geometry' scheme section to the system/fvSchemes:

geometry
{
    type            highAspectRatio;
}

These 'fvGeometryMethod's are used to calculate
- deltaCoeffs
- nonOrthoCoeffs
etc and can even modify the basic face/cellCentres calculation.
2020-12-11 10:31:34 +00:00
Mark Olesen
9c7aa9727a ENH: add UPstream::parRun(const bool)
- slightly neater code when disabling/restoring parallel state

  New
      const bool oldParRun = Pstream::parRun(false);
      ...
      Pstream::parRun(oldParRun);

  Old
      const bool oldParRun = Pstream::parRun();
      Pstream::parRun() = false;
      ...
      Pstream::parRun() = oldParRun;
2020-10-12 12:51:22 +02:00
Mark Olesen
5dc5ea928a ENH: add UPstream::subProcs() static method
- returns a range of `int` values that can be iterated across.
  For example,

      for (const int proci : Pstream::subProcs()) { ... }

  instead of

      for
      (
          int proci = Pstream::firstSlave();
          proci <= Pstream::lastSlave();
          ++proci
      )
      {
          ...
      }
2020-09-28 14:26:07 +02:00
OpenFOAM bot
e9219558d7 GIT: Header file updates 2019-10-31 14:48:44 +00:00
OpenFOAM bot
154029ddd0 BOT: Cleaned up header files 2019-02-06 12:28:23 +00:00
mattijs
a6a46d23b8 BUG: polyMesh: set AUTO_WRITE flag. See #1147. 2019-01-07 17:51:47 +00:00
Mark Olesen
02ad76df4f ENH: allow wordHashSet filter for IOobjectList::names
- simplifies usage.
  Support syncPar check on names() to detect inconsistencies.

- simplify readFields, ReadFields and other routines by using these
  new methods.
2018-07-26 14:56:52 +02:00
Mark Olesen
d58c142404 ENH: use restricted dictionary lookup for utilities (issue #762)
- get<label>, get<scalar> instead of readLabel, readScalar, etc.
2018-07-24 08:08:30 +02:00
Mark Olesen
3d608bf06a ENH: remove reliance on the Xfer class (issue #639)
This class is largely a pre-C++11 holdover. It is now possible to
simply use move construct/assignment directly.

In a few rare cases (eg, polyMesh::resetPrimitives) it has been
replaced by an autoPtr.
2018-03-05 13:28:53 +01:00
Mark Olesen
a59c87d5ac ENH: improved zone constructors
- constructor for empty cell/face/point Zones, with contents to be
  transferred in later.

- ZoneMesh::operator(const word&) to return existing zone or a new empty one.
2017-11-10 02:09:37 +01:00
mattijs
6fd6610b57 ENH: redistributePar: create dummy sets on non-existing processors 2017-10-26 09:10:21 +01:00
Andrew Heather
d8d6030ab6 INT: Integration of Mattijs' collocated parallel IO additions
Original commit message:
------------------------

Parallel IO: New collated file format

When an OpenFOAM simulation runs in parallel, the data for decomposed fields and
mesh(es) has historically been stored in multiple files within separate
directories for each processor.  Processor directories are named 'processorN',
where N is the processor number.

This commit introduces an alternative "collated" file format where the data for
each decomposed field (and mesh) is collated into a single file, which is
written and read on the master processor.  The files are stored in a single
directory named 'processors'.

The new format produces significantly fewer files - one per field, instead of N
per field.  For large parallel cases, this avoids the restriction on the number
of open files imposed by the operating system limits.

The file writing can be threaded allowing the simulation to continue running
while the data is being written to file.  NFS (Network File System) is not
needed when using the the collated format and additionally, there is an option
to run without NFS with the original uncollated approach, known as
"masterUncollated".

The controls for the file handling are in the OptimisationSwitches of
etc/controlDict:

OptimisationSwitches
{
    ...

    //- Parallel IO file handler
    //  uncollated (default), collated or masterUncollated
    fileHandler uncollated;

    //- collated: thread buffer size for queued file writes.
    //  If set to 0 or not sufficient for the file size threading is not used.
    //  Default: 2e9
    maxThreadFileBufferSize 2e9;

    //- masterUncollated: non-blocking buffer size.
    //  If the file exceeds this buffer size scheduled transfer is used.
    //  Default: 2e9
    maxMasterFileBufferSize 2e9;
}

When using the collated file handling, memory is allocated for the data in the
thread.  maxThreadFileBufferSize sets the maximum size of memory in bytes that
is allocated.  If the data exceeds this size, the write does not use threading.

When using the masterUncollated file handling, non-blocking MPI communication
requires a sufficiently large memory buffer on the master node.
maxMasterFileBufferSize sets the maximum size in bytes of the buffer.  If the
data exceeds this size, the system uses scheduled communication.

The installation defaults for the fileHandler choice, maxThreadFileBufferSize
and maxMasterFileBufferSize (set in etc/controlDict) can be over-ridden within
the case controlDict file, like other parameters.  Additionally the fileHandler
can be set by:
- the "-fileHandler" command line argument;
- a FOAM_FILEHANDLER environment variable.

A foamFormatConvert utility allows users to convert files between the collated
and uncollated formats, e.g.
    mpirun -np 2 foamFormatConvert -parallel -fileHandler uncollated

An example case demonstrating the file handling methods is provided in:
$FOAM_TUTORIALS/IO/fileHandling

The work was undertaken by Mattijs Janssens, in collaboration with Henry Weller.
2017-07-07 11:39:56 +01:00
Andrew Heather
45381b1085 MRG: Integrated Foundation code to commit 19e602b 2017-03-28 11:30:10 +01:00
Henry Weller
96ad725a0b Updated UPstream::commsTypes to use the C++11 enum class 2017-03-10 19:54:55 +00:00
Andrew Heather
9fbd612672 GIT: Initial state after latest Foundation merge 2016-09-20 14:49:08 +01:00
Henry Weller
2d5ff31649 boundaryField() -> boundaryFieldRef() 2016-04-24 22:07:37 +01:00
Andrew Heather
3f55f752fc GIT: Resolve conflict with upstream merge from Foundation 2015-12-07 17:07:20 +00:00
mattijs
916dcb8685 ENH: parallel: overhaul of parallel mapping
- redistributePar to have almost (complete) functionality of decomposePar+reconstructPar
- low-level distributed Field mapping
- support for mapping surfaceFields (including flipping faces)
- support for decomposing/reconstructing refinement data
2015-11-17 15:05:05 +00:00
Henry Weller
e2ef006b91 applications: Update ...ErrorIn -> ...ErrorInFunction
Avoids the clutter and maintenance effort associated with providing the
function signature string.
2015-11-10 17:53:31 +00:00
mattijs
8e76c66558 ENH: redistributePar: start with all patches present 2014-03-20 16:46:38 +00:00
mattijs
4b5b9b809f BUG: loadOrCreateMesh: remove polyMesh directory 2012-05-24 11:45:44 +01:00
mattijs
50bddce02e STYLE: redistributeParMesh: split off mesh loading 2012-03-30 09:21:57 +01:00