- the very old 'writer' class was fully stateless and always templated
on an particular output type.
This is now replaced with a 'coordSetWriter' with similar concepts
as previously introduced for surface writers (#1206).
- writers change from being a generic state-less set of routines to
more properly conforming to the normal notion of a writer.
- Parallel data is done *outside* of the writers, since they are used
in a wide variety of contexts and the caller is currently still in
a better position for deciding how to combine parallel data.
ENH: update sampleSets to sample on per-field basis (#2347)
- sample/write a field in a single step.
- support for 'sampleOnExecute' to obtain values at execution
intervals without writing.
- support 'sets' input as a dictionary entry (as well as a list),
which is similar to the changes for sampled-surface and permits use
of changeDictionary to modify content.
- globalIndex for gather to reduce parallel communication, less code
- qualify the sampleSet results (properties) with the name of the set.
The sample results were previously without a qualifier, which meant
that only the last property value was actually saved (previous ones
overwritten).
For example,
```
sample1
{
scalar
{
average(line,T) 349.96521;
min(line,T) 349.9544281;
max(line,T) 350;
average(cells,T) 349.9854619;
min(cells,T) 349.6589286;
max(cells,T) 350.4967271;
average(line,epsilon) 0.04947733869;
min(line,epsilon) 0.04449639927;
max(line,epsilon) 0.06452856475;
}
label
{
size(line,T) 79;
size(cells,T) 1720;
size(line,epsilon) 79;
}
}
```
ENH: update particleTracks application
- use globalIndex to manage original parcel addressing and
for gathering. Simplify code by introducing a helper class,
storing intermediate fields in hash tables instead of
separate lists.
ADDITIONAL NOTES:
- the regionSizeDistribution largely retains separate writers since
the utility of placing sum/dev/count for all fields into a single file
is questionable.
- the streamline writing remains a "soft" upgrade, which means that
scalar and vector fields are still collected a priori and not
on-the-fly. This is due to how the streamline infrastructure is
currently handled (should be upgraded in the future).
- if the system/decomposeParDict is missing, skip check for matching
number of processor dirs. This can make job dispatch easier.
Does not apply if -decomposeParDict was explicitly specified.
STYLE: adjust naming of host/slaves in argList
- previously always called dlclose on opened libraries when destroying
the dlLibraryTable. However, by force closing the libraries the
situation can arise that the library is missing its own code that it
needs on unload (#1524). This is also sometimes evident when closing
VTK libraries for runTimePostProcessing (#354, #1585).
- The new default is to not forcibly dlclose any libraries, unless
the dlcloseOnTerminate OptimisationSwitch specifies otherwise.
- The dlLibraryTable::close() method can be used to explicitly close
all libraries and clear the list.
- The dlLibraryTable::clear() method now only clears the entries,
without a dlclose.
- timeVaryingUniformFixedValue -> uniformFixedValue
- allows a variety of functions (eg, coded, expressions, tables, ...)
- more similarity to finiteVolume patch type
STYLE: remove unused timeVarying... from etc/controlDict
- remove (unused) Istream constructors, prune some unused methods,
rationalize write() vs writeDict().
Deprecate inconsistent construction order.
- handle empty names for ".ftr" surface patches (for plain triSurface
format) with double-quoted strings for more reliable streaming.
Written on a single line.
This is _backward_ compatible, but if users have been parsing these
files manually, they will need to adjust their code.
Previously:
```
(
frt-fairing:001%1
empty
windshield:002%2
empty
...
)
```
Updated (with example handling of empty name):
```
(
frt-fairing:001%1 empty
windshield:002%2 ""
...
)
```
- now use debug 2 for scanner and debug 4 for parser.
Provided better feedback about what is being parsed (debug mode)
- relocate debug application to applications/tools/foamExprParserInfo
- this cannot be left as a configurable value (on windows), since it
needs to be enabled even prior to reading the etc/controlDict file,
in case the OpenFOAM installation path itself contains spaces.
- can be used to check the validity of input values.
Example:
dict.getCheck<label>("nIters", greaterOp1<label>(0));
dict.getCheck<scalar>("relax", scalarMinMax::zero_one());
- use 'get' prefix for more regular dictionary methods.
Eg, getOrDefault() as alternative to lookupOrDefault()
- additional ops for convenient construction of predicates
ENH: make dictionary writeOptionalEntries integer
- allow triggering of Fatal if default values are used
ENH: additional scalarRange static methods: ge0, gt0, zero_one
- use GREAT instead of VGREAT for internal placeholders
- additional MinMax static methods: gt, le
- having whitespace in fileName can be somewhat fragile since it means
that the fileName components do not necessarily correspond to a
'Foam::word'. But in many cases it will work provided that spaces
are not present in the final portion of the simulation directory
itself.
InfoSwitches
{
// Allow space character in fileName (use with caution)
allowSpaceInFileName 0;
}
- now use doClean=true as default for fileName::validate(). Was false.
Unlike fileName::clean() this requires no internal string rewrite
since the characters are being copied. Also handle any path
separator transformations (ie, backslash => forward slash) at the
same time. This makes it resemble the std::filesystem a bit more.
- Extended runTimePostProcessing to include access to "live"
simulation objects such a geometry patches and sampled surfaces
stored on the "functionObjectObjects" registry.
- Add 'live' runTimePostProcessing of cloud data.
Extracts position and fields from the cloud via its objectRegistry writer
- For the "live" simulation objects, there are two new volume filters
that work directly with the OpenFOAM volume fields:
* iso-surface
* cutting planes
Both use the VTK algorithms directly and support multiple values.
Eg, can make multiple iso-levels or multiple planes parallel to each
other.
- When VTK has been compiled with MPI-support, parallel rendering will
be used.
- Additional title text properties (shadow, italic etc)
- Simplified handling of scalar-bar and visibility switches
- Support multiple text positions. Eg, for adding watermark text.
- changed the sectorCoeffs keyword to 'point' from 'axisPt'
for more similarity with other dictionaries.
Continue to accept 'axisPt' for compatibility.
- this helps for many cases outlined in issue #1007, but can also be
useful when simply using symlinks for shorter or reorganized
directory structures.
- with the 'cwd' optimization switch it is possible to select the
preferred behaviour for the cwd() function.
A value of 0 causes cwd() to return the physical directory,
which is what getcwd() and `pwd -P` return.
Until now, this was always the standard behaviour.
With a value of 1, cwd() instead returns the logical directory,
which what $PWD contains and `pwd -L` returns.
If any of the sanity checks fail (eg, PWD points to something other
than ".", etc), a warning is emitted and the physical cwd() is
returned instead.
Apart from the optical difference in the output, this additional
control helps workaround file systems with whitespace or other
characters in the directory that normally cause OpenFOAM to balk.
Using a cleaner symlink elsewhere should skirt this issue.
Eg,
cd $HOME
ln -s "/mounted volume/user/workdir" workdir
cd workdir
# start working with OpenFOAM
Previously the coordinate system functionality was split between
coordinateSystem and coordinateRotation. The coordinateRotation stored
the rotation tensor and handled all tensor transformations.
The functionality has now been revised and consolidated into the
coordinateSystem classes. The sole purpose of coordinateRotation
is now just to provide a selectable mechanism of how to define the
rotation tensor (eg, axis-angle, euler angles, local axes) for user
input, but after providing the appropriate rotation tensor it has
no further influence on the transformations.
--
The coordinateSystem class now contains an origin and a base rotation
tensor directly and various transformation methods.
- The origin represents the "shift" for a local coordinate system.
- The base rotation tensor represents the "tilt" or orientation
of the local coordinate system in general (eg, for mapping
positions), but may require position-dependent tensors when
transforming vectors and tensors.
For some coordinate systems (currently the cylindrical coordinate system),
the rotation tensor required for rotating a vector or tensor is
position-dependent.
The new coordinateSystem and its derivates (cartesian, cylindrical,
indirect) now provide a uniform() method to define if the rotation
tensor is position dependent/independent.
The coordinateSystem transform and invTransform methods are now
available in two-parameter forms for obtaining position-dependent
rotation tensors. Eg,
... = cs.transform(globalPt, someVector);
In some cases it can be useful to use query uniform() to avoid
storage of redundant values.
if (cs.uniform())
{
vector xx = cs.transform(someVector);
}
else
{
List<vector> xx = cs.transform(manyPoints, someVector);
}
Support transform/invTransform for common data types:
(scalar, vector, sphericalTensor, symmTensor, tensor).
====================
Breaking Changes
====================
- These changes to coordinate systems and rotations may represent
a breaking change for existing user coding.
- Relocating the rotation tensor into coordinateSystem itself means
that the coordinate system 'R()' method now returns the rotation
directly instead of the coordinateRotation. The method name 'R()'
was chosen for consistency with other low-level entities (eg,
quaternion).
The following changes will be needed in coding:
Old: tensor rot = cs.R().R();
New: tensor rot = cs.R();
Old: cs.R().transform(...);
New: cs.transform(...);
Accessing the runTime selectable coordinateRotation
has moved to the rotation() method:
Old: Info<< "Rotation input: " << cs.R() << nl;
New: Info<< "Rotation input: " << cs.rotation() << nl;
- Naming consistency changes may also cause code to break.
Old: transformVector()
New: transformPrincipal()
The old method name transformTensor() now simply becomes transform().
====================
New methods
====================
For operations requiring caching of the coordinate rotations, the
'R()' method can be used with multiple input points:
tensorField rots(cs.R(somePoints));
and later
Foam::transformList(rots, someVectors);
The rotation() method can also be used to change the rotation tensor
via a new coordinateRotation definition (issue #879).
The new methods transformPoint/invTransformPoint provide
transformations with an origin offset using Cartesian for both local
and global points. These can be used to determine the local position
based on the origin/rotation without interpreting it as a r-theta-z
value, for example.
================
Input format
================
- Streamline dictionary input requirements
* The default type is cartesian.
* The default rotation type is the commonly used axes rotation
specification (with e1/e2/3), which is assumed if the 'rotation'
sub-dictionary does not exist.
Example,
Compact specification:
coordinateSystem
{
origin (0 0 0);
e2 (0 1 0);
e3 (0.5 0 0.866025);
}
Full specification (also accepts the longer 'coordinateRotation'
sub-dictionary name):
coordinateSystem
{
type cartesian;
origin (0 0 0);
rotation
{
type axes;
e2 (0 1 0);
e3 (0.5 0 0.866025);
}
}
This simplifies the input for many cases.
- Additional rotation specification 'none' (an identity rotation):
coordinateSystem
{
origin (0 0 0);
rotation { type none; }
}
- Additional rotation specification 'axisAngle', which is similar
to the -rotate-angle option for transforming points (issue #660).
For some cases this can be more intuitive.
For example,
rotation
{
type axisAngle;
axis (0 1 0);
angle 30;
}
vs.
rotation
{
type axes;
e2 (0 1 0);
e3 (0.5 0 0.866025);
}
- shorter names (or older longer names) for the coordinate rotation
specification.
euler EulerRotation
starcd STARCDRotation
axes axesRotation
================
Coding Style
================
- use Foam::coordSystem namespace for categories of coordinate systems
(cartesian, cylindrical, indirect). This reduces potential name
clashes and makes a clearer declaration. Eg,
coordSystem::cartesian csys_;
The older names (eg, cartesianCS, etc) remain available via typedefs.
- added coordinateRotations namespace for better organization and
reduce potential name clashes.
- corrected the mass based correction and updated the misleading function
arguments
- moved the option to the optimisation switches, e.g.:
OptimisationSwitches
{
experimentalDdtCorr 1;
}
- default remains off/no (0)
- controlled by the the 'printExecutionFormat' InfoSwitch in
etc/controlDict
// Style for "ExecutionTime = " output
// - 0 = seconds (with trailing 's')
// - 1 = day-hh:mm:ss
ExecutionTime = 112135.2 s ClockTime = 113017 s
ExecutionTime = 1-07:08:55.20 ClockTime = 1-07:23:37
- Callable via the new Time::printExecutionTime() method,
which also helps to reduce clutter in the applications.
Eg,
runTime.printExecutionTime(Info);
vs
Info<< "ExecutionTime = " << runTime.elapsedCpuTime() << " s"
<< " ClockTime = " << runTime.elapsedClockTime() << " s"
<< nl << endl;
--
ENH: return elapsedClockTime() and clockTimeIncrement as double
- previously returned as time_t, which is less portable.
- The bitSet class replaces the old PackedBoolList class.
The redesign provides better block-wise access and reduced method
calls. This helps both in cases where the bitSet may be relatively
sparse, and in cases where advantage of contiguous operations can be
made. This makes it easier to work with a bitSet as top-level object.
In addition to the previously available count() method to determine
if a bitSet is being used, now have simpler queries:
- all() - true if all bits in the addressable range are empty
- any() - true if any bits are set at all.
- none() - true if no bits are set.
These are faster than count() and allow early termination.
The new test() method tests the value of a single bit position and
returns a bool without any ambiguity caused by the return type
(like the get() method), nor the const/non-const access (like
operator[] has). The name corresponds to what std::bitset uses.
The new find_first(), find_last(), find_next() methods provide a faster
means of searching for bits that are set.
This can be especially useful when using a bitSet to control an
conditional:
OLD (with macro):
forAll(selected, celli)
{
if (selected[celli])
{
sumVol += mesh_.cellVolumes()[celli];
}
}
NEW (with const_iterator):
for (const label celli : selected)
{
sumVol += mesh_.cellVolumes()[celli];
}
or manually
for
(
label celli = selected.find_first();
celli != -1;
celli = selected.find_next()
)
{
sumVol += mesh_.cellVolumes()[celli];
}
- When marking up contiguous parts of a bitset, an interval can be
represented more efficiently as a labelRange of start/size.
For example,
OLD:
if (isA<processorPolyPatch>(pp))
{
forAll(pp, i)
{
ignoreFaces.set(i);
}
}
NEW:
if (isA<processorPolyPatch>(pp))
{
ignoreFaces.set(pp.range());
}
- In addition to the traditional Flex-based parser, added a Ragel-based
parser and a handwritten one.
Some representative timings for reading 5874387 points (1958129 tris):
Flex Ragel Manual
5.2s 4.8s 6.7s total reading time
3.8s 3.4s 5.3s without point merging
- now avoid Istream and token mechanism in favour of a simpler string
parser. This makes the code clearer, smaller, robuster.
- provide convenience ge/gt/le/lt static constructors for scalarRange
for using bounds directly with specifying via a string parameter.
- scalarRange, scalarRanges now follow the unary predicate pattern
(using an operator() for testing). This allows their reuse in
other contexts. Eg, for filtering operations:
myHash.filterValues(scalarRange::ge(100));
- remove unused scalarRanges methods that were specific to handling
lists of time values. These were superseded by timeSelector methods
several versions ago.
- this compact form shows the subscription per host in the unsorted
mpi order
nProcs : 18
Hosts :
(
(node1 6)
(node2 8)
(node3 4)
)
This provides a succinct overview of which hosts have been
subscribed or oversubscribed.
- The longer list of "slave.pid" ... remains available on the
InfoSwitch 'writeHosts'
old "positions" file form
The change to barycentric-based tracking changed the contents of the
cloud "positions" file to a new format comprising the barycentric
co-ordinates and other cell position-based info. This broke
backwards compatibility, providing no option to restart old cases
(v1706 and earlier), and caused difficulties for dependent code, e.g.
for post-processing utilities that could only infer the contents only
after reading.
The barycentric position info is now written to a file called
"coordinates" with provision to restart old cases for which only the
"positions" file is available. Related utilities, e.g. for parallel
running and data conversion have been updated to be able to support both
file types.
To write the "positions" file by default, use set the following option
in the InfoSwitches section of the controlDict:
writeLagrangianPositions 1;
Original commit message:
------------------------
Parallel IO: New collated file format
When an OpenFOAM simulation runs in parallel, the data for decomposed fields and
mesh(es) has historically been stored in multiple files within separate
directories for each processor. Processor directories are named 'processorN',
where N is the processor number.
This commit introduces an alternative "collated" file format where the data for
each decomposed field (and mesh) is collated into a single file, which is
written and read on the master processor. The files are stored in a single
directory named 'processors'.
The new format produces significantly fewer files - one per field, instead of N
per field. For large parallel cases, this avoids the restriction on the number
of open files imposed by the operating system limits.
The file writing can be threaded allowing the simulation to continue running
while the data is being written to file. NFS (Network File System) is not
needed when using the the collated format and additionally, there is an option
to run without NFS with the original uncollated approach, known as
"masterUncollated".
The controls for the file handling are in the OptimisationSwitches of
etc/controlDict:
OptimisationSwitches
{
...
//- Parallel IO file handler
// uncollated (default), collated or masterUncollated
fileHandler uncollated;
//- collated: thread buffer size for queued file writes.
// If set to 0 or not sufficient for the file size threading is not used.
// Default: 2e9
maxThreadFileBufferSize 2e9;
//- masterUncollated: non-blocking buffer size.
// If the file exceeds this buffer size scheduled transfer is used.
// Default: 2e9
maxMasterFileBufferSize 2e9;
}
When using the collated file handling, memory is allocated for the data in the
thread. maxThreadFileBufferSize sets the maximum size of memory in bytes that
is allocated. If the data exceeds this size, the write does not use threading.
When using the masterUncollated file handling, non-blocking MPI communication
requires a sufficiently large memory buffer on the master node.
maxMasterFileBufferSize sets the maximum size in bytes of the buffer. If the
data exceeds this size, the system uses scheduled communication.
The installation defaults for the fileHandler choice, maxThreadFileBufferSize
and maxMasterFileBufferSize (set in etc/controlDict) can be over-ridden within
the case controlDict file, like other parameters. Additionally the fileHandler
can be set by:
- the "-fileHandler" command line argument;
- a FOAM_FILEHANDLER environment variable.
A foamFormatConvert utility allows users to convert files between the collated
and uncollated formats, e.g.
mpirun -np 2 foamFormatConvert -parallel -fileHandler uncollated
An example case demonstrating the file handling methods is provided in:
$FOAM_TUTORIALS/IO/fileHandling
The work was undertaken by Mattijs Janssens, in collaboration with Henry Weller.
