diff --git a/doc/Doxygen/css/doxyMod.css b/doc/Doxygen/css/doxyMod.css index 11407eedd7..1831a683c8 100644 --- a/doc/Doxygen/css/doxyMod.css +++ b/doc/Doxygen/css/doxyMod.css @@ -12,6 +12,10 @@ list-style-type: square; } +.contents ol { + margin-left: 20px; +} + /* colour and underline to match OpenFOAM style */ div.title { diff --git a/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.C b/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.C index 101eb3b7c9..d004c617b8 100644 --- a/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.C +++ b/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.C @@ -21,95 +21,6 @@ License You should have received a copy of the GNU General Public License along with OpenFOAM. If not, see . - From scotch forum: - - By: Francois PELLEGRINI RE: Graph mapping 'strategy' string [ reply ] - 2008-08-22 10:09 Strategy handling in Scotch is a bit tricky. In order - not to be confused, you must have a clear view of how they are built. - Here are some rules: - - 1- Strategies are made up of "methods" which are combined by means of - "operators". - - 2- A method is of the form "m{param=value,param=value,...}", where "m" - is a single character (this is your first error: "f" is a method name, - not a parameter name). - - 3- There exist different sort of strategies : bipartitioning strategies, - mapping strategies, ordering strategies, which cannot be mixed. For - instance, you cannot build a bipartitioning strategy and feed it to a - mapping method (this is your second error). - - To use the "mapCompute" routine, you must create a mapping strategy, not - a bipartitioning one, and so use stratGraphMap() and not - stratGraphBipart(). Your mapping strategy should however be based on the - "recursive bipartitioning" method ("b"). For instance, a simple (and - hence not very efficient) mapping strategy can be : - - "b{sep=f}" - - which computes mappings with the recursive bipartitioning method "b", - this latter using the Fiduccia-Mattheyses method "f" to compute its - separators. - - If you want an exact partition (see your previous post), try - "b{sep=fx}". - - However, these strategies are not the most efficient, as they do not - make use of the multi-level framework. - - To use the multi-level framework, try for instance: - - "b{sep=m{vert=100,low=h,asc=f}x}" - - The current default mapping strategy in Scotch can be seen by using the - "-vs" option of program gmap. It is, to date: - - b - { - job=t, - map=t, - poli=S, - sep= - ( - m - { - asc=b - { - bnd=d{pass=40,dif=1,rem=1}f{move=80,pass=-1,bal=0.005}, - org=f{move=80,pass=-1,bal=0.005}, - width=3 - }, - low=h{pass=10}f{move=80,pass=-1,bal=0.0005}, - type=h, - vert=80, - rat=0.8 - } - | m - { - asc=b - { - bnd=d{pass=40,dif=1,rem=1}f{move=80,pass=-1,bal=0.005}, - org=f{move=80,pass=-1,bal=0.005}, - width=3 - }, - low=h{pass=10}f{move=80,pass=-1,bal=0.0005}, - type=h, - vert=80, - rat=0.8 - } - ) - } - - - - Note: writeGraph=true : writes out .dgr files for debugging. Run with e.g. - - mpirun -np 4 dgpart 2 'region0_%r.dgr' - - - %r gets replaced by current processor rank - - decompose into 2 domains - \*---------------------------------------------------------------------------*/ #include "ptscotchDecomp.H" diff --git a/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.H b/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.H index a4ec01d1c9..6aa45a2903 100644 --- a/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.H +++ b/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.H @@ -25,7 +25,24 @@ Class Foam::ptscotchDecomp Description - PTScotch domain decomposition + PTScotch domain decomposition. + For the main details about how to define the strategies, see scotchDecomp. + + Nonetheless, when decomposing in parallel, using writeGraph=true + will write out \c .dgr files for debugging. For example, use these files + with \c dgpart as follows: + + \verbatim + mpirun -np 4 dgpart 2 'region0_%r.dgr' + \endverbatim + + where: + + \li \c %r gets replaced by current processor rank + \li it will decompose into 2 domains + +SeeAlso + Foam::scotchDecomp SourceFiles ptscotchDecomp.C diff --git a/src/parallel/decompose/scotchDecomp/scotchDecomp.C b/src/parallel/decompose/scotchDecomp/scotchDecomp.C index dd72d4e3b1..52c0f173fb 100644 --- a/src/parallel/decompose/scotchDecomp/scotchDecomp.C +++ b/src/parallel/decompose/scotchDecomp/scotchDecomp.C @@ -21,102 +21,6 @@ License You should have received a copy of the GNU General Public License along with OpenFOAM. If not, see . - From scotch forum: - - By: Francois PELLEGRINI RE: Graph mapping 'strategy' string [ reply ] - 2008-08-22 10:09 Strategy handling in Scotch is a bit tricky. In order - not to be confused, you must have a clear view of how they are built. - Here are some rules: - - 1- Strategies are made up of "methods" which are combined by means of - "operators". - - 2- A method is of the form "m{param=value,param=value,...}", where "m" - is a single character (this is your first error: "f" is a method name, - not a parameter name). - - 3- There exist different sort of strategies : bipartitioning strategies, - mapping strategies, ordering strategies, which cannot be mixed. For - instance, you cannot build a bipartitioning strategy and feed it to a - mapping method (this is your second error). - - To use the "mapCompute" routine, you must create a mapping strategy, not - a bipartitioning one, and so use stratGraphMap() and not - stratGraphBipart(). Your mapping strategy should however be based on the - "recursive bipartitioning" method ("b"). For instance, a simple (and - hence not very efficient) mapping strategy can be : - - "b{sep=f}" - - which computes mappings with the recursive bipartitioning method "b", - this latter using the Fiduccia-Mattheyses method "f" to compute its - separators. - - If you want an exact partition (see your previous post), try - "b{sep=fx}". - - However, these strategies are not the most efficient, as they do not - make use of the multi-level framework. - - To use the multi-level framework, try for instance: - - "b{sep=m{vert=100,low=h,asc=f}x}" - - The current default mapping strategy in Scotch can be seen by using the - "-vs" option of program gmap. It is, to date: - - r - { - job=t, - map=t, - poli=S, - sep= - ( - m - { - asc=b - { - bnd= - ( - d{pass=40,dif=1,rem=1} - | - ) - f{move=80,pass=-1,bal=0.002491}, - org=f{move=80,pass=-1,bal=0.002491}, - width=3 - }, - low=h{pass=10} - f{move=80,pass=-1,bal=0.002491}, - type=h, - vert=80, - rat=0.8 - } - | m - { - asc=b - { - bnd= - ( - d{pass=40,dif=1,rem=1} - | - ) - f{move=80,pass=-1,bal=0.002491}, - org=f{move=80,pass=-1,bal=0.002491}, - width=3 - }, - low=h{pass=10} - f{move=80,pass=-1,bal=0.002491}, - type=h, - vert=80, - rat=0.8 - } - ) - } - - - Note: instead of gmap run gpart \ -vs \ - where \ can be obtained by running with 'writeGraph=true' - \*---------------------------------------------------------------------------*/ #include "scotchDecomp.H" diff --git a/src/parallel/decompose/scotchDecomp/scotchDecomp.H b/src/parallel/decompose/scotchDecomp/scotchDecomp.H index 714bdb76fc..a2a2a298ca 100644 --- a/src/parallel/decompose/scotchDecomp/scotchDecomp.H +++ b/src/parallel/decompose/scotchDecomp/scotchDecomp.H @@ -2,7 +2,7 @@ ========= | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox \\ / O peration | - \\ / A nd | Copyright (C) 2011-2015 OpenFOAM Foundation + \\ / A nd | Copyright (C) 2011-2016 OpenFOAM Foundation \\/ M anipulation | ------------------------------------------------------------------------------- License @@ -27,9 +27,184 @@ Class Description Scotch domain decomposition. When run in parallel will collect the whole graph on to the master, - decompose and send back. Run ptscotchDecomp for proper distributed + decompose and send back. Use ptscotchDecomp for proper distributed decomposition. + Quoting from the Scotch forum, on the 2008-08-22 10:09, Francois + PELLEGRINI posted the following details: + \verbatim + RE: Graph mapping 'strategy' string + + Strategy handling in Scotch is a bit tricky. In order + not to be confused, you must have a clear view of how they are built. + Here are some rules: + + 1- Strategies are made up of "methods" which are combined by means of + "operators". + + 2- A method is of the form "m{param=value,param=value,...}", where "m" + is a single character (this is your first error: "f" is a method name, + not a parameter name). + + 3- There exist different sort of strategies : bipartitioning strategies, + mapping strategies, ordering strategies, which cannot be mixed. For + instance, you cannot build a bipartitioning strategy and feed it to a + mapping method (this is your second error). + + To use the "mapCompute" routine, you must create a mapping strategy, not + a bipartitioning one, and so use stratGraphMap() and not + stratGraphBipart(). Your mapping strategy should however be based on the + "recursive bipartitioning" method ("b"). For instance, a simple (and + hence not very efficient) mapping strategy can be : + + "b{sep=f}" + + which computes mappings with the recursive bipartitioning method "b", + this latter using the Fiduccia-Mattheyses method "f" to compute its + separators. + + If you want an exact partition (see your previous post), try + "b{sep=fx}". + + However, these strategies are not the most efficient, as they do not + make use of the multi-level framework. + + To use the multi-level framework, try for instance: + + "b{sep=m{vert=100,low=h,asc=f}x}" + + The current default mapping strategy in Scotch can be seen by using the + "-vs" option of program gmap. It is, to date: + + r + { + job=t, + map=t, + poli=S, + sep= + ( + m + { + asc=b + { + bnd= + ( + d{pass=40,dif=1,rem=1} + | + ) + f{move=80,pass=-1,bal=0.002491}, + org=f{move=80,pass=-1,bal=0.002491}, + width=3 + }, + low=h{pass=10} + f{move=80,pass=-1,bal=0.002491}, + type=h, + vert=80, + rat=0.8 + } + | m + { + asc=b + { + bnd= + ( + d{pass=40,dif=1,rem=1} + | + ) + f{move=80,pass=-1,bal=0.002491}, + org=f{move=80,pass=-1,bal=0.002491}, + width=3 + }, + low=h{pass=10} + f{move=80,pass=-1,bal=0.002491}, + type=h, + vert=80, + rat=0.8 + } + ) + } + \endverbatim + + Given that this information was written in 2008, this example strategy will + unlikely work as-is with the more recent Scotch versions. Therefore, the + steps for getting the current default strategy from within Scotch, is to do + the following steps: + +
    +
  1. Edit the file system/decomposeParDict and use the following + settings: + + \verbatim + method scotch; + + scotchCoeffs + { + writeGraph true; + } + \endverbatim +
  2. + +
  3. Run \c decomposePar. For example, it will write a file named + region0.grf. +
  4. + +
  5. Now, instead of using \c gmap, run \c gpart with the following + command structure to get the default strategy: + + \verbatim + gpart \ -vs \ + \endverbatim + + where: + +
      +
    • \ is the file that was obtained with the option + writeGraph=true, namely region0.grf. +
    • +
    • \ is the \c numberOfSubdomains defined in the dictionary + file. +
    • +
    +
  6. + +
  7. At the end of the execution will be shown a long string, similar to + the following example (complete line was cropped at [...]): + + \verbatim + S Strat=m{asc=b{width=3,bnd=d{pass=40,dif=1,rem=0}[...],type=h} + \endverbatim +
  8. + +
  9. Edit the file system/decomposeParDict once again and add + the \c strategy entry as exemplified: + + \verbatim + method scotch; + + scotchCoeffs + { + //writeGraph true; + strategy "m{asc=b{width=3,bnd=d{pass=40,dif=1,rem=0}[...],type=h}"; + } + \endverbatim +
  10. + +
  11. Finally, run \c decomposePar once again, to at least test if it + works as intended. +
  12. + +
+ +Note + \c gpart can be found in the current search path by adding the respective + \c bin folder from the Scotch installation, namely by running the following + commands: + + \verbatim + source $(foamEtcFile config.sh/scotch) + export PATH=$PATH:$SCOTCH_ARCH_PATH/bin + \endverbatim + SourceFiles scotchDecomp.C