TransientMultiApp

MultiApp for performing coupled simulations with the parent and sub-application both progressing in time.

Overview

The TransientMultiApp is designed to perform simulations with sub-applications that progress in time with the main application. A TransientMultiApp requires that your "sub-apps" use an Executioner derived from Transient.

Time step size

By default, the time step size used by the main app and the sub-apps is the same and is determined by taking the minimum over the time steps computed in the main app and all sub apps.

Sub-cycling, where sub apps may perform multiple (smaller) time steps for each main app time step, may be enabled using the "sub_cycling" parameter. When performing sub-cycling, transferred auxiliary variables on sub-apps are allowed to be interpolated between the start time and the end time of the main app with "interpolate_transfers" parameter.

Start and end time

The "start_time" of the sub app may differ from that of the main app. If the start time of the sub app is smaller than that of the main app, then if there is sub-cycling, then in the first step, the sub app will take steps until its time becomes synchronized with the main app time (taking a total time step larger than the main app's first step). If there is no sub-cycling, then the sub app will be out of sync with the main app. If the start time of the sub app is larger than the main app's start time, then sub apps will not take steps until their start time is reached, regardless of sub-cycling.

The "end_time" of the sub app may also differ from the main app. If the sub app's end time is smaller than the parent's end time, then the main app will continue to solve past the sub app's end time, but the sub app will not solve anymore. If the sub app's end time is larger than the parent's end time, then the sub app will not reach its end time, since the solve will not go past the parent app's end time.

Time state of TransientMultiApps

TransientMultiApps are "auto-advanced" by default whenever we are not doing Picard iterations between the main and sub-application. This means that the Transient::endStep and Transient::postStep methods of the sub-applications executioner are called, regardless of whether the sub-application solve fails or not. The endStep method increments the time and also performs EXEC_TIMESTEP_END output. When sub-applications are auto-advanced, their endStep call happens before the main application's endStep call. This has the important benefit that when main application output occurs, the sub-application's and main application's time states are the same, which enables MOOSE restart/recovery capability.

Handling sub-application solve failures

As noted above, the default behavior when running TransientMultiApps is that their time state is incremented, e.g. they are "auto-advanced", regardless of whether their solve is actually successful. This is undesirable behavior, but we believe that the syncing of main and sub-application states, under normal operation, to enable correct checkpoint output is a good trade. Given the constraints of the elected design, there are still multiple ways to turn a failed sub-application solve from a warning into an exception that will force corrective behavior in either the sub- or main-application:

The user can set auto_advance = false in the Executioner block of the main application . This will cause the main application to immediately cut its time-step when the sub-application fails. However, setting this parameter to false also eliminates the possibility of doing restart/recover because the main and sub will be out of sync if/when checkpoint output occurs.
The user can set catch_up = true in the TransientMultiApp block. This will cause the sub-application to try and catch up to the main application after a sub-app failed solve. If catch-up is unsuccessful, then MOOSE registers this as a true failure of the solve, and the main dt will then get cut. This option has the advantage of keeping the main and sub transient states in sync, enabling accurate restart/recover data.

In general, if the user wants sub-application failed solves to be treated as exceptions, we recommend option 2 over option 1.

Example Input File Syntax

The following input file shows the creation of a TransientMultiApp object with the time step size being governed by the main application.

[MultiApps]
  [sub_app]
    type = TransientMultiApp
    app_type = MooseTestApp
    input_files = 'dt_from_parent_sub.i'
    positions = '0   0   0
                 0.5 0.5 0
                 0.6 0.6 0
                 0.7 0.7 0'
  []
[]

Input Parameters

input_filesThe input file for each App. If this parameter only contains one input file it will be used for all of the Apps. When using 'positions_from_file' it is also admissable to provide one input_file per file.
C++ Type:std::vector<FileName>
Controllable:No
Description:The input file for each App. If this parameter only contains one input file it will be used for all of the Apps. When using 'positions_from_file' it is also admissable to provide one input_file per file.

Required Parameters

app_typeThe type of application to build (applications not registered can be loaded with dynamic libraries. Parent application type will be used if not provided.
C++ Type:MooseEnum
Options:FerretApp
Controllable:No
Description:The type of application to build (applications not registered can be loaded with dynamic libraries. Parent application type will be used if not provided.
bounding_box_inflation0.01Relative amount to 'inflate' the bounding box of this MultiApp.
Default:0.01
C++ Type:double
Controllable:No
Description:Relative amount to 'inflate' the bounding box of this MultiApp.
bounding_box_padding0 0 0Additional padding added to the dimensions of the bounding box. The values are added to the x, y and z dimension respectively.
Default:0 0 0
C++ Type:libMesh::Point
Controllable:No
Description:Additional padding added to the dimensions of the bounding box. The values are added to the x, y and z dimension respectively.
clone_parent_meshFalseTrue to clone parent app mesh and use it for this MultiApp.
Default:False
C++ Type:bool
Controllable:No
Description:True to clone parent app mesh and use it for this MultiApp.
execute_onTIMESTEP_BEGINThe list of flag(s) indicating when this object should be executed, the available options include NONE, INITIAL, LINEAR, NONLINEAR, TIMESTEP_END, TIMESTEP_BEGIN, MULTIAPP_FIXED_POINT_END, MULTIAPP_FIXED_POINT_BEGIN, FINAL, CUSTOM, ALWAYS.
Default:TIMESTEP_BEGIN
C++ Type:ExecFlagEnum
Options:NONE, INITIAL, LINEAR, NONLINEAR, TIMESTEP_END, TIMESTEP_BEGIN, MULTIAPP_FIXED_POINT_END, MULTIAPP_FIXED_POINT_BEGIN, FINAL, CUSTOM, ALWAYS
Controllable:No
Description:The list of flag(s) indicating when this object should be executed, the available options include NONE, INITIAL, LINEAR, NONLINEAR, TIMESTEP_END, TIMESTEP_BEGIN, MULTIAPP_FIXED_POINT_END, MULTIAPP_FIXED_POINT_BEGIN, FINAL, CUSTOM, ALWAYS.
global_time_offset0The time offset relative to the parent application for the purpose of starting a subapp at a different time from the parent application. The global time will be ahead by the offset specified here.
Default:0
C++ Type:double
Controllable:No
Description:The time offset relative to the parent application for the purpose of starting a subapp at a different time from the parent application. The global time will be ahead by the offset specified here.

Optional Parameters

catch_upFalseIf true this will allow failed solves to attempt to 'catch up' using smaller timesteps.
Default:False
C++ Type:bool
Controllable:No
Description:If true this will allow failed solves to attempt to 'catch up' using smaller timesteps.
max_catch_up_steps2Maximum number of steps to allow an app to take when trying to catch back up after a failed solve.
Default:2
C++ Type:double
Controllable:No
Description:Maximum number of steps to allow an app to take when trying to catch back up after a failed solve.

Recovering Failed Solutions Parameters

cli_argsAdditional command line arguments to pass to the sub apps. If one set is provided the arguments are applied to all, otherwise there must be a set for each sub app.
C++ Type:std::vector<std::string>
Controllable:Yes
Description:Additional command line arguments to pass to the sub apps. If one set is provided the arguments are applied to all, otherwise there must be a set for each sub app.
cli_args_filesFile names that should be looked in for additional command line arguments to pass to the sub apps. Each line of a file is set to each sub app. If only one line is provided, it will be applied to all sub apps.
C++ Type:std::vector<FileName>
Controllable:No
Description:File names that should be looked in for additional command line arguments to pass to the sub apps. Each line of a file is set to each sub app. If only one line is provided, it will be applied to all sub apps.

Passing Command Line Argument Parameters

control_tagsAdds user-defined labels for accessing object parameters via control logic.
C++ Type:std::vector<std::string>
Controllable:No
Description:Adds user-defined labels for accessing object parameters via control logic.
enableTrueSet the enabled status of the MooseObject.
Default:True
C++ Type:bool
Controllable:Yes
Description:Set the enabled status of the MooseObject.
implicitTrueDetermines whether this object is calculated using an implicit or explicit form
Default:True
C++ Type:bool
Controllable:No
Description:Determines whether this object is calculated using an implicit or explicit form
use_displaced_meshFalseWhether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.
Default:False
C++ Type:bool
Controllable:No
Description:Whether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.
wait_for_first_app_initFalseCreate the first sub-application on rank 0, then MPI_Barrier before creating the next N-1 apps (on all ranks). This is only needed if your sub-application needs to perform some setup actions in quiet, without other sub-applications working at the same time.
Default:False
C++ Type:bool
Controllable:No
Description:Create the first sub-application on rank 0, then MPI_Barrier before creating the next N-1 apps (on all ranks). This is only needed if your sub-application needs to perform some setup actions in quiet, without other sub-applications working at the same time.

Advanced Parameters

detect_steady_stateFalseIf true then while sub_cycling a steady state check will be done. In this mode output will only be done once the MultiApp reaches the target time or steady state is reached
Default:False
C++ Type:bool
Controllable:No
Description:If true then while sub_cycling a steady state check will be done. In this mode output will only be done once the MultiApp reaches the target time or steady state is reached
interpolate_transfersFalseOnly valid when sub_cycling. This allows transferred values to be interpolated over the time frame the MultiApp is executing over when sub_cycling
Default:False
C++ Type:bool
Controllable:No
Description:Only valid when sub_cycling. This allows transferred values to be interpolated over the time frame the MultiApp is executing over when sub_cycling
max_failures0Maximum number of solve failures tolerated while sub_cycling.
Default:0
C++ Type:unsigned int
Controllable:No
Description:Maximum number of solve failures tolerated while sub_cycling.
output_sub_cyclesFalseIf true then every sub-cycle will be output.
Default:False
C++ Type:bool
Controllable:No
Description:If true then every sub-cycle will be output.
print_sub_cyclesTrueToggle the display of sub-cycles on the screen.
Default:True
C++ Type:bool
Controllable:No
Description:Toggle the display of sub-cycles on the screen.
steady_state_tol1e-08The relative difference between the new solution and the old solution that will be considered to be at steady state
Default:1e-08
C++ Type:double
Controllable:No
Description:The relative difference between the new solution and the old solution that will be considered to be at steady state
sub_cyclingFalseSet to true to allow this MultiApp to take smaller timesteps than the rest of the simulation. More than one timestep will be performed for each parent application timestep
Default:False
C++ Type:bool
Controllable:No
Description:Set to true to allow this MultiApp to take smaller timesteps than the rest of the simulation. More than one timestep will be performed for each parent application timestep

Sub Cycling Parameters

keep_solution_during_restoreFalseThis is useful when doing MultiApp coupling iterations. It takes the final solution from the previous coupling iterationand re-uses it as the initial guess for the next coupling iteration
Default:False
C++ Type:bool
Controllable:No
Description:This is useful when doing MultiApp coupling iterations. It takes the final solution from the previous coupling iterationand re-uses it as the initial guess for the next coupling iteration
relaxation_factor1Fraction of newly computed value to keep.Set between 0 and 2.
Default:1
C++ Type:double
Controllable:No
Description:Fraction of newly computed value to keep.Set between 0 and 2.
transformed_postprocessorsList of subapp postprocessors to use coupling algorithm on during Multiapp coupling iterations
C++ Type:std::vector<PostprocessorName>
Controllable:No
Description:List of subapp postprocessors to use coupling algorithm on during Multiapp coupling iterations
transformed_variablesList of subapp variables to use coupling algorithm on during Multiapp coupling iterations
C++ Type:std::vector<std::string>
Controllable:No
Description:List of subapp variables to use coupling algorithm on during Multiapp coupling iterations

Fixed Point Acceleration Of Multiapp Quantities Parameters

library_load_dependenciesFalseTells MOOSE to manually load library dependencies. This should not be necessary and is here for debugging/troubleshooting.
Default:False
C++ Type:bool
Controllable:No
Description:Tells MOOSE to manually load library dependencies. This should not be necessary and is here for debugging/troubleshooting.
library_nameThe file name of the library (*.la file) that will be dynamically loaded.
C++ Type:std::string
Controllable:No
Description:The file name of the library (*.la file) that will be dynamically loaded.
library_pathPath to search for dynamic libraries (please avoid committing absolute paths in addition to MOOSE_LIBRARY_PATH)
C++ Type:std::string
Controllable:No
Description:Path to search for dynamic libraries (please avoid committing absolute paths in addition to MOOSE_LIBRARY_PATH)

Dynamic Loading Parameters

max_procs_per_app4294967295Maximum number of processors to give to each App in this MultiApp. Useful for restricting small solves to just a few procs so they don't get spread out
Default:4294967295
C++ Type:unsigned int
Controllable:No
Description:Maximum number of processors to give to each App in this MultiApp. Useful for restricting small solves to just a few procs so they don't get spread out
min_procs_per_app1Minimum number of processors to give to each App in this MultiApp. Useful for larger, distributed mesh solves.
Default:1
C++ Type:unsigned int
Controllable:No
Description:Minimum number of processors to give to each App in this MultiApp. Useful for larger, distributed mesh solves.

Parallelism Parameters

move_appsApps, designated by their 'numbers' starting with 0 corresponding to the order of the App positions, to be moved at move_time to move_positions
C++ Type:std::vector<unsigned int>
Controllable:No
Description:Apps, designated by their 'numbers' starting with 0 corresponding to the order of the App positions, to be moved at move_time to move_positions
move_positionsThe positions corresponding to each move_app.
C++ Type:std::vector<libMesh::Point>
Controllable:No
Description:The positions corresponding to each move_app.
move_time1.79769e+308The time at which Apps designated by move_apps are moved to move_positions.
Default:1.79769e+308
C++ Type:double
Controllable:No
Description:The time at which Apps designated by move_apps are moved to move_positions.

Timed Move Of Multiapps Parameters

output_in_positionFalseIf true this will cause the output from the MultiApp to be 'moved' by its position vector
Default:False
C++ Type:bool
Controllable:No
Description:If true this will cause the output from the MultiApp to be 'moved' by its position vector
positionsThe positions of the App locations. Each set of 3 values will represent a Point. This and 'positions_file' cannot be both supplied. If this and 'positions_file'/'_objects' are not supplied, a single position (0,0,0) will be used
C++ Type:std::vector<libMesh::Point>
Controllable:No
Description:The positions of the App locations. Each set of 3 values will represent a Point. This and 'positions_file' cannot be both supplied. If this and 'positions_file'/'_objects' are not supplied, a single position (0,0,0) will be used
positions_fileFilename(s) that should be looked in for positions. Each set of 3 values in that file will represent a Point. This and 'positions(_objects)' cannot be both supplied
C++ Type:std::vector<FileName>
Controllable:No
Description:Filename(s) that should be looked in for positions. Each set of 3 values in that file will represent a Point. This and 'positions(_objects)' cannot be both supplied
positions_objectsThe name of a Positions object that will contain the locations of the sub-apps created. This and 'positions(_file)' cannot be both supplied
C++ Type:std::vector<PositionsName>
Controllable:No
Description:The name of a Positions object that will contain the locations of the sub-apps created. This and 'positions(_file)' cannot be both supplied
run_in_positionFalseIf true this will cause the mesh from the MultiApp to be 'moved' by its position vector
Default:False
C++ Type:bool
Controllable:No
Description:If true this will cause the mesh from the MultiApp to be 'moved' by its position vector

Positions / Transformations Of The Multiapp Frame Of Reference Parameters

reset_appsThe Apps that will be reset when 'reset_time' is hit. These are the App 'numbers' starting with 0 corresponding to the order of the App positions. Resetting an App means that it is destroyed and recreated, possibly modeling the insertion of 'new' material for that app.
C++ Type:std::vector<unsigned int>
Controllable:No
Description:The Apps that will be reset when 'reset_time' is hit. These are the App 'numbers' starting with 0 corresponding to the order of the App positions. Resetting an App means that it is destroyed and recreated, possibly modeling the insertion of 'new' material for that app.
reset_timeThe time(s) at which to reset Apps given by the 'reset_apps' parameter. Resetting an App means that it is destroyed and recreated, possibly modeling the insertion of 'new' material for that app.
C++ Type:std::vector<double>
Controllable:No
Description:The time(s) at which to reset Apps given by the 'reset_apps' parameter. Resetting an App means that it is destroyed and recreated, possibly modeling the insertion of 'new' material for that app.

Reset Multiapp Parameters

tolerate_failureFalseIf true this MultiApp won't participate in dt decisions and will always be fast-forwarded to the current time.
Default:False
C++ Type:bool
Controllable:No
Description:If true this MultiApp won't participate in dt decisions and will always be fast-forwarded to the current time.

Accepting Failed Solutions Parameters

(../moose/test/tests/multiapps/transient_multiapp/dt_from_parent.i)

[Mesh]
  type = GeneratedMesh
  dim = 2
  nx = 10
  ny = 10
[]

[Variables]
  [./u]
  [../]
[]

[Kernels]
  [./diff]
    type = Diffusion
    variable = u
  [../]
  [./td]
    type = TimeDerivative
    variable = u
  [../]
[]

[BCs]
  [./left]
    type = DirichletBC
    variable = u
    boundary = left
    value = 0
  [../]
  [./right]
    type = DirichletBC
    variable = u
    boundary = right
    value = 1
  [../]
[]

[Executioner]
  type = Transient
  num_steps = 10
  dt = 0.2

  solve_type = 'PJFNK'

  petsc_options_iname = '-pc_type -pc_hypre_type'
  petsc_options_value = 'hypre boomeramg'
[]

[Outputs]
  exodus = true
[]

[MultiApps]
  [sub_app]
    type = TransientMultiApp
    app_type = MooseTestApp
    input_files = 'dt_from_parent_sub.i'
    positions = '0   0   0
                 0.5 0.5 0
                 0.6 0.6 0
                 0.7 0.7 0'
  []
[]