Schedule Bins

Using the schedule.ccl files, thorn functions can be scheduled to run in the different timebins which are executed by the Cactus flesh. This chapter describes these standard timebins, and shows the flow of program execution through them.

Scheduled functions must be declared as

In C:

#include "cctk_Arguments.h"
void MyFunction (CCTK_ARGUMENTS);

In C++:

#include "cctk_Arguments.h"
extern "C" void MyFunction (CCTK_ARGUMENTS);

In Fortran:

#include "cctk_Arguments.h"
subroutine MyFunction (CCTK_ARGUMENTS)
   DECLARE_CCTK_ARGUMENTS
end

Exceptions are the functions that are scheduled in the bins CCTK_STARTUP, CCTK_RECOVER_PARAMETERS, and CCTK_SHUTDOWN. They do not take arguments, and they return an integer. They must be declared as

In C:

int MyFunction (void);

In C++

extern "C" int MyFunction ();

In Fortran:

integer function MyFunction ()
end

The return value in CCTK_STARTUP and CCTK_SHUTDOWN is unused, and might in the future be used to indicate whether an error occurred. You should return 0.

The return value in CCTK_RECOVER_PARAMETERS should be zero, positive, or negative, indicating that no parameters were recovered, that parameters were recovered successfully, or that an error occurred, respectively. Routines in this bin are executed in alphabetical order, according to the owning thorn's name, until one returns a positive value. All later routines are ignored. Schedule clauses BEFORE, AFTER, WHILE, IF, etc., are ignored.

CCTK_RECOVER_PARAMETERS

Used by thorns with relevant I/O methods as the point to read parameters when recovering from checkpoint files. Grid variables are not available in this timebin. Scheduling in this timebin is special (see above).

CCTK_STARTUP

Run before any grids are constructed, this is the timebin for example where grid independent information (e.g. output methods, reduction operators) is registered. Note that since no grids are setup at this point, grid variables cannot be used in routines scheduled here.

CCTK_WRAGH

This timebin is executed when all parameters are known, but before the driver thorn constructs the grid. It should only be used to set up information that is needed by the driver.

CCTK_PARAMCHECK

This timebin is for thorns to check the validity of parameter combinations. This bin is also executed before the grid hierarchy is made, so that routines scheduled here only have access to the global grid size and the parameters.

CCTK_PREREGRIDINITIAL

This timebin is used in mesh refinement settings. It is ignored for unigrid runs. This bin is executed whenever the grid hierarchy is about to change during evolution; compare CCTK_PREREGRID. Routines that decide the new grid structure should be scheduled in this bin.

CCTK_POSTREGRIDINITIAL

This timebin is used in mesh refinement settings. It is ignored for unigrid runs. This bin is executed whenever the grid hierarchy or patch setup has changed during evolution; see CCTK_POSTREGRID. It is, e.g., necessary to re-apply the boundary conditions or recalculate the grid points' coordinates after every changing the grid hierarchy.

CCTK_BASEGRID

This timebin is executed very early after a driver thorn constructs grid; this bin should only be used to set up coordinate systems on the newly created grids.

CCTK_INITIAL

This is the place to set up any required initial data. This timebin is not run when recovering from a checkpoint file.

CCTK_POSTINITIAL

This is the place to modify initial data, or to calculate data that depend on the initial data. This timebin is also not run when recovering from a checkpoint file.

CCTK_POSTRESTRICTINITIAL

This timebin is used only in mesh refinement settings. It is ignored for unigrid runs. This bin is executed after each restriction operation while initial data are set up; compare CCTK_POSTRESTRICT. It is, e.g., necessary to re-apply the boundary conditions after every restriction operation.

CCTK_POSTPOSTINITIAL

This is the place to modify initial data, or to calculate data that depend on the initial data. This timebin is executed after the recursive initialisation of finer grids if there is a mesh refinement hierarchy, and it is also not run when recovering from a checkpoint file.

CCTK_RECOVER_VARIABLES

Used by thorns with relevant I/O methods as the point to read in all the grid variables when recovering from checkpoint files.

CCTK_POST_RECOVER_VARIABLES

This timebin exists for scheduling any functions which need to modify grid variables after recovery.

CCTK_CPINITIAL

Used by thorns with relevant I/O methods as the point to checkpoint initial data if required.

CCTK_CHECKPOINT

Used by thorns with relevant I/O methods as the point to checkpoint data during the iterative loop when required.

CCTK_PREREGRID

This timebin is used in mesh refinement settings. It is ignored for unigrid runs. This bin is executed whenever the grid hierarchy is about to change during evolution; compare CCTK_PREREGRIDINITIAL. Routines that decide the new grid structure should be scheduled in this bin.

CCTK_POSTREGRID

This timebin is used in mesh refinement settings. It is ignored for unigrid runs. This bin is executed whenever the grid hierarchy or patch setup has changed during evolution; see CCTK_POSTREGRIDINITIAL. It is, e.g., necessary to re-apply the boundary conditions or recalculate the grid points' coordinates after every changing the grid hierarchy.

CCTK_PRESTEP

The timebin for scheduling any routines which need to be executed before any routines in the main evolution step. This timebin exists for thorn writers convenience, the BEFORE, AFTER etc. functionality of the schedule.ccl file should allow all functions to be scheduled in the main CCTK_EVOL timebin.

CCTK_EVOL

The timebin for the main evolution step.

CCTK_POSTRESTRICT

This timebin is used only in mesh refinement settings. It is ignored for unigrid runs. This bin is executed after each restriction operation during evolution; compare CCTK_POSTRESTRICTINITIAL. It is, e.g., necessary to re-apply the boundary conditions after every restriction operation.

CCTK_POSTSTEP

The timebin for scheduling any routines which need to be executed after all the routines in the main evolution step. This timebin exists for thorn writers convenience, the BEFORE, AFTER etc. functionality of the schedule.ccl file should allow all functions to be scheduled in the main CCTK_EVOL timebin.

CCTK_ANALYSIS

The ANALYSIS timebin is special, in that it is closely coupled with output, and routines which are scheduled here are typically only executed if output of analysis variables is required. Routines which perform analysis should be independent of the main evolution loop (that is, it should not matter for the results of a simulation whether routines in this timebin are executed or not).

CCTK_TERMINATE

Called after the main iteration loop when Cactus terminates. Note that sometime in this timebin a driver thorn should be destroying the grid hierarchy and removing grid variables.

CCTK_SHUTDOWN

Cactus final shutdown routines, after the grid hierarchy has been destroyed. Grid variables are no longer available.