The schedule.ccl File

By default, no routine of a thorn will be run. The schedule.ccl file defines those that should be run, and when and under which conditions they should be run.

The specification of routine scheduling is via a schedule block which consists of lines of the form

schedule <name> at <time bin> [other options]
{
  LANG:     <FORTRAN|C>
  STORAGE:  [group list with timelevels]
  TRIGGERS: [group list]
  SYNC:     [group list]
} "A description"

where <name> is the name of the routine, and <time bin> is the name of a schedule bin (the CCTK_ prefix is optional). A list of the most useful schedule bins for application thorns is given here, a complete and more descriptive list is provided in Appendix E3:

CCTK_STARTUP

For routines, run before the grid hierarchy is set up, for example, function registration.

CCTK_PARAMCHECK

For routines that check parameter combinations, routines registered here only have access to the grid size and the parameters.

CCTK_BASEGRID

Responsible for setting up coordinates, etc.

CCTK_INITIAL

For generating initial data.

CCTK_POSTINITIAL

Tasks which must be applied after initial data is created.

CCTK_PRESTEP

Stuff done before the evolution step.

CCTK_EVOL

The evolution step.

CCTK_POSTSTEP

Stuff done after the evolution step.

CCTK_ANALYSIS

For analysing data.

The other options allow finer-grained control of the scheduling. It is possible to state that the routine must run BEFORE or AFTER another routine or set of routines. It is also possible to schedule the routine under an alias name by using AS <alias_name>.

LANG

The LANG keyword specifies the linkage of the scheduled routine which determines how to call it from the scheduler. C and Fortran linkage are possible here. C++ routines should be defined as extern "C" and registered as LANG: C.

STORAGE

The STORAGE keyword specifies any groups for which memory should be allocated for the duration of the routine. The storage status reverts to its previous status after the routine returns. The format of the STORAGE statement includes specifying the number of timelevels of each group for which storage should be activated.

STORAGE: <group1>[timelevels1], <group2>[timelevels2]

This number can range from one to the maximum number of timelevels for the group, as specified in the group definition in its interface.ccl file. If this maximum number is one, the timelevel specification can be omitted from the STORAGE statement.

TRIGGERS

TRIGGERS is used when the routine is registered at ANALYSIS. This is a special time bin; a routine registered here will only be called if one of the variables from a group in TRIGGERS is due for output. (A routine without TRIGGERS declaration will always be called.)

SYNC

The keyword SYNC specifies groups of variables which should be synchronised (that is, their ghostzones should be exchanged between processors) on exit from the routine. Specifying synchronisation of grid variables in schedule.ccl is an alternative to calling the functions CCTK_SyncGroup() or CCTK_SyncGroupsI() (see the Reference Manual) from inside a routine. Using the SYNC keyword in the schedule.ccl is the preferred method, since it provides the flesh with more information about the behaviour of your code.

Besides schedule blocks, it's possible to embed C style if/else statements in the schedule.ccl file. These can be used to schedule things based upon the value of a parameter.

Example I:

If the parameter evolve_hydro is positively set, the Fortran routine hydro_predictor is scheduled to run in the evolution loop, after the routine metric_predictor and before metric_corrector. The routine names metric_predictor and metric_corrector, may either be real routine names from the same or a different thorn, or they may be aliased routine names (see the next example).

Before entry to hydro_predictor, storage will be allocated for one timelevel for the group of grid variables hydro_variables on exit from the routine this storage will be deallocated and the contents of the variables will be lost.

if(CCTK_Equals(evolve_hydro,"yes"))
{
  SCHEDULE hydro_predictor AT evol AFTER metric_predictor BEFORE metric_corrector
  {
    LANG:     FORTRAN
    STORAGE:  hydro_variables[1]
  } "Do a predictor step on the hydro variables"
}

If the parameter evolve_hydro is set negatively, the hydro_predictor routine will not be called by the scheduler. Note that if the evolve_hydro parameter is STEERABLE, it can be dynamically scheduled and de-scheduled during a run if a steering interface is available.

Example II:

The thorns WaveToy77 and WaveToyC, each provide a routine to evolve the 3D wave equation: WaveToyF77_Evolution and WaveToyC_Evolution. The routine names have to be different, so that both thorns can be compiled at the same time, their functionality is identical though. Either one of them can then be activated at run time in the parameter file via ActiveThorns.

Since each evolution routine provides the same functionality, it makes sense to schedule them under the common alias WaveToy_Evolution to allow relative scheduling (BEFORE/AFTER) independent of the actual routine name (which may change depending on the activation in the parameter file).

In both cases, the group of variables scalarfield are synchronised across processes when the routine is exited.

schedule WaveToyF77_Evolution AS WaveToy_Evolution AT evol
{
  LANG: Fortran
  STORAGE: scalartmps
  SYNC: scalarfield
} "Evolution of 3D wave equation"

schedule WaveToyC_Evolution AS WaveToy_Evolution AT evol
{
  LANG: C
  STORAGE: scalartmps
  SYNC: scalarfield
} "Evolution of 3D wave equation"

The thorn IDScalarWave schedules the routine WaveBinary after the alias WaveToy_Evolution. It is scheduled independently of the C or Fortran routine name.

schedule WaveBinary AT evol AFTER WaveToy_Evolution
{
  STORAGE: wavetoy::scalarevolve
  LANG:    Fortran
} "Provide binary source during evolution"