Interfacing PETSc

The main task when interfacing PETSc consists of transferring data from the Cactus parallel data structures (gridfunctions) to the parallel structures provided by PETSc.

Here we explain the main steps, to be read with the code at hands.

1. The indices imin,imax ... are calculated and describe the starting/ending points in 3D local index space: ghostzones are not included here.
2. A linear global index is calculated describing the starting/ending points in linear global index space. Ghostzone are not included here.
3. A lookup gridfunction wsp is loaded identifying the 3D local index with the linear global index. Values of zero indicate boundaries.
4. PETSc matrices/vectors are created specifying the linear size: global endpoint - global startpoint.
5. For the elliptic class LinFlat the stencil functions are initialized with the standard 7-point stencil, the class LinMetric and LinConfMetric require a more sophisticated treatment described later.
6. Looping over the processor local grid points (in 3D local index space) the PETSc vectors and coefficient matrix is loaded if no boundary is present (wsp[i,j,k] not equal zero.);
7. Starting the PETSc vector and matrix assembly, nested for performance as recommended by PETSc.
8. Creation of the elliptic solver context and setting of options, followed by the call to the PETSc solver.
9. Upon completion of the solve, the PETSc solution has to transferred to the Cactus data structures.