Implementation Notes

At first, CCTK_InterpGridArrays() checks its function arguments for invalid values passed by the caller. In case of an error, the routine will issue an error message and return with an error code of either UTIL_ERROR_BAD_HANDLE for an invalid coordinate system and/or parameter options table, or UTIL_ERROR_BAD_INPUT otherwise. Currently there is the restriction that only CCTK_VARIABLE_REAL is accepted as the CCTK data type for the interpolation points coordinates.

Then the parameter options table is parsed and evaluated for additional information about the interpolation call (see section I2.3.2 for details).

In the single-processor case, CCTK_InterpGridArrays() would now invoke the local interpolation operator (as specified by its handle) by a call to CCTK_InterpLocalUniform() to perform the actual interpolation. The return code from this call is then also passed back to the user.

For the multi-processor case, PUGHInterp does a query call to the local interpolator first to find out whether it can deal with the number of interprocessor ghostzones available. For that purpose it sets up an array of two interpolation points which denote the extremes of the physical coordinates on a processor: the lower-left and upper-right point of the processor-local grid's bounding box^part731. The query gets passed the same user-supplied function arguments as for the real interpolation call, apart from the interpolation points coordinates (which now describe a processor's physical bounding box coordinates) and the output array pointers (which are all set to NULL in order to indicate that this is a query call only). A return code of CCTK_ERROR_INTERP_POINT_OUTSIDE from CCTK_InterpLocalUniform() for this query call (meaning the local interpolator potentially requires values from grid points which are outside of the available processor-local patch of the global grid) causes CCTK_InterpGridArrays() to return immediately with a CCTK_ERROR_INTERP_GHOST_SIZE_TOO_SMALL error code on all processors.

Otherwise the CCTK_InterpGridArrays() routine will continue and map the user-supplied interpolation points onto the processors which own these points. In a subsequent global communication all processors receive "their" interpolation points coordinates and call CCTK_InterpLocalUniform() with those. The interpolation results are then sent back to the processors which originally requested the interpolation points.

Like the PUGH driver thorn, PUGHInterp uses MPI for the necessary interprocessor communication. Note that the MPI_Alltoall()/MPI_Alltoallv() calls for the distribution of interpolation points coordinates to their owning processors and the back transfer of the interpolation results to the requesting processors are collective communication operations. So in the multi-processor case you must call CCTK_InterpGridArrays() in parallel on each processor (even if a processor doesn't request any points to interpolate at), otherwise the program will run into a deadlock.