[Developers] optimising the size of checkpoints

Erik Schnetter schnetter at aei.mpg.de
Tue Jun 14 14:51:29 CDT 2005


On Tuesday 14 June 2005 09:12, Thomas Radke wrote:
> Currently the checkpoint methods available in Cactus save all the
> timelevels of all grid variables which have storage allocated,
> resulting in a maximum-size checkpoint.
> At least for some variables (eg. NaNChecker::NaNmask or other
> analysis grid functions), checkpointing doesn't seem necessary; other
> variables, especially scalars (eg. IOBasic::next_info_output_time),
> can be easily recomputed after recovery, thus saving another item in
> the checkpoint plus its attached metadata (which in this case would
> be larger than the actual data).
>
> I propose to tag such grid variables with something like 'CHECKPOINT
> = no' in a thorn's interface.ccl. The checkpoint routines could then
> query the tags table and save only non-tagged variables.
> Of course, one would also have to add the logic to properly
> initialise tagged variables in CCTK_POST_RECOVER_VARIABLES if need
> to.
>
> Comments ?

Good idea.

Since the tags are tables, it would need to be checkpoint=0 or 
checkpoint="no" (with quotes).  I would prefer the former, which would 
make it CCTK_INT, the same as boolean parameters.

When it comes to Carpet, note that by default all variables always have 
storage.  You could try Carpet::enable_all_storage=no and see whether 
this already has an effect.

A new F5 output format would also reduce the file sizes if it stores 
less metadata.  I'm working on a thorn that I should push.

How much would be saved?  Did you try estimating what fraction of data 
is unnecessary?

-erik

-- 
Erik Schnetter <schnetter at aei.mpg.de>   http://www.aei.mpg.de/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20050614/04eedf02/attachment-0002.bin 


More information about the Developers mailing list