[Developers] optimising the size of checkpoints
Erik Schnetter
schnetter at aei.mpg.de
Tue Jun 14 14:51:29 CDT 2005
On Tuesday 14 June 2005 09:12, Thomas Radke wrote:
> Currently the checkpoint methods available in Cactus save all the
> timelevels of all grid variables which have storage allocated,
> resulting in a maximum-size checkpoint.
> At least for some variables (eg. NaNChecker::NaNmask or other
> analysis grid functions), checkpointing doesn't seem necessary; other
> variables, especially scalars (eg. IOBasic::next_info_output_time),
> can be easily recomputed after recovery, thus saving another item in
> the checkpoint plus its attached metadata (which in this case would
> be larger than the actual data).
>
> I propose to tag such grid variables with something like 'CHECKPOINT
> = no' in a thorn's interface.ccl. The checkpoint routines could then
> query the tags table and save only non-tagged variables.
> Of course, one would also have to add the logic to properly
> initialise tagged variables in CCTK_POST_RECOVER_VARIABLES if need
> to.
>
> Comments ?
Good idea.
Since the tags are tables, it would need to be checkpoint=0 or
checkpoint="no" (with quotes). I would prefer the former, which would
make it CCTK_INT, the same as boolean parameters.
When it comes to Carpet, note that by default all variables always have
storage. You could try Carpet::enable_all_storage=no and see whether
this already has an effect.
A new F5 output format would also reduce the file sizes if it stores
less metadata. I'm working on a thorn that I should push.
How much would be saved? Did you try estimating what fraction of data
is unnecessary?
-erik
--
Erik Schnetter <schnetter at aei.mpg.de> http://www.aei.mpg.de/~eschnett/
My email is as private as my paper mail. I therefore support encrypting
and signing email messages. Get my PGP key from www.keyserver.net.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20050614/04eedf02/attachment-0002.bin
More information about the Developers
mailing list