pocomc icon indicating copy to clipboard operation
pocomc copied to clipboard

Feature request: option for more frequent/regular checkpoints

Open ajdittmann opened this issue 7 months ago • 2 comments

For context, one of the main clusters I have access to places an unfortunate limit of 12 hours on each job submission, and another has a 16-hour limit. Because my log likelihood (already written in C) evaluations take a few seconds, as I have increased N_active and N_effective into the 10,000s in search of converged results (even in this limit, pocomc seems quite a bit more efficient than many other samplers), things have gotten to the point where the jobs can time out before making it to the next iteration and dumping a pickled output file (using save_every = 1).

Would it be possible to add an option along the lines of saving the sampler state after N likelihood evaluations, or some other more predictable metric?

ajdittmann avatar Jul 16 '24 15:07 ajdittmann