cobaya icon indicating copy to clipboard operation
cobaya copied to clipboard

[WIP] fixes for parallel calls inside Jupyter

Open JesusTorrado opened this issue 1 year ago • 4 comments

For now just a workaround, that fixes issues with pickling in ipyparallel.

One of the problems can be fixed by not having __getstate__ of HasLogger return a copy, but the original dict minus the logger. This is indeed what a __getstate__ is supposed to do, but I still need to test this change.

The other fix removes the fast prior pdfs, because the way the method is assigned maybe does not propagate correctly across copies. It can probably be superseded by creating a custom scipy prior object inheriting from the scipy one and having the new logpdf as a first-class method.

EDIT: Copy fixed cherrypicked and added to https://github.com/CobayaSampler/cobaya/pull/349. Only fast-pdf missing.

JesusTorrado avatar Dec 06 '23 14:12 JesusTorrado

Hello Jesus, what kind of error was this giving you? I'm asking cause also the Minuit object was giving problems with the same __getstate__ (#332).

ggalloni avatar Dec 07 '23 09:12 ggalloni

Hi @ggalloni,

It's a recursion error produced by pickling while in the IPP cluster. You can try if the one-line change to log.py solves if in your case.

JesusTorrado avatar Dec 07 '23 16:12 JesusTorrado

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (ea587ce) 81.78% compared to head (81e389e) 81.73%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #343      +/-   ##
==========================================
- Coverage   81.78%   81.73%   -0.05%     
==========================================
  Files         133      133              
  Lines       11033    11026       -7     
==========================================
- Hits         9023     9012      -11     
- Misses       2010     2014       +4     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Dec 07 '23 21:12 codecov-commenter

Yeah, it was a recursion error also in my case, I'll try your workaround.

ggalloni avatar Dec 08 '23 10:12 ggalloni

@ggalloni could you check if this reworking of fast-pdf works? https://github.com/CobayaSampler/cobaya/pull/373

cmbant avatar Aug 09 '24 09:08 cmbant