stacks icon indicating copy to clipboard operation
stacks copied to clipboard

`multisession` failure when using extension packages

Open simonpcouch opened this issue 1 year ago • 2 comments

When making patches to fix runs on general-meta, I noticed that multisession workers required that needed parsnip extension packages be loaded via options.future inside of fit_members():

https://github.com/tidymodels/stacks/blob/f2e33419b5712ae7df3ca4d689c652188dce4b59/R/fit_members.R#L157

I wonder if we could find some way to restore the current parsnip model environment parsnip::get_model_env() in each worker. I also wonder how/why we don't tend to see this with normal tune runs.

simonpcouch avatar May 03 '24 14:05 simonpcouch

Related-ish to #118.

simonpcouch avatar May 03 '24 14:05 simonpcouch

This is likely a matter of passing required_pkgs(workflow) to .options.future.

simonpcouch avatar May 07 '24 14:05 simonpcouch

This may actually be a model registration issue--see https://github.com/tidymodels/parsnip/issues/1128.

simonpcouch avatar Jul 14 '24 18:07 simonpcouch

Based on that reprex in parsnip, I'm going to go ahead and close—we shouldn't need to pass any future options there, instead relying on set_dependency() calls in the model registration. If we run into this again, check that first!

simonpcouch avatar Jul 15 '24 13:07 simonpcouch