dill>0.3.5.1 unable to pickle CompiledFFI object

[Jmast]

fails starting with dill==0.3.6. Not sure what changed in dill; however, at the very least the maestro deps need to be dill<=0.3.5.1

pickler._batch_setitems(iter(source.items()))

File "/usr/lib/python3.8/pickle.py", line 997, in _batch_setitems save(v) File "/home/mast2/spl_flux_test/local/lib/python3.8/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "/usr/lib/python3.8/pickle.py", line 603, in save self.save_reduce(obj=obj, *rv) File "/usr/lib/python3.8/pickle.py", line 717, in save_reduce save(state) File "/home/mast2/spl_flux_test/local/lib/python3.8/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "/usr/lib/python3.8/pickle.py", line 560, in save f(self, obj) # Call unbound method with explicit self File "/home/mast2/spl_flux_test/local/lib/python3.8/site-packages/dill/_dill.py", line 1186, in save_module_dict StockPickler.save_dict(pickler, obj) File "/usr/lib/python3.8/pickle.py", line 971, in save_dict self._batch_setitems(obj.items()) File "/usr/lib/python3.8/pickle.py", line 997, in _batch_setitems save(v) File "/home/mast2/spl_flux_test/local/lib/python3.8/site-packages/dill/_dill.py", line 388, in save StockPickler.save(self, obj, save_persistent_id) File "/usr/lib/python3.8/pickle.py", line 578, in save rv = reduce(self.proto) TypeError: cannot pickle 'CompiledFFI' object 2022-11-22 17:20:10,629:name : 453:ERRO: Maestro workflow finished with exit status: 1

[FrankD412]

Thanks for the issue @Jmast -- I'll fix this by the end of the weekend. Is this holding you up at all? Do you need something sooner?

[Jmast]

Thanks, no rush, @FrankD412 , but there are a few other things:

1. I have a issue outstanding regarding corona flux version and maestro being merged that I commented on in PR: "added flux adaptor v0.31.0 #395") comment is here
2. Until a more suitable approach is implemented, the version checks for flux probably need to be entirely removed as I have found that the "strict" version throws an exception for "unreleased" installs of flux. Ultimately I think that the flux plugin needs to support a range of versions and should just be backward compatible as much as possible with logic based on version numbers. Right now in my forked version I just commented out the entire try/except block in connect_to_flux in maestrowf/abstracts/interfaces/flux.py because the StrictVersion() call throws an exception for versions like 0.42.0-84-g1dcbcd97d

[FrankD412]

Oh -- sorry about the delayed fix. Slipped through the cracks on my end. I can take a look at that soon.