aiida-core icon indicating copy to clipboard operation
aiida-core copied to clipboard

Currently all processes (calcfunctions) have to be toplevel (importable) in modules, otherwise uncachable

Open broeder-j opened this issue 4 years ago • 1 comments

Since is_valid_cache assumes that calcfunctions (all process classes/functions) are top-level (importable) in modules, calcfunctions which are not, become uncacheable. (If I understood this right)

I had classes of the following design to keep the provenance:

XY.py

def class XY:
    
    def function_x(self, *args, **kwargs):
        do something
        call function_for_provenance(AiiDA_datatypes)
    
    @calcfunction
    def function_for_provenance(AiiDA_datatypes):
        do stuff  

Caching does not seem to work in these cases and will throw errors of the type:

File "/aiida-core/aiida/orm/nodes/process/process.py", line 485, in is_valid_cache
    process_class = self.process_class
  File "/aiida-core/aiida/orm/nodes/process/process.py", line 126, in process_class
    process_class = getattr(module, class_name)
AttributeError: module '....XY' has no attribute 'function_for_provenance' 

A solution is of course to put calcfunctions out of classes and make them top level and importable, which should be always possible I guess. If this enforcement is wanted, this should be documented in the developer guide and caching sections.

I have not thought about if is_valid_cache implementation can be changed to allow for this, do we even want to? @greschd what do you think?

broeder-j avatar Mar 02 '20 20:03 broeder-j