Neuraxle icon indicating copy to clipboard operation
Neuraxle copied to clipboard

Bug: Steps shouldn't need to be initialised to be saved.

Open vincent-antaki opened this issue 3 years ago • 0 comments

The ExecutionContext's save function uses a method call initialize_if_needed which, in case for a full dump, is called recursively on every step. This functions calls each steps setup function. However, there are cases in some of our projects for which the setup has to be called at a precise moment during the pipeline execution.

I believe we should get rid of the requirement to have steps initialized to be saved. However, this modification also requires the following:

  • We need to be able to distinguish between savers which can and can't be applied when a step isn't initialized (with a mixin?). Alternatively expect every saver to handle the case where the step is not initialized
  • the should_save method should be adapted accordingly.
  • When full_dump=True, we should not initialize steps.

Do you see something I am missing @guillaume-chevalier?

vincent-antaki avatar Apr 04 '21 02:04 vincent-antaki