Tasks failing in complete() or requires() have unknown state and last forever.

[dmsfabiano]

Issue is very self explanatory, and discussed thoroughly in 2070. However, there never was any kind of feedback reported.

If a tasks raises an exception, during the requires() or complete(), the task is marked as unknown and it hangs forever. If this is done by design, could someone helps us on elaborating on how to handle this properly? It really is not documented anywhere

Here is a quick example on my situation:

class CollectionGenerateWrapper(CollectionGenerateTask):
     
    def requires(self):
          if metadata['generator_type'] == 'img':
               return self.clone(OCRWrapper)
          return []

class OCRWrapper(OCRTask):

   def required(self):
      return []

In this case, due to metadata creation failing on transient errors, it may not exists, in which case, requires throws an exception. In which case CollectionGenerateWrapper is marked as unknown, and it's parent's task/workflow its stuck pending forever.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions.