Status Manager API invalidate not working

[cschloer]

In order to submit an issue, please ensure you can check the following. Thanks!

• [ 3.7.5] Declare which version of Python you are using (python --version)
• [ ubuntu ] Declare which operating system you are using

This is somewhat deep in the weeds, but there is a bug in the status API. Pretty much all calls to the "deregister" function (https://github.com/frictionlessdata/datapackage-pipelines/blob/master/datapackage_pipelines/status/status_manager.py#L41) including the initialize function (https://github.com/frictionlessdata/datapackage-pipelines/blob/master/datapackage_pipelines/status/status_manager.py#L29) fail because the wrong pipeline_id is being used to invalidate each execution.

The deregister function in pipeline_status (https://github.com/frictionlessdata/datapackage-pipelines/blob/master/datapackage_pipelines/status/pipeline_status.py#L165) calls deregister with pipeline_id. It should infact be f'PipelineStatus:{pipeline_id}'. The same is true in pipeline_execution (https://github.com/frictionlessdata/datapackage-pipelines/blob/master/datapackage_pipelines/status/pipeline_execution.py#L104) where it should be f'PipelineExecution:{pipeline_id}'. I don't know why these don't work, I'm assuming the API got changed a while back and never properly updated the other functions. This is rather low priority as it's easy to work around by calling the functions myself.

[akariv]

@cschloer are you using the filesystem backend or the redis one? There indeed seems to be some inconsistency there.

[cschloer]

the filesystem backend I believe. I haven't messed with any the parameters related to which backend you use.