datatrove
                                
                                 datatrove copied to clipboard
                                
                                    datatrove copied to clipboard
                            
                            
                            
                        [BUG Fix] Launching dependent `LocalPipelineExecutor`s with `skip_completed=False` lead to waiting
When launching dependent LocalPipelineExecutor, using the flag skip_completed=False in previous executor will lead to the following exector wait forever.
For example:
executor1 = LocalPipelineExecutor(
    pipeline=[
            ...
        ],
    tasks=10,
    logging_dir=f"logs/tokz",
    skip_completed=False
)
executor2 = LocalPipelineExecutor(
    pipeline=[
            ...
        ],
    tasks=10,
    logging_dir=f"logs/tokz",
)
if __name__ == "__main__":
    executor2.run()
The above code snippet will lead to
datatrove.executor.local:run:102 - Dependency job still has 10/10 tasks. Waiting...
even if executor1 has finished all its jobs.