jobflow
jobflow copied to clipboard
BUG: run_locally log=False does work in second run
Description Consider the following code snippet which executes a simple add job:
from jobflow import job, run_locally
@job
def add(a, b):
""" Add job. """
return a + b
first_job = add(1, 1)
print(run_locally(first_job))
print("\nNew job:\n")
second_job = add(2, 2)
print(run_locally(second_job, log=False))
For whatever reason you want to execute two jobs directly after one another. This happens for example also in jupyter notebooks. Jupyter does not reset the python environment between cells. This will produce the following output:
2023-11-15 15:57:44,643 INFO Started executing jobs locally
2023-11-15 15:57:44,743 INFO Starting job - add (0eed5318-058e-421d-8e47-0258ad30953e)
2023-11-15 15:57:44,978 INFO Finished job - add (0eed5318-058e-421d-8e47-0258ad30953e)
2023-11-15 15:57:44,978 INFO Finished executing jobs locally
{'0eed5318-058e-421d-8e47-0258ad30953e': {1: Response(output=2, detour=None, addition=None, replace=None, stored_data=None, stop_children=False, stop_jobflow=False)}}
New job:
2023-11-15 15:57:44,978 INFO Started executing jobs locally
2023-11-15 15:57:44,978 INFO Starting job - add (2f04d58e-0035-433c-b729-ce047282dbcd)
2023-11-15 15:57:45,012 INFO Finished job - add (2f04d58e-0035-433c-b729-ce047282dbcd)
2023-11-15 15:57:45,012 INFO Finished executing jobs locally
{'2f04d58e-0035-433c-b729-ce047282dbcd': {1: Response(output=4, detour=None, addition=None, replace=None, stored_data=None, stop_children=False, stop_jobflow=False)}}
Both jobs are executed and produce log messages on the screen.
Expected behavior
Since I specified log=False
for the second job, there shouldn't be any log messages. Expected output:
2023-11-15 15:57:44,643 INFO Started executing jobs locally
2023-11-15 15:57:44,743 INFO Starting job - add (0eed5318-058e-421d-8e47-0258ad30953e)
2023-11-15 15:57:44,978 INFO Finished job - add (0eed5318-058e-421d-8e47-0258ad30953e)
2023-11-15 15:57:44,978 INFO Finished executing jobs locally
{'0eed5318-058e-421d-8e47-0258ad30953e': {1: Response(output=2, detour=None, addition=None, replace=None, stored_data=None, stop_children=False, stop_jobflow=False)}}
New flow:
{'2f04d58e-0035-433c-b729-ce047282dbcd': {1: Response(output=4, detour=None, addition=None, replace=None, stored_data=None, stop_children=False, stop_jobflow=False)}}
Solution
In managers/local.py
is a call to initialize_logger()
if log
is True
. One would need to call something like logging.getLogger("jobflow").handlers = []
in the else case to reset the logger.
Or one could reset the logger after every run.