Not all loggers output are captured in the GridWorkers

[vmarois]

When using a GridWorker, only the output of the logger of the current Worker (e.g. OfflineTrainer or Tester) is captured. The loggers of the Model & Problem are not able to write to the console or to the .log file. This is fairly annoying as these loggers have some important information (e.g. whether the Sampler was properly instantiated, when is saved the Model etc).

Not sure yet how to fix this. I think this is related to the logging.config and also how the output of a child process is captured in the GridWorker:

https://github.com/IBM/mi-prometheus/blob/3d4f868320c8914f37543298c634ab6ba6a1425d/miprometheus/grid_workers/grid_trainer_cpu.py#L316-L317

We should probably change stdout=os.devnull and do something instead like

with open(worker.log, 'w') as devnull: result = subprocess.run(command_str.split(" "), stdout=devnull)