Duplicated log lines

[markus-hinsche]

When running train.py, I get duplicated log lines:

(env_p_3)$  python src/models/CNNDepthMap/CNNDepthMap-height/2021q3-depthmap-plaincnn-rgbd/src/train.py
2021-08-02 09:36:56,432 - INFO - Creating temp folder - /Users/markus/Development/cgm/cgm-ml/src/models/CNNDepthMap/CNNDepthMap-height/2021q3-depthmap-plaincnn-rgbd/src/train_util.py: line 14
INFO - Creating temp folder
...

When running toolkit.py, I get duplicated log lines:

(env_p_3)$ python src/common/depthmap_toolkit/toolkit.py src/common/depthmap_toolkit/tests/huawei_p40pro src/common/depthmap_toolkit/tests/huawei_p40pro/camera_calibration.txt
2021-08-02 09:55:35,023 - INFO - height=0.432887m - /Users/markus/Development/cgm/cgm-ml/src/common/depthmap_toolkit/visualisation.py: line 184
INFO:visualisation:height=0.432887m

[jasminziegler]

Endless logging story.. @markus-hinsche I am again not sure what is happening here.. I do not have duplicate log lines. @prajurock Could you please check what's the case for you on your local machine? Do you see duplicate log lines?

[markus-hinsche]

During a training run on the cluster, I also only see the logs once (not duplicated), i.e. so far I only observed the behavior on my local setup.

[prajurock]

I think the problem is with the handle objects. Handler objected get added logger.addHandler(handler) but you didn't delete that handler. try logger.handlers.pop() or logger.removeHandler(streamhandler) may help. I am not sure btw

[jasminziegler]

@prajurock does not see duplicated log lines in his local setup.

Buuuut I have bad news from the vm. While working on the new EDA notebook, I am receiving this:

looks pretty double again.. I will keep researching this behavior

[jasminziegler]

I added logger.handlers.clear() here

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
logger.handlers.clear()
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter('%(asctime)s - %(levelname)s - %(message)s - %(pathname)s: line %(lineno)d'))
logger.addHandler(handler)

and now I am not seeing duplicated logs any longer, see here:

@markus-hinsche @prajurock what do you think about this solution? Might this cause some issues elsewhere?

[markus-hinsche]

I tried adding this to src/common/depthmap_toolkit/toolkit.py and src/common/depthmap_toolkit/visualization.py.

However when running python src/common/depthmap_toolkit/toolkit.py src/common/depthmap_toolkit/tests/huawei_p40pro src/common/depthmap_toolkit/tests/huawei_p40pro/camera_calibration.txt, I still see duplicate logs.

python src/common/depthmap_toolkit/toolkit.py src/common/depthmap_toolkit/tests/huawei_p40pro src/common/depthmap_toolkit/tests/huawei_p40pro/camera_calibration.txt
DEPRECATION WARNING: The system version of Tk is deprecated and may be removed in a future release. Please don't rely on it. Set TK_SILENCE_DEPRECATION=1 to suppress this warning.
/Users/markus/Development/cgm/cgm-ml/src/common/depthmap_toolkit/depthmap.py:502: RuntimeWarning: invalid value encountered in true_divide
  return vectors / length
2021-08-04 11:15:10,768 - INFO - height=0.432887m - /Users/markus/Development/cgm/cgm-ml/src/common/depthmap_toolkit/visualisation.py: line 182
INFO:visualisation:height=0.432887m