Crash when reusing logging Handlers with unhashable type: 'types.SimpleNamespace'

[aviadfiant]

Hello, First of all thank you for all the effort you've put in to this project so far.

I am running into an issue I can't seem to resolve and would love it if you could give your opinion.

I am trying to use both alive-progress and sshtunnel together. All I've done is create the sshtunnel definition before the alive example code as below, and I am getting the error from the headline :\

from alive_progress import alive_bar
import time
from sshtunnel import SSHTunnelForwarder

ssh_tun = SSHTunnelForwarder(
    "0.0.0.0",
    ssh_username="mooo",
    remote_bind_address=('127.0.0.1', 8080)
)

for x in 1000, 1500, 700, 0:
    with alive_bar(x) as bar:
        for i in range(1000):
            time.sleep(.005)
            bar()

Thoughts? Thank you in advance for any reply.

[TheTechRobo]

Is this related to #107?

[rsalmei]

Hello @aviadfiant, thank you, man!

Well, it does seem similar to that one @TheTechRobo has reminded us of, but it is different. This SSHTunnelForwarder uses logging internally, but in a kinda unusual way: it reuses handlers...

Note the highlighted ids, they are the addresses of some object, all pointing to the same Handler 107c648c8. So, when alive-progress processes them, it tries to instrument again an already ready handler, and thus it breaks like in a nested use case...

I'll support this case in a new version 👍

[aviadfiant]

Thank you so much. Let me know if I can help with testing :)

[Zhaopudark]

Same with me. Conflict with logging when I set individual loggers. unhashable type: 'types.SimpleNamespace

[rsalmei]

Are you also reusing handlers?

[Zhaopudark]

@rsalmei Thanks you very much! I really like alive-progress indeed! But the bug is confusing me. I'm using RotatingFileHandler. Here are my simplified codes:

import time
import os
import logging
import logging.handlers
import tempfile

from alive_progress import alive_bar

with tempfile.TemporaryDirectory() as temp_dir:
    logging.basicConfig()
    logger = logging.getLogger("main_logger")
    logger.setLevel(logging.DEBUG)
    file_handler = logging.handlers.RotatingFileHandler(os.path.normpath(f"{temp_dir}//temp_log.log"),maxBytes=1*1024*1024, backupCount=3)
    file_handler.setLevel(logging.DEBUG)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s',datefmt="%Y/%m/%d %I:%M:%S")
    logger.addHandler(file_handler)
    with alive_bar(100) as bar:
        for item in range(100):
            logger.info("Bug info")
            bar()
            time.sleep(1)

with the bug AttributeError: 'types.SimpleNamespace' object has no attribute 'seek'

[emanuelen5]

Hi,

I get the same issue when I try to configure a logger for my own code. Here is a MRE:

import logging.config

from alive_progress import alive_bar

logging_dictconfig = {
    'version': 1, 
    'formatters': {
        'simple': {'format': '%(asctime)s - %(name)s - %(levelname)s - %(message)s'}
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler', 'level': 'DEBUG', 'formatter': 'simple', 'stream': 'ext://sys.stdout'
    }},
    'loggers': {
        'a': {
            'level': 'DEBUG', 'handlers': ['console']
        },
        # Remove either logger, e.g. 'b' and it works fine...
        'b': {
            'level': 'DEBUG', 'handlers': ['console']
        }
    }
}

logging.config.dictConfig(logging_dictconfig)

total = 20
with alive_bar(total) as bar:
    for _ in range(total):
        bar()

and it produces the following error:

$ python alive-test.py  --log-config .logging_config.yaml 
Traceback (most recent call last):
  File "alive-test.py", line 30, in <module>
|████████████████████████████████████████| ▁▃▅ 20/20 [100%] in 0s (105956.3/s, eta: 0s)     bar()
  File "/usr/local/lib/python3.9/contextlib.py", line 124, in __exit__
    next(self.gen)
  File "alive_progress/core/progress.py", line 284, in __alive_bar
    stop_monitoring()
  File "alive_progress/core/progress.py", line 170, in stop_monitoring
    hook_manager.uninstall()
  File "alive_progress/core/hook_manager.py", line 85, in uninstall
    [_set_stream(handler, original_stream)
  File "alive_progress/core/hook_manager.py", line 85, in <listcomp>
    [_set_stream(handler, original_stream)
  File "alive_progress/core/hook_manager.py", line 134, in _set_stream
    return handler.setStream(stream)
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1103, in setStream
    self.flush()
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1063, in flush
    self.stream.flush()
  File "alive_progress/core/hook_manager.py", line 33, in flush
    if buffers[stream]:
TypeError: unhashable type: 'types.SimpleNamespace'
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/logging/__init__.py", line 2129, in shutdown
    h.flush()
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1063, in flush
    self.stream.flush()
  File "alive_progress/core/hook_manager.py", line 33, in flush
    if buffers[stream]:
TypeError: unhashable type: 'types.SimpleNamespace'

The interesting thing is that it works if I remove one of the loggers... It does not matter which one.

Environment

• alive-progress version: 2.4.1
• Python version: 3.9.1

[rsalmei]

Hey @emanuelen5, thanks for the detailed report!!

Yes, it is the exact same problem, it is a code that reuses a handler... Notice both your loggers reuse the handler named console, defined above:

        'a': {'level': 'DEBUG', 'handlers': ['console']}
        'b': {'level': 'DEBUG', 'handlers': ['console']}

I need to somehow detect I've already instrumented it, so I do not try it again. I was going to fix this together with another issue regarding seek support, which is way more difficult, but I'll try to look into this one separately. It seems way simpler. As soon as I get some free time I'll take a look and fix this 👍

[rsalmei]

Hey @Zhaopudark, thanks man! Appreciate the words!

Yeah, loggers that use seek are not supported yet, the issue for that is #104 It's on my list too.

[rsalmei]

Thanks for the example @emanuelen5, I'll use it to fix this!

[emanuelen5]

@rsalmei No problem. Thanks for the awesome work 😊

[rsalmei]

Thanks, man 👍

[rsalmei]

Hey, this is fixed!! Should be available in the next release! 👍

[Crinibus]

Hi @rsalmei, is there an update to when the next release is with the fix? I experience a similar issue.

Have a great day :)

[rsalmei]

Hi folks!

I'd just like to let you know I'm committing it at this exact moment! Should be released soon!! 👍

Thanks, @Crinibus, you have a great week!! 👋