ImportError: cannot import name 'StatsdStatsLogger' from 'superset.stats_logger' (/app/superset/stats_logger.py)

[anilkumar000]

Hello, im trying with Statsd logging feature of superset using the following references but nothing worked out. Even references from GitHub issues also didnt help. Could anyone help to identify and fix the issue. Superset version - v2.0 Superset instance - On Kubernetes container using Helm Configuration added in superset_config.py -

from superset.stats_logger import StatsdStatsLogger
STATS_LOGGER = StatsdStatsLogger(host='<host-IP/name>', port=9125)

Error Logs :

Installing collected packages: statsd, python-statsd, cx_Oracle, psycopg2, elasticsearch, trino, pydruid, elasticsearch-dbapi, authlib 8/12/2022 11:12:55 AM Attempting uninstall: pydruid 8/12/2022 11:12:55 AM Found existing installation: pydruid 0.6.2 8/12/2022 11:12:55 AM Uninstalling pydruid-0.6.2: 8/12/2022 11:12:55 AM Successfully uninstalled pydruid-0.6.2 8/12/2022 11:12:55 AM Successfully installed authlib-1.0.1 cx_Oracle-8.3.0 elasticsearch-7.13.4 elasticsearch-dbapi-0.2.9 psycopg2-2.8.5 pydruid-0.6.3 python-statsd-2.1.0 statsd-3.3.0 trino-0.315.0 8/12/2022 11:12:55 AM WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv 8/12/2022 11:12:56 AM WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available. 8/12/2022 11:12:56 AM You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command. 8/12/2022 11:12:56 AM [2022-08-12 05:42:56 +0000] [147] [INFO] Starting gunicorn 20.1.0 8/12/2022 11:12:56 AM [2022-08-12 05:42:56 +0000] [147] [INFO] Listening at: http://0.0.0.0:8088/ (147) 8/12/2022 11:12:56 AM [2022-08-12 05:42:56 +0000] [147] [INFO] Using worker: gthread 8/12/2022 11:12:56 AM [2022-08-12 05:42:56 +0000] [149] [INFO] Booting worker with pid: 149 8/12/2022 11:12:59 AM Found but failed to import local superset_config 8/12/2022 11:12:59 AM Traceback (most recent call last): 8/12/2022 11:12:59 AM File "/app/superset/config.py", line 1336, in 8/12/2022 11:12:59 AM import superset_config 8/12/2022 11:12:59 AM File "/app/pythonpath/superset_config.py", line 5, in 8/12/2022 11:12:59 AM from superset.stats_logger import StatsdStatsLogger 8/12/2022 11:12:59 AM ImportError: cannot import name 'StatsdStatsLogger' from 'superset.stats_logger' (/app/superset/stats_logger.py) 8/12/2022 11:12:59 AM Found but failed to import local superset_config 8/12/2022 11:12:59 AM Traceback (most recent call last): 8/12/2022 11:12:59 AM File "/app/superset/config.py", line 1336, in 8/12/2022 11:12:59 AM import superset_config 8/12/2022 11:12:59 AM File "/app/pythonpath/superset_config.py", line 5, in 8/12/2022 11:12:59 AM from superset.stats_logger import StatsdStatsLogger 8/12/2022 11:12:59 AM ImportError: cannot import name 'StatsdStatsLogger' from 'superset.stats_logger' (/app/superset/stats_logger.py) 8/12/2022 11:12:59 AM Found but failed to import local superset_config 8/12/2022 11:12:59 AM Traceback (most recent call last): 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/werkzeug/utils.py", line 879, in import_string 8/12/2022 11:12:59 AM module = import(module_name, globals(), locals(), [obj_name]) 8/12/2022 11:12:59 AM File "/app/superset/config.py", line 1336, in 8/12/2022 11:12:59 AM import superset_config 8/12/2022 11:12:59 AM File "/app/pythonpath/superset_config.py", line 5, in 8/12/2022 11:12:59 AM from superset.stats_logger import StatsdStatsLogger 8/12/2022 11:12:59 AM ImportError: cannot import name 'StatsdStatsLogger' from 'superset.stats_logger' (/app/superset/stats_logger.py) 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM During handling of the above exception, another exception occurred: 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM Traceback (most recent call last): 8/12/2022 11:12:59 AM File "/app/superset/config.py", line 1336, in 8/12/2022 11:12:59 AM import superset_config 8/12/2022 11:12:59 AM File "/app/pythonpath/superset_config.py", line 5, in 8/12/2022 11:12:59 AM from superset.stats_logger import StatsdStatsLogger 8/12/2022 11:12:59 AM ImportError: cannot import name 'StatsdStatsLogger' from 'superset.stats_logger' (/app/superset/stats_logger.py) 8/12/2022 11:12:59 AM Found but failed to import local superset_config 8/12/2022 11:12:59 AM Traceback (most recent call last): 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/werkzeug/utils.py", line 879, in import_string 8/12/2022 11:12:59 AM module = import(module_name, globals(), locals(), [obj_name]) 8/12/2022 11:12:59 AM File "/app/superset/config.py", line 1336, in 8/12/2022 11:12:59 AM import superset_config 8/12/2022 11:12:59 AM File "/app/pythonpath/superset_config.py", line 5, in 8/12/2022 11:12:59 AM from superset.stats_logger import StatsdStatsLogger 8/12/2022 11:12:59 AM ImportError: cannot import name 'StatsdStatsLogger' from 'superset.stats_logger' (/app/superset/stats_logger.py) 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM During handling of the above exception, another exception occurred: 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM Traceback (most recent call last): 8/12/2022 11:12:59 AM File "/app/superset/config.py", line 1336, in 8/12/2022 11:12:59 AM import superset_config 8/12/2022 11:12:59 AM File "/app/pythonpath/superset_config.py", line 5, in 8/12/2022 11:12:59 AM from superset.stats_logger import StatsdStatsLogger 8/12/2022 11:12:59 AM ImportError: cannot import name 'StatsdStatsLogger' from 'superset.stats_logger' (/app/superset/stats_logger.py) 8/12/2022 11:12:59 AM Failed to create app 8/12/2022 11:12:59 AM Traceback (most recent call last): 8/12/2022 11:12:59 AM File "/app/superset/app.py", line 34, in create_app 8/12/2022 11:12:59 AM app.config.from_object(config_module) 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/flask/config.py", line 162, in from_object 8/12/2022 11:12:59 AM obj = import_string(obj) 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/werkzeug/utils.py", line 887, in import_string 8/12/2022 11:12:59 AM raise ImportStringError(import_name, e).with_traceback( 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/werkzeug/utils.py", line 879, in import_string 8/12/2022 11:12:59 AM module = import(module_name, globals(), locals(), [obj_name]) 8/12/2022 11:12:59 AM File "/app/superset/config.py", line 1336, in 8/12/2022 11:12:59 AM import superset_config 8/12/2022 11:12:59 AM File "/app/pythonpath/superset_config.py", line 5, in 8/12/2022 11:12:59 AM from superset.stats_logger import StatsdStatsLogger 8/12/2022 11:12:59 AM werkzeug.utils.ImportStringError: import_string() failed for 'superset.config'. Possible reasons are: 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM - missing init.py in a package; 8/12/2022 11:12:59 AM - package or module path not included in sys.path; 8/12/2022 11:12:59 AM - duplicated package or module name taking precedence in sys.path; 8/12/2022 11:12:59 AM - missing module, class, function or variable; 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM Debugged import: 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM - 'superset' found in '/app/superset/init.py'. 8/12/2022 11:12:59 AM - 'superset.config' not found. 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM Original exception: 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM ImportError: cannot import name 'StatsdStatsLogger' from 'superset.stats_logger' (/app/superset/stats_logger.py) 8/12/2022 11:12:59 AM [2022-08-12 05:42:59 +0000] [149] [ERROR] Exception in worker process 8/12/2022 11:12:59 AM Traceback (most recent call last): 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker 8/12/2022 11:12:59 AM worker.init_process() 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/gthread.py", line 92, in init_process 8/12/2022 11:12:59 AM super().init_process() 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process 8/12/2022 11:12:59 AM self.load_wsgi() 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi 8/12/2022 11:12:59 AM self.wsgi = self.app.wsgi() 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi 8/12/2022 11:12:59 AM self.callable = self.load() 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load 8/12/2022 11:12:59 AM return self.load_wsgiapp() 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp 8/12/2022 11:12:59 AM return util.import_app(self.app_uri) 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 412, in import_app 8/12/2022 11:12:59 AM app = app(*args, **kwargs) 8/12/2022 11:12:59 AM File "/app/superset/app.py", line 44, in create_app 8/12/2022 11:12:59 AM raise ex 8/12/2022 11:12:59 AM File "/app/superset/app.py", line 34, in create_app 8/12/2022 11:12:59 AM app.config.from_object(config_module) 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/flask/config.py", line 162, in from_object 8/12/2022 11:12:59 AM obj = import_string(obj) 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/werkzeug/utils.py", line 887, in import_string 8/12/2022 11:12:59 AM raise ImportStringError(import_name, e).with_traceback( 8/12/2022 11:12:59 AM File "/usr/local/lib/python3.8/site-packages/werkzeug/utils.py", line 879, in import_string 8/12/2022 11:12:59 AM module = import(module_name, globals(), locals(), [obj_name]) 8/12/2022 11:12:59 AM File "/app/superset/config.py", line 1336, in 8/12/2022 11:12:59 AM import superset_config 8/12/2022 11:12:59 AM File "/app/pythonpath/superset_config.py", line 5, in 8/12/2022 11:12:59 AM from superset.stats_logger import StatsdStatsLogger 8/12/2022 11:12:59 AM werkzeug.utils.ImportStringError: import_string() failed for 'superset.config'. Possible reasons are: 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM - missing init.py in a package; 8/12/2022 11:12:59 AM - package or module path not included in sys.path; 8/12/2022 11:12:59 AM - duplicated package or module name taking precedence in sys.path; 8/12/2022 11:12:59 AM - missing module, class, function or variable; 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM Debugged import: 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM - 'superset' found in '/app/superset/init.py'. 8/12/2022 11:12:59 AM - 'superset.config' not found. 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM Original exception: 8/12/2022 11:12:59 AM 8/12/2022 11:12:59 AM ImportError: cannot import name 'StatsdStatsLogger' from 'superset.stats_logger' (/app/superset/stats_logger.py) 8/12/2022 11:12:59 AM [2022-08-12 05:42:59 +0000] [149] [INFO] Worker exiting (pid: 149) 8/12/2022 11:12:59 AM [2022-08-12 05:42:59 +0000] [147] [INFO] Shutting down: Master 8/12/2022 11:12:59 AM [2022-08-12 05:42:59 +0000] [147] [INFO] Reason: Worker failed to boot.

Expected : Metrics should be exposed to provided statsd host References : https://superset.apache.org/docs/installation/event-logging/#statsd-logging https://apache-superset.readthedocs.io/en/latest/installation.html#statsd-logging https://github.com/apache/superset/issues/8285 https://github.com/apache/superset/issues/9420

[RahulRaj97]

Hi @anilkumar000 . Can you try adding the python package for statsd in install_requires in setup.py as "statsd>=3.3.0" ? It seems to be working for me. Will create a PR if you confirm so anyone can easily use StatsD metrics. Thanks

[anilkumar000]

Hi @RahulRaj97 i have added statsd==3.3.0 python-statsd==2.1.0 these two packages but didnt work out. The First line in Error logs of first comment states the same. Installing collected packages: statsd, python-statsd, cx_Oracle, psycopg2, elasticsearch, trino, pydruid, elasticsearch-dbapi, authlib Could please share your configurations, Thanks for response

[rusackas]

Anyone still interested in tackling this, or is it no longer an issue? Tempted to close as stale, but it sounds like it might nust need revisiting.

[rusackas]

Ok, closing as stale. Ping me if this needs a revisit!

[SamraHanifCareem]

Setting statsd >= 3.3.0 in setup.py worked fine for me as well