apache
Failed to setup alerts and reports based pypi installation
Bug description
I have installed superset 4.0.1 with pypi way, seems no redis service activated while start the superset service with superset run -p 8088 --with-threads --reload --debugger -h 0.0.0.0.Currently i failed to setup alerts and reports following the document from the https://superset.apache.org/docs/configuration/alerts-reports
Please kindly help.
How to reproduce the bug
Following is my config.
FEATURE_FLAGS = {
'ALERT_REPORTS' : True,
'DASHBOARD_RBAC': True,
'ENABLE_TEMPLATE_PROCESSING': True,
}
from celery.schedules import crontab
class CeleryConfig:
beat_schedule = {
"reports.scheduler": {
"task": "reports.scheduler",
"schedule": crontab(minute="*", hour="*"),
},
"reports.prune_log": {
"task": "reports.prune_log",
"schedule": crontab(minute=0, hour=0),
},
}
CELERY_CONFIG = CeleryConfig
SCREENSHOT_LOCATE_WAIT = 100
SCREENSHOT_LOAD_WAIT = 600
ALERT_REPORTS = True
EMAIL_NOTIFICATIONS = True
ALERT_REPORTS_NOTIFICATION_DRY_RUN = False
EMAIL_NOTIFICATIONS = False # all the emails are sent using dryrun
SMTP_HOST = "SMTP_HOST"
SMTP_STARTTLS = True
SMTP_SSL = True
SMTP_USER = "SMTP_USER"
SMTP_PORT = 465
SMTP_PASSWORD = "SMTP_PASSWORD"
SMTP_MAIL_FROM = "SMTP_MAIL_FROM"
SMTP_SSL_SERVER_AUTH = False
#ENABLE_CHUNK_ENCODING = False
# WebDriver configuration
# If you use Firefox, you can stick with default values
# If you use Chrome, then add the following WEBDRIVER_TYPE and WEBDRIVER_OPTION_ARGS
WEBDRIVER_TYPE = "chrome"
WEBDRIVER_OPTION_ARGS = [
"--force-device-scale-factor=2.0",
"--high-dpi-support=2.0",
"--headless",
"--disable-gpu",
"--disable-dev-shm-usage",
"--no-sandbox",
"--disable-setuid-sandbox",
"--disable-extensions",
]
# This is for internal use, you can keep http
WEBDRIVER_BASEURL = "http://localhost:8088"
# This is the link sent to the recipient. Change to your domain, e.g. https://superset.mydomain.com
WEBDRIVER_BASEURL_USER_FRIENDLY = "http://IP:8088"
from superset.tasks.types import ExecutorType
THUMBNAIL_SELENIUM_USER = 'admin'
ALERT_REPORTS_EXECUTE_AS = [ExecutorType.SELENIUM]
pip list install
Package Version
------------------------- -----------
alembic 1.13.1
amqp 5.2.0
apache-superset 4.0.1
apispec 6.6.1
apsw 3.46.0.0
async-timeout 4.0.3
attrs 23.2.0
Babel 2.15.0
backoff 2.2.1
bcrypt 4.1.3
billiard 4.2.0
blinker 1.8.2
Bottleneck 1.3.8
Brotli 1.1.0
cachelib 0.9.0
cachetools 5.3.3
cattrs 23.2.3
celery 5.4.0
certifi 2024.6.2
cffi 1.16.0
charset-normalizer 3.3.2
click 8.1.7
click-didyoumean 0.3.1
click-option-group 0.5.6
click-plugins 1.1.1
click-repl 0.3.0
colorama 0.4.6
cron-descriptor 1.4.3
croniter 2.0.5
cryptography 42.0.4
Deprecated 1.2.14
deprecation 2.1.0
dnspython 2.6.1
email_validator 2.1.1
exceptiongroup 1.2.1
Flask 2.3.3
Flask-AppBuilder 4.5.0
Flask-Babel 2.0.0
Flask-Caching 2.3.0
Flask-Compress 1.15
Flask-JWT-Extended 4.6.0
Flask-Limiter 3.7.0
Flask-Login 0.6.3
Flask-Migrate 3.1.0
Flask-Session 0.8.0
Flask-SQLAlchemy 2.5.1
flask-talisman 1.1.0
Flask-WTF 1.2.1
func-timeout 4.3.5
geographiclib 2.0
geopy 2.4.1
google-auth 2.30.0
greenlet 3.0.3
gunicorn 21.2.0
h11 0.14.0
hashids 1.3.1
holidays 0.25
humanize 4.9.0
idna 3.7
importlib_metadata 7.1.0
importlib_resources 6.4.0
isodate 0.6.1
itsdangerous 2.2.0
Jinja2 3.1.4
jsonschema 4.22.0
jsonschema-specifications 2023.12.1
kombu 5.3.7
korean-lunar-calendar 0.3.1
limits 3.12.0
llvmlite 0.42.0
Mako 1.3.5
Markdown 3.6
markdown-it-py 3.0.0
MarkupSafe 2.1.5
marshmallow 3.21.3
marshmallow-sqlalchemy 0.28.2
mdurl 0.1.2
msgpack 1.0.8
msgspec 0.18.6
mysqlclient 2.2.4
nh3 0.2.17
numba 0.59.1
numexpr 2.10.0
numpy 1.23.5
ordered-set 4.1.0
outcome 1.3.0.post0
packaging 24.1
pandas 2.0.3
paramiko 3.4.0
parsedatetime 2.6
pgsanity 0.2.9
pillow 10.3.0
pip 22.0.2
platformdirs 4.2.2
polyline 2.0.2
prison 0.2.1
prompt_toolkit 3.0.47
pyarrow 14.0.2
pyasn1 0.6.0
pyasn1_modules 0.4.0
pycparser 2.22
pydoris 1.0.5.3
Pygments 2.18.0
PyJWT 2.8.0
PyNaCl 1.5.0
pyOpenSSL 24.1.0
pyparsing 3.1.2
PySocks 1.7.1
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
python-geohash 0.8.5
python-ldap 3.4.4
pytz 2024.1
PyYAML 6.0.1
redis 4.6.0
referencing 0.35.1
requests 2.32.3
requests-cache 1.2.0
rich 13.7.1
rpds-py 0.18.1
rsa 4.9
selenium 4.9.1
setuptools 59.6.0
shillelagh 1.2.19
shortid 0.1.2
simplejson 3.19.2
six 1.16.0
slack_sdk 3.28.0
sniffio 1.3.1
sortedcontainers 2.4.0
SQLAlchemy 1.4.52
SQLAlchemy-Utils 0.38.3
sqlglot 23.17.0
sqlparse 0.4.4
sshtunnel 0.4.0
tabulate 0.8.10
trio 0.25.1
trio-websocket 0.11.1
typing_extensions 4.12.2
tzdata 2024.1
url-normalize 1.4.3
urllib3 2.2.1
vine 5.1.0
wcwidth 0.2.13
Werkzeug 3.0.3
wrapt 1.16.0
wsproto 1.2.0
WTForms 3.1.2
WTForms-JSON 0.3.5
XlsxWriter 3.0.9
zipp 3.19.2
zstandard 0.22.0
Screenshots/recordings
No response
Superset version
4.0.1
Python version
3.9
Node version
16
Browser
Chrome
Additional context
No response
Checklist
- [X] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [X] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
To resolve the issue with setting up alerts and reports in Apache Superset, you need to ensure that the Redis service is properly configured and activated. Based on your provided configuration, it appears that the broker_url and result_backend for Celery are not set, which are essential for connecting to the Redis service.
Here is the updated configuration with the necessary Redis settings:
FEATURE_FLAGS = {
'ALERT_REPORTS' : True,
'DASHBOARD_RBAC': True,
'ENABLE_TEMPLATE_PROCESSING': True,
}
from celery.schedules import crontab
REDIS_HOST = "localhost" # Change this to your Redis host
REDIS_PORT = "6379" # Change this to your Redis port if different
REDIS_CELERY_DB = "0"
REDIS_RESULTS_DB = "1"
class CeleryConfig:
broker_url = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}"
result_backend = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}"
beat_schedule = {
"reports.scheduler": {
"task": "reports.scheduler",
"schedule": crontab(minute="*", hour="*"),
},
"reports.prune_log": {
"task": "reports.prune_log",
"schedule": crontab(minute=0, hour=0),
},
}
CELERY_CONFIG = CeleryConfig
SCREENSHOT_LOCATE_WAIT = 100
SCREENSHOT_LOAD_WAIT = 600
ALERT_REPORTS = True
EMAIL_NOTIFICATIONS = True
ALERT_REPORTS_NOTIFICATION_DRY_RUN = False
EMAIL_NOTIFICATIONS = False # all the emails are sent using dryrun
SMTP_HOST = "SMTP_HOST"
SMTP_STARTTLS = True
SMTP_SSL = True
SMTP_USER = "SMTP_USER"
SMTP_PORT = 465
SMTP_PASSWORD = "SMTP_PASSWORD"
SMTP_MAIL_FROM = "SMTP_MAIL_FROM"
SMTP_SSL_SERVER_AUTH = False
WEBDRIVER_TYPE = "chrome"
WEBDRIVER_OPTION_ARGS = [
"--force-device-scale-factor=2.0",
"--high-dpi-support=2.0",
"--headless",
"--disable-gpu",
"--disable-dev-shm-usage",
"--no-sandbox",
"--disable-setuid-sandbox",
"--disable-extensions",
]
WEBDRIVER_BASEURL = "http://localhost:8088"
WEBDRIVER_BASEURL_USER_FRIENDLY = "http://IP:8088"
from superset.tasks.types import ExecutorType
THUMBNAIL_SELENIUM_USER = 'admin'
ALERT_REPORTS_EXECUTE_AS = [ExecutorType.SELENIUM]
Make sure that the Redis service is running and accessible at the specified REDIS_HOST and REDIS_PORT. This should enable the Celery workers to connect to Redis and process the tasks for alerts and reports. Additionally, ensure that the REDIS_CELERY_DB and REDIS_RESULTS_DB are correctly set to match your Redis configuration [1][2].
To continue talking to Dosu, mention @dosu.
![dosubot[bot] avatar](https://avatars.githubusercontent.com/in/324583?v=4 "Published by a pub.dev verified publisher"){kind=small}
I installed the redis manually, and update the config as following, but NO LUCK
from celery.schedules import crontab
REDIS_HOST = "localhost" # Change this to your Redis host
REDIS_PORT = "6379" # Change this to your Redis port if different
REDIS_CELERY_DB = "0"
REDIS_RESULTS_DB = "1"
class CeleryConfig:
broker_url = f"redis://{REDIS_HOST}:{REDIS_PORT}/0"
imports = (
"superset.sql_lab",
"superset.tasks.scheduler",
)
result_backend = f"redis://{REDIS_HOST}:{REDIS_PORT}/0"
worker_prefetch_multiplier = 10
task_acks_late = True
task_annotations = {
"sql_lab.get_sql_results": {
"rate_limit": "100/s",
},
}
beat_schedule = {
"reports.scheduler": {
"task": "reports.scheduler",
"schedule": crontab(minute="*", hour="*"),
},
"reports.prune_log": {
"task": "reports.prune_log",
"schedule": crontab(minute=0, hour=0),
},
}
In report log, seems no error message, but no email was sent or received.
It looks like you have correctly configured the Redis settings for Celery. However, there are a few additional configurations and checks you should ensure are in place:
-
Enable the
ALERT_REPORTSFeature Flag: Ensure that theALERT_REPORTSfeature flag is enabled in your configuration:FEATURE_FLAGS = { 'ALERT_REPORTS': True, 'DASHBOARD_RBAC': True, 'ENABLE_TEMPLATE_PROCESSING': True, } -
Set
ALERT_REPORTS_NOTIFICATION_DRY_RUNtoFalse: Make sure this setting is correctly set toFalseto enable actual notifications:ALERT_REPORTS_NOTIFICATION_DRY_RUN = False -
Configure the WebDriver: Ensure that the WebDriver is correctly configured for taking screenshots:
WEBDRIVER_TYPE = "chrome" WEBDRIVER_OPTION_ARGS = [ "--force-device-scale-factor=2.0", "--high-dpi-support=2.0", "--headless", "--disable-gpu", "--disable-dev-shm-usage", "--no-sandbox", "--disable-setuid-sandbox", "--disable-extensions", ] WEBDRIVER_BASEURL = "http://localhost:8088" WEBDRIVER_BASEURL_USER_FRIENDLY = "http://IP:8088" -
Set the User for Rendering Dashboards: Ensure that the user for rendering dashboards is set:
from superset.tasks.types import ExecutorType THUMBNAIL_SELENIUM_USER = 'admin' ALERT_REPORTS_EXECUTE_AS = [ExecutorType.SELENIUM] -
Check Celery Workers: Ensure that Celery workers are running and can connect to Redis. You can start the Celery worker with:
celery -A superset.tasks.celery_app:app worker --loglevel=info -
Check Celery Beat: Ensure that Celery beat is running to schedule tasks:
celery -A superset.tasks.celery_app:app beat --loglevel=info -
Verify Redis Connection: Ensure that Redis is running and accessible. You can test the connection using a Redis client:
redis-cli -h localhost -p 6379 ping -
SMTP Configuration: Ensure that your SMTP settings are correctly configured for email notifications:
SMTP_HOST = "SMTP_HOST" SMTP_STARTTLS = True SMTP_SSL = True SMTP_USER = "SMTP_USER" SMTP_PORT = 465 SMTP_PASSWORD = "SMTP_PASSWORD" SMTP_MAIL_FROM = "SMTP_MAIL_FROM" SMTP_SSL_SERVER_AUTH = False
By ensuring these configurations and checks are in place, you should be able to set up alerts and reports in Apache Superset 4.0.1. If the issue persists, check the Superset logs for any errors or warnings that might provide more insights [1][2].
To continue talking to Dosu, mention @dosu.
after config as above, the log report STARTTLS extension not supported by server.
If you followed the PyPi installation, have you set up a Celery instance?
YES, celery is running
Currently, the log shows failed as STARTTLS extension not supported by server.
And cerely is configured as
from celery.schedules import crontab
REDIS_HOST = "localhost" # Change this to your Redis host
REDIS_PORT = "6379" # Change this to your Redis port if different
REDIS_CELERY_DB = "0"
REDIS_RESULTS_DB = "1"
class CeleryConfig:
broker_url = f"redis://{REDIS_HOST}:{REDIS_PORT}/0"
imports = (
"superset.sql_lab",
"superset.tasks.scheduler",
)
result_backend = f"redis://{REDIS_HOST}:{REDIS_PORT}/0"
worker_prefetch_multiplier = 10
task_acks_late = True
task_annotations = {
"sql_lab.get_sql_results": {
"rate_limit": "100/s",
},
}
beat_schedule = {
"reports.scheduler": {
"task": "reports.scheduler",
"schedule": crontab(minute="*", hour="*"),
},
"reports.prune_log": {
"task": "reports.prune_log",
"schedule": crontab(minute=0, hour=0),
},
}
CELERY_CONFIG = CeleryConfig
from flask_caching.backends.rediscache import RedisCache
RESULTS_BACKEND = RedisCache(
host=REDIS_HOST, port=REDIS_PORT, key_prefix='superset_results')
Moving this to a Help discussion since it's not a "bug" per se. It's been quiet for quite a while... are you still facing this issue?
