elementary icon indicating copy to clipboard operation
elementary copied to clipboard
verified publisher icon elementary-data

EDR CLI send-report filters doesn't work as expected

Open LeoLuo123 opened this issue 11 months ago • 0 comments

Describe the bug

  • EDR CLI send-report --select invocation_id:xxxx doesn't work
  • EDR CLI send-report --select tags:elementary partially works

To Reproduce Steps to reproduce the behavior:

  1. Run dbt test -s tag:elementary to run elementary tagged anomaly detection test ONLY
  2. Go to BQ dbt_invocations table to get the invocation_id by run_job_id. SELECT invocation_id FROM maven-clinic-bi.dbt_metadata.dbt_invocations where job_run_id = "358771750"
  3. Run edr send-report --select invocation_id:65d29673-08a7-43c9-9e6b-0e3817592e63 --slack-token <SLACK_TOKEN> --slack-channel-name <CHANNEL_NAME>

Here is log:


    ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - https://bit.ly/slack-elementary

2025-01-09 08:29:14 — INFO — Running with edr=0.16.1
2025-01-09 08:29:18 — WARNING — You are using incompatible versions between edr (0.16.1) and Elementary's dbt package (0.15.2).
 To fix please update your packages.yml, and run:
dbt deps && dbt run --select elementary

2025-01-09 08:29:19 — INFO — Elementary's database and schema: '"maven-clinic-bi.dbt_metadata"'
2025-01-09 08:29:19 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:33 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_source_freshness_results", "macro_args": {"days_back": 7, "invocations_per_test": 720}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:37 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models", "macro_args": {"exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:41 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_sources", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:43 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_exposures", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:46 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_singular_tests", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:48 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_runs", "macro_args": {"days_back": 7, "exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:31:28 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_dbt_models_test_coverage", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:31:30 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_last_invocation", "macro_args": {"invocation_id": "65d29673-08a7-43c9-9e6b-0e3817592e63"}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:32:00 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_nodes_depends_on_nodes", "macro_args": {"exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:32:03 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocation", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:33:00 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocations_data", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:33:57 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:34:18 — INFO — Sent test results summary to Slack
2025-01-09 08:39:27 — INFO — Sent report to Slack.

Issue 2

  1. I also tried to use --select tag:elementary, it partially work, it can send a summary report to Slack, but threw out exception when generating html file

    ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - https://bit.ly/slack-elementary

2025-01-09 08:46:53 — INFO — Running with edr=0.16.1
2025-01-09 08:46:57 — WARNING — You are using incompatible versions between edr (0.16.1) and Elementary's dbt package (0.15.2).
 To fix please update your packages.yml, and run:
dbt deps && dbt run --select elementary

2025-01-09 08:46:57 — INFO — Elementary's database and schema: '"maven-clinic-bi.dbt_metadata"'
2025-01-09 08:46:57 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:18 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_source_freshness_results", "macro_args": {"days_back": 7, "invocations_per_test": 720}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:20 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models", "macro_args": {"exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:24 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_sources", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:26 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_exposures", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:28 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_singular_tests", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:30 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_runs", "macro_args": {"days_back": 7, "exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:45 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_dbt_models_test_coverage", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:47 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_nodes_depends_on_nodes", "macro_args": {"exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:50 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocation", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:52 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocations_data", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:59 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:48:54 — INFO — Sent test results summary to Slack
Traceback (most recent call last):
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/bin/edr", line 8, in <module>
    sys.exit(cli())
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/cli/cli.py", line 67, in invoke
    return super().invoke(ctx)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/cli.py", line 713, in send_report
    sent_report_successfully = data_monitoring.send_report(
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/data_monitoring/report/data_monitoring_report.py", line 227, in send_report
    self.validate_report_selector()
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/data_monitoring/report/data_monitoring_report.py", line 136, in validate_report_selector
    self.selector_filter.validate_report_selector()
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/data_monitoring/schema.py", line 102, in validate_report_selector
    raise InvalidSelectorError(
elementary.monitor.data_monitoring.schema.InvalidSelectorError: ('Selector is invalid for report: ', 'tag:elementary')

Expected behavior

  • When option --select invocation_id:xxx is passed, cli will only generate and send report for this specific dbt invocation
  • When option --select tags:elementary, cli will send message to Slack channel and also includes html report as attachment

Screenshots Issue 1: I only have one anomaly detection test tagged with elementary, but the summary is for all dbt tests image

Issue2: In the screenshot, you can see filter tag:elementary is applied image

But there is no attachment since exception is thrown.

Environment (please complete the following information):

  • Elementary CLI (edr) version: [0.16.1], can be found by running pip show elementary-data
  • Elementary dbt package version: [0.15.2], can be found in packages.yml file
  • dbt version you're using [1.8.7]
  • Data warehouse [BigQuery]
  • Infrastructure details (dev)

Additional context Add any other context about the problem here.

Would you be willing to contribute a fix for this issue? I'd love to give a shot if guideline and code reference is provided.

LeoLuo123 avatar Jan 09 '25 16:01 LeoLuo123