elementary icon indicating copy to clipboard operation
elementary copied to clipboard

Add orchestrator jobs data to alerts

Open quickcoffee opened this issue 3 months ago β€’ 4 comments

This is a first attempt for https://github.com/elementary-data/elementary/issues/1944.
I tested it locally with a failed model and a failed test:
image image

I'm not quite sure if the tests that Claude Code created make that much sense - looking forward to some feedback here.

Summary by CodeRabbit

  • New Features
    • Alerts now include orchestrator context (job_id, job_name, job_run_id/run_id, job_url, job_run_url/run_url, orchestrator) propagated from data generation through rendering.
    • Alert subtitles can show an inline "Job" line and include deep links ("View in …") to runs and jobs; messages show a link icon.
  • Tests
    • Added tests covering orchestrator info extraction, job/run link creation, and message rendering with/without orchestrator data.

quickcoffee avatar Aug 29 '25 11:08 quickcoffee

Walkthrough

Adds optional job/orchestrator fields to alert payloads, exposes them via AlertModel.orchestrator_info, adds Icon.LINK and its Unicode, introduces orchestrator link utilities, renders job/orchestrator links in alert subtitles, updates dbt macros/models to populate invocation fields, and adds unit tests. (49 words)

Changes

Cohort / File(s) Summary
Icons & Unicode
elementary/messages/blocks.py, elementary/messages/formats/unicode.py
Added Icon.LINK enum member and mapped it in ICON_TO_UNICODE to "πŸ”—".
Alert models & schemas
elementary/monitor/alerts/alert.py, elementary/monitor/alerts/model_alert.py, elementary/monitor/alerts/source_freshness_alert.py, elementary/monitor/alerts/test_alert.py, elementary/monitor/fetchers/alerts/schema/alert_data.py
Added optional fields job_id, job_name, job_run_id, job_url, job_run_url, orchestrator to schemas and constructors; added AlertModel.orchestrator_info property; subclasses forward new params.
Alert message builder
elementary/monitor/alerts/alert_messages/builder.py
Extended subtitle builder API to accept orchestrator_info, render inline "Job" line when present, and aggregate/emit orchestrator/job links into subtitle blocks.
Orchestrator link utilities (new)
elementary/monitor/data_monitoring/alerts/integrations/utils/orchestrator_link.py
New OrchestratorLinkData pydantic model plus create_orchestrator_link and create_job_link helpers that produce optional link metadata (with icons) from orchestrator_info.
DBT macros & models
elementary/monitor/dbt_project/macros/alerts/population/*.sql, .../get_source_freshness_results.sql, .../utils/normalized_source_freshness_status.sql, elementary/monitor/dbt_project/models/alerts/alerts_v2.sql
Joined dbt_run_results/dbt_invocations, added invocation/job fields to alert payloads, tightened status references, parameterized normalized freshness macro, and added model dependencies on invocation/run-results.
Tests
tests/unit/alerts/alert_messages/test_orchestrator_message_simple.py, tests/unit/alerts/test_orchestrator_integration.py
Added tests for orchestrator_info, create_orchestrator_link/create_job_link, and alert message rendering with/without orchestrator data.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant DBT as dbt (macros/models)
  participant Fetcher as Fetcher (alert_data)
  participant Alert as AlertModel
  participant Builder as AlertMessageBuilder
  participant Utils as OrchestratorLinkUtils
  participant Renderer as Message Renderer

  DBT->>Fetcher: emit rows including invocation fields
  Fetcher->>Alert: construct alert with job/orchestrator fields
  Alert-->>Builder: orchestrator_info (dict or None)
  alt orchestrator_info present
    Builder->>Utils: create_job_link(orchestrator_info)
    Builder->>Utils: create_orchestrator_link(orchestrator_info)
    Utils-->>Builder: OrchestratorLinkData (optional)
    Builder->>Renderer: LinesBlock (inline Job line + links)
  else no orchestrator_info
    Builder->>Renderer: LinesBlock (summary + existing links)
  end
  Renderer-->>Builder: formatted message (e.g., Block Kit)

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Files/areas to pay extra attention to:

  • SQL macros and joins introducing dbt_invocations/dbt_run_results (data correctness and NULL handling).
  • AlertModel.orchestrator_info construction logic and its consumers in the message builder.
  • New orchestrator link utilities and their icon usages (Icon enum consistency).
  • Tests that assert rendering β€” ensure test data reflects SQL-side names and alert schema propagation.

Possibly related PRs

  • elementary-data/elementary#1958 β€” Touches alert message builder; may overlap on message rendering changes.
  • elementary-data/elementary#1971 β€” Modifies SourceFreshnessAlertModel; related to the added job/orchestrator constructor fields and freshness behavior.

Suggested reviewers

  • elazarlachkar
  • michael-myaskovsky

Poem

I hop with joy, a link in paw πŸ”—
Jobs and runs now show where they are.
Gears and clouds in tidy sight,
Alerts now point you to the right.
A rabbit cheers β€” follow the star πŸ‡

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
βœ… Passed checks (2 passed)
Check name Status Explanation
Description Check βœ… Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check βœ… Passed The title 'Add orchestrator jobs data to alerts' directly and clearly summarizes the main change: adding orchestrator job data to alert objects.
✨ Finishing touches
  • [ ] πŸ“ Generate docstrings
πŸ§ͺ Generate unit tests (beta)
  • [ ] Create PR with unit tests
  • [ ] Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❀️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Aug 29 '25 11:08 coderabbitai[bot]

πŸ‘‹ @quickcoffee Thank you for raising your pull request. Please make sure to add tests and document all user-facing changes. You can do this by editing the docs files in this pull request.

github-actions[bot] avatar Aug 29 '25 11:08 github-actions[bot]

@quickcoffee can you enable "Allow edits by maintainers" on the PR?

arbiv avatar Nov 11 '25 10:11 arbiv

@quickcoffee can you enable "Allow edits by maintainers" on the PR?

@arbiv done, though was already ticked as far as I can see πŸ€”
image

quickcoffee avatar Nov 11 '25 11:11 quickcoffee