airflow-provider-great-expectations icon indicating copy to clipboard operation
airflow-provider-great-expectations copied to clipboard

Add operator extra link for Data Docs in the `GreatExpectationsOperator`

Open josh-fell opened this issue 3 years ago • 2 comments

In Airflow, there is a feature which allows for external links to be present as buttons on the Task Instance modal in the Airflow UI called an operator extra link. This feature is really useful to directly navigate from the Airflow UI to an external site which may be for monitoring a third-party execution that Airflow just triggered, handy documentation, etc.

IMO linking to Data Docs directly from a GreatExpectationsOperator task would be really beneficial to users and increase the holistic visibility that's available at their fingertips.

josh-fell avatar Feb 22 '22 20:02 josh-fell

Hi @josh-fell - this is a really cool idea! The location for the Data Docs sites would be available through the Data Context, by using DataContext.get_docs_sites_urls(), which would be available via the main Operator. What would be the best way of passing this information out of the Operator and into a Plugin for these purposes?

talagluck avatar Mar 15 '22 16:03 talagluck

The easiest, and the most widely-used, way to get that data into an operator link is via XCom. If you can push a URL to XCom or whatever values would be needed to build the URL string, you can have the get_link() method that would be overwritten to return the URL.

For some examples, check out Azure Data Factory and dbt Cloud providers. Then in the GreatExpectationOperator you can add the link as an attribute. Something like:

class GreatExpectationsOperator:
    ...
    operator_extra_links = (DataDocsLink(),)

    ...
    def __init__(...):

josh-fell avatar Mar 28 '22 19:03 josh-fell

Completed in 0.2.0

denimalpaca avatar Nov 22 '22 14:11 denimalpaca