spline icon indicating copy to clipboard operation
spline copied to clipboard

Report: missing lineage for data source

Open DaimonPl opened this issue 4 years ago • 2 comments

Just an idea, in our current setup spline is not enabled everywhere (currently on purpose, initial phase).

It would be helpful if there was report in UI showing which datasources do not have their lineage - sorted by number of usages

So if i have 3 jobs which are using datasource "users" and i don't have lineage for "users" itself, report could show that this datasource with that URI is used by 3 jobs but lineage is not available

DaimonPl avatar Jun 17 '20 06:06 DaimonPl

Good idea We'll think about it in a scope of a new UI Meantime you can use the following AQL query to achieve that:

FOR ds IN dataSource
    LET lineage_cnt = LENGTH(FOR a IN affects FILTER a._to == ds._id RETURN ds)
    LET impacts_cnt = LENGTH(FOR d IN depends FILTER d._to == ds._id RETURN ds)
    FILTER lineage_cnt == 0
    RETURN [impacts_cnt, ds.uri]

wajda avatar Jun 17 '20 10:06 wajda

related to #804

wajda avatar Dec 10 '20 23:12 wajda