spline
spline copied to clipboard
Report: missing lineage for data source
Just an idea, in our current setup spline is not enabled everywhere (currently on purpose, initial phase).
It would be helpful if there was report in UI showing which datasources do not have their lineage - sorted by number of usages
So if i have 3 jobs which are using datasource "users" and i don't have lineage for "users" itself, report could show that this datasource with that URI is used by 3 jobs but lineage is not available
Good idea We'll think about it in a scope of a new UI Meantime you can use the following AQL query to achieve that:
FOR ds IN dataSource
LET lineage_cnt = LENGTH(FOR a IN affects FILTER a._to == ds._id RETURN ds)
LET impacts_cnt = LENGTH(FOR d IN depends FILTER d._to == ds._id RETURN ds)
FILTER lineage_cnt == 0
RETURN [impacts_cnt, ds.uri]
related to #804