oso
oso copied to clipboard
Data completeness dashboard
Describe the feature you'd like to request
Would be nice to have a status page on the web app that allows us to easily visualize what data is missing
Describe the solution you'd like
I think we can just visualize the event pointer table, looking for artifacts where the end date is older than a certain buffer from the current date.
Open question - how do we know if a start date is insufficiently early? CC @ravenac95
Describe alternatives you've considered
TBD
Can we just put this in the project page so people can clearly see what data is missing?
I'm leaving this here but it's possible that with the tools we are intending on using, dagster/airbyte we may already get a UI to see the status of all data ingestion.
When was this added to sprint 65?
Looks like we're on track to using Dagster for this purpose once we get some public instances running in our k8s cluster
Lets repurpose this as populating metadata in MaterializedResult for all assets
Also closed with #1633