dlt
dlt copied to clipboard
upgrade built-in streamlit app
Background
We generate a Streamlit app with dlt pipeline ... show
. It needs updates to catch up with existing features.
Tasks
-
- [ ] display the staging destination information
-
- [ ] display the staging dataset information (if there are any merge resources)
-
- [x] allow to select schemas if there's more than 1
-
- [x] display more information on a resource: primary keys, merge keys and other hints
look at dlt pipeline
commands and port some of the functionalities. there are nice data structures below
-
- [ ] display source state for current schema
-
- [ ] display resource state for particular resource
-
- [ ] list extracted, normalized and loaded packages and allow to see the job, their statuses and errors inside
-
- [ ] display last run trace
Bonus task:
Use decompose method of the DltSource
to show default execution DAG. you can use networkx to vis (make it optional dependency though)
can you assign this to me? @rudolfix
@rudolfix should we keep this open to work on the remaining points?