dlt icon indicating copy to clipboard operation
dlt copied to clipboard

upgrade built-in streamlit app

Open rudolfix opened this issue 1 year ago • 2 comments

Background We generate a Streamlit app with dlt pipeline ... show. It needs updates to catch up with existing features.

Tasks

    • [ ] display the staging destination information
    • [ ] display the staging dataset information (if there are any merge resources)
    • [x] allow to select schemas if there's more than 1
    • [x] display more information on a resource: primary keys, merge keys and other hints

look at dlt pipeline commands and port some of the functionalities. there are nice data structures below

    • [ ] display source state for current schema
    • [ ] display resource state for particular resource
    • [ ] list extracted, normalized and loaded packages and allow to see the job, their statuses and errors inside
    • [ ] display last run trace

Bonus task: Use decompose method of the DltSource to show default execution DAG. you can use networkx to vis (make it optional dependency though)

rudolfix avatar Jun 27 '23 11:06 rudolfix

can you assign this to me? @rudolfix

munish7771 avatar Jun 29 '23 17:06 munish7771

@rudolfix should we keep this open to work on the remaining points?

sultaniman avatar Mar 21 '24 14:03 sultaniman