rudolfix
rudolfix
**Motivation** Most of the APIs have their openAPI definitions accessible and it is a common approach to generate python clients automatically. We want to do the same and generate the...
### Feature description I want to free space in the staging dataset and truncate the tables after load ### Are you a dlt user? Yes, I'm already a dlt user....
**Background** ibis https://github.com/ibis-project/ibis is a library that translates dataframe expressions into SQL statement and then executes them in the destination. they do nice work of compiling final SQL statement with...
**Background** The way we insert rows into mssql is ineffective. We should switch to bulk copy. MS odbc drivers come with `bcp` command that we can use. https://github.com/yehoshuadimarsky/bcpandas/blob/master/bcpandas/utils.py is a...
**Background** Allow to annotate Pydantic models with `dlt` hints. see #942 for PoC 1. Ideally we use `Annotated` PEP to allow to annotate any Python "model". 2. Ideally we allow...
Following items are left or improving over #1026 1. * [ ] use SQL MERGE statement for merging (#1129 ) 3. * [ ] add gcp and azure storage 4....
**Background** filesystem still does not support state sync `dlt` will sync local state with destination by downloading the newest set of schemas and retrieving the newest version of the state....
**Background** We could build ad-hoc pipelines by just attaching dlt sources, resources or any other iterators to destinations. Initially only custom destinations will be able to consume iterators - because...
disable rows deduplication if Incremental is attached to a resource with `merge` write disposition
### Feature description Incremental will remove duplicating rows (only with the same cursor field value, read the docs) based on content hash or primary key. This is not expected behavior...
**Background** We generate a Streamlit app with `dlt pipeline ... show`. It needs updates to catch up with existing features. **Tasks** 1. * [ ] display the staging destination information...