verified-sources
verified-sources copied to clipboard
rest_api: accept `DltResource` instances in resource list
Source name
rest_api
Describe the data you'd like to see
We could combine instances of regular resources with a. Such config could look like this:
@dlt.resource(name="stories")
def read_stories(limit: int = 10):
...
@dlt.source(name="hacker_news_source", max_table_nesting=2)
def hacker_news_source(
base_url: str = dlt.config.value,
) -> List[DltResource]:
# source configuration
source_config: RESTAPIConfig = {
"client": {
"base_url": base_url,
},
"resources": [
{
"name": "item",
"table_name": "item",
"endpoint": {
"path": "/item/{id}.json",
"data_selector": "$",
"paginator": "single_page",
"params": {
"id": {
"type": "resolve",
"field": "$",
"resource": "stories"
}
},
},
},
read_stories(20)
],
}
return rest_api_source(source_config)
Above we add resource instance with the name "stories" and also attach it to a transformer that is fully declared
Are you a dlt user?
Yes, I'm already a dlt user.
Do you ready to contribute this extension?
Yes, I'm ready.
dlt destination
No response
Additional information
No response