datapusher-plus
datapusher-plus copied to clipboard
dathere
A standalone web service that pushes data into the CKAN Datastore fast & reliably. It pushes real good!
If a resource extra field called "dpplus_job_config" is defined and contains a valid DP+ job configuration JSON, DP+ will apply the configuration just for that job. This will allow us...
There was a detail missing from the installation instructions, here's my analysis and how I solved it, although I still have a problem when trying to upload to the datastore,...
currently, all the log messages are in English. It'd be great if they can be internationalized using the same mechanism as CKAN https://docs.ckan.org/en/2.9/contributing/i18n.html
"Datagroomers" as the name implies, continuously "groom" the data in the background based on certain rules/recipes. At the moment, I envision them as CKAN service provider jobs. Several "datagroomers" come...
Automatic deduplication works well (#25), however, when duplicates are found and removed, the datastore table and the resource file are no longer in sync. Smarter dedup can be handled three...
For an existing resource, when a job fails after it has already been deleted from the datastore, there is no easy way to recover. To avoid this, DP+ should: -...
Currently, DP+ like Datapusher and xloader, only does drop & replace and doesn't do upserts. It'd be great if DP+ can support upserts in a performant way. This can be...
adapt them to DP+. Until they're fixed, disabling the GHA test workflow.
**Describe the bug** I'm uploading a basic TSV file with two columns (`fruit` and `price`) but the resource on my CKAN instance only shows an `id` column and `fruit price`...
**Describe the bug** qsv applydp 0.128.0 doesn't appear to support the `--prefer-dmy` flag, added here: https://github.com/dathere/datapusher-plus/blob/master/datapusher/jobs.py#L1120 qsv 0.108 seems to work, so I've downgraded there for now. ``` ckan@96a3a268e3a3:/$ /usr/local/bin/qsvdp...
