datapusher-plus
datapusher-plus copied to clipboard
Smart resource download
Create a DOWNLOAD_ALWAYS_WHITELIST - a list of hosts from which DP+ will always download the entire dataset even if DOWNLOAD_PREVIEW_ONLY is true.
DOWNLOAD_ALWAYS_WHITELIST are typically local hosts or hosts where the CKAN instance has fast connections/peering arrangements.
Doing so will allow us to always do comprehensive metadata inferencing even if we're only pushing PREVIEW_ROWS into the Datastore.