datapackage-pipelines icon indicating copy to clipboard operation
datapackage-pipelines copied to clipboard

Example showing processing zipped xls files

Open Vanuan opened this issue 9 years ago • 1 comments

I'm trying to construct an example for processing zipped xls. Downloader seems to only support csv.

A couple of questions:

  1. Can I use /tmp? How can I make sure it's deleted after I'm done? Even if some error occurs. Is there some kind of with statement across the pipeline?
  2. How can I spew parameters to the next processor (i.e. filenames in a zip)? Should I use yield or modify datapackage?
  3. Could find a link to excel processor. Is there a list of all processors available somewhere?

Vanuan avatar Dec 17 '16 13:12 Vanuan

  1. Downloader should support Excel files as well - see here for example: https://github.com/openspending/os-data-importers/blob/master/america/uruguay/fiscal.source-spec.yaml
  2. The way to do it is to create a separate 'resource' in the datapackage for each of these files.
  3. I need to add proper documentation for 'standard library' processors.

akariv avatar Dec 18 '16 10:12 akariv