specs icon indicating copy to clipboard operation
specs copied to clipboard

Use more strict multiple files definition

Open roll opened this issue 6 months ago • 2 comments

What Data Package version are you using?

v2

Describe the Issue

Currently, there is no well-defined rules for multiple files - https://datapackage.org/standard/data-resource/#multiple-files. As an implementer, I don't have enough information to make reading them in a reproducible way.

I would suggest:

  • for now, allowing multiple files only for tabular data
  • clearly defining the 2-N files expectations e.g. no header row (probably based on current data engines expectations)

Participation

  • [x] I am willing to submit a pull request for this issue.

roll avatar Jun 30 '25 15:06 roll

I personally didn't have issues with interpreting it for frictionless-r, but might be good to define it better.

for now, allowing multiple files only for tabular data

Perhaps, but that would forbid having a bunch of images defined as a single (non-tabular) resource. Do we want to restrict that for non-tabular data? I'd rather have clearer expectations for tabular data, such as: "All files in the array MUST be the same in terms of structure, format, dialect, etc. Implementations MUST be able to concatenate the files in the simplest way and treat the result as one large file."

clearly defining the 2-N files expectations e.g. no header row (probably based on current data engines expectations)

Definitely no for me: it should follow the same dialect as the first file. So if the first file has a header (and defined as such in dialect), so must the consecutive files.

peterdesmet avatar Jul 01 '25 08:07 peterdesmet

It should follow the same dialect as the first file. So if the first file has a header (and defined as such in dialect), so must the consecutive files.

It sounds good! What I mean is that there should a clear rule for those files because at the moment it might be ambiguous to implement (if we need to cover all of edge cased as how current wording can be understood)

roll avatar Jul 07 '25 07:07 roll