Simplifying iterationEncoding and iterationFormat

[DavidSagan]

Proposal: Simplify the standard by combining iterationEncoding and iterationFormat into one attribute iterationGrouping (which should be snapshotGrouping if iteration is changed to snapshot (See Issue https://github.com/openPMD/openPMD-standard/issues/148)).

For multiple file based grouping iterationGrouping would look something like: file://filename_%T.h5 And the file:// prefix would indicate that the grouping is file based. and filename_%T.h5 is the pattern used for file matching.

For single file based grouping iterationGrouping would be set to group. One variant of this proposal would be to make iterationGrouping optional with group the default value.

[ax3l]

Thank you for the proposal!

The proposal would reduce the attributes from two to one by moving the logic from the attributes to string parsing. If we agree that string parsing might be simpler than reacting on a single-purpose attribute (this is a pure design decision) we could also go one step further:

• remove the basePath attribute
• only take the new iterationGrouping / snapshotGrouping
• allow formats such as:
  • file://filename_%T.h5:/data/%T or (file and potentially group based)
  • :/data/%T (group based, e.g. useful for streams without a file name)

We would also have to add a note to implementors on the attribute that file:// is relative to the currently open "file", not to the executable that opens it (can be confusing from an app perspective).

What do you think? Update: in the last VC we agreed this looks feasible and could simplify attributes.

Linked issues: #121

[ax3l]

ccing @C0nsultant

[ax3l]

cc @mccoys

[ax3l]

Note: we should check if / is allowed "alone" or if this would collide in groups or logic.

[DavidSagan]

Thinking about it, since files are relative to the currently open file, I would use something like file:: as the prefix instead if file:// as less confusing.

[RemiLehe]

Agree with @DavidSagan's suggestion. @DavidSagan : Do you want to do a corresponding PR?

[ax3l]

Good point, but let's not reinvent the wheel here :-)

We would use file if present as well-defined as in URIs, where it denotes the scheme (protocol). The following characters decide if it's relative or absolute, depending if they start with a / or not.

An absolute path just starts with file:/// https://en.wikipedia.org/wiki/File_URI_scheme

Nevertheless, I would encourage writing relative paths by default since it makes data series more portable.

[DavidSagan]

@ax3l

An absolute path just starts with file:/// https://en.wikipedia.org/wiki/File_URI_scheme

This might be a bit confusing since in the URI scheme something like file://abc/etc... means that abc is the host name which is not how it should be interpreted. So how about just removing the // so that something likefile:abc/etc is a relative path and file:/abc/etc is an absolute path?

[ax3l]

that's a good point with the hostname in file URIs!

Yes, either without the // (some) or skipping the file:// protocol completely (e.g. curl) seem to be common ways to express relative files. Will research it a bit more. I think I prefer the latter already for relative files.

• https://www.ietf.org/rfc/rfc3986.txt