openPMD-standard icon indicating copy to clipboard operation
openPMD-standard copied to clipboard

Put in **Definitions** section in the Standard

Open DavidSagan opened this issue 7 years ago • 3 comments

I propose to put in a Definitions section in the standard to make it easier for people to look up what things mean.

Some quick notes:

  1. If a Definitions section is put in there would have to be some cleanup in other parts of the Standard.
  2. Do we want "dataset" instead of "data set" since what is being referred to here is not a generic set of data but a particular construct?
  3. The Naming conventions for records should be merged into the Definitions section.

Here is a quick draft to open the discussion:

  • Group: A group is a container structure containing a set of zero or more attributes, a set of zero or more groups (which can be called sub-groups), and a set of zero or more datasets. Note: In HDF5, these are also called groups.

  • Dataset: A dataset is a structure that contains a set of zero or more attributes and a data aray (which may be multidimensional). Note: In HDF5, these are also called datasets.

  • Record: A record is a group (without any sub-groups) or a dataset that contains data on a physical quantity like particle charge or electric field. There are three types of records:

    • scalar records hold scalar quantity values (like particle charge). If all the particles have the same charge, the value of the charge is stored as an attribute of the record and there is no associated data array. That is, the record is a group. If the particles have differing charges, the values are stored in an array of the scalar record. In this case the scalar record is a dataset.
    • vector records hold a set of datasets. In this case the record is a group.
      • Example: A record named E for holding electric field values may have three datasets holding the components of the field named E/x, E/y, and E/z.
    • tensor record hold tensor quantities.

    Records with only a scalar component are stored in a data set with the same name as the record. Vector and tensor records shall be represented component-wise as a collection of individual scalar data sets using a common sub-group that is equal to the record name.

    We refer to the scalar record itself and the vector sub-group as record, to the data sets in the vector sub-group as components. For scalar records, the record is the component (and vice versa).

  • Attribute: An attribute is a variable associated with a group along with a value. Example: snapshotPath is a string variable associated with the root / group.

  • Particle Root Group: A Particle Root Group is a group for specifying a set of particles. There can be multiple particle root groups in a data file. Particle root groups are subgroups of the particlesPath group.

  • (Bool) datum: A boolean datum is an datum that has two states: True and False. If a particular storage standard does not support boolean then the datum will be encoded as an integer with states 0 defined as False and 1 defined as True.

  • (float) datum: A (float) datum represents a floating point number. How the value of the datum is stored is not specified by this Standard. That is, integer representations may be used and data reader programs should make the appropriate conversion to floating point.

DavidSagan avatar Feb 26 '18 02:02 DavidSagan

@DavidSagan This looks useful. Would you like to open a pull request with the above?

RemiLehe avatar Apr 03 '18 04:04 RemiLehe

VC today: we will address this in it's whole at the end of the 2.0.0 merge window to see what is still missing or lately-defined to improve structure and readability.

ax3l avatar Apr 03 '18 17:04 ax3l

@ChristopherMayes posted in https://github.com/openPMD/openPMD-standard/pull/193#discussion_r188371209:

BTW, I see a few REAL8 mentioned, but this word isn't actually standard (Neither is real(8)) in Fortran. Fortran 2008 provides standard kinds: REAL32, REAL64, REAL128 http://fortranwiki.org/fortran/show/Real+precision

ax3l avatar Nov 07 '18 17:11 ax3l