Meshes with Non-Cartesian Geometries

[mccoys]

The argument dataOrder may not be sufficient because for future non-Cartesian geometries: we have to be able to specify which of the coordinates is curved. For instance, let's take cylindrical, which is not defined yet in the standard. An array written in the order (r , theta and z) does not have the same representation as the same array written in the order (r , z and theta).

Note that there is no problem in Cartesian geometry because you can always define labels ordered differently (y,x,z). The data reading does not change.

Now, curved geometries need a specific ordering of the axes. Currently, there is no way for the user to make this explicit. We cannot force one particular order of these variables, and we cannot rely on labels to be chosen among mandatory options. This means we have to define some other optional argument, for example, a permutation like (1,3,2) that provides a simple way for the reader to permute the data before plotting. In fact, this could completely replace the dataOrder argument. I know that there could be some performance issues, but the usual values F or C could still be used.

This is related to #188 where new geometries are considered.

Approximate proposal:

The dataOrder definition must be modified to accommodate lists of integers like (1,3,2) representing the permutation of axes that the data should undergo before being plotted. It can still accept C, meaning no changes or F, meaning the permutation (2,1) in 2D, (2,1,3) in 3D, etc. (if I'm correct). The attributes axisLabels, gridSpacing, gridGlobalOffset, gridUnitSI and position are ALSO permuted by the reader.

[DavidSagan]

@mccoys

and we cannot rely on labels to be chosen among mandatory options.

And why not? For example, if cylindrical coords are being used why cannot the Standard mandate that there be an attribute for each of the three sub-records that indicates which component the sub-record represents?

[mccoys]

@DavidSagan I think that the naming of axes could provide more qualitative information to the plots (or other visualization) for the user. A cylindrical geometry might have various names for axes in order to match naming conventions that were chosen outside OpenPMD. It's true that, technically, the standard could enforce names, but I find it a bit restrictive.

[DavidSagan]

@mccoys

What could be done is to have two attributes for the component sub-records. One attribute unambiguously identifies that component. For example, (r , z and theta) for cylindrical coordinates. The other attribute could be the any local name. And there would be no restrictions on what the local name could be. This solves the data order problem since the component sub-records are identified by name rather than order while putting no restrictions on what local names could be.

[mccoys]

@DavidSagan that definitely is a possibility

[ax3l]

@mccoys can you take a look at #194 ? We are in the process of getting rid of dataOrder since the index order to flattened (logical) memory is sufficient. This also works in curved geometries as long as they are structured.

[mccoys]

@ax3l I agree with #194, although I haven't given it much thinking. I wonder if the HDF5 interface will make things confusing with the apparent shape of datasets being in a different order than the axes labels. And maybe even give different results depending on whether the HDF5 library has the fortran or C default.

[ax3l]

are you referring to the h5py library's .shape in the type h5py.Dataset? It's order is the same as the numpy ndarray.shape index order which is by default the C order (unless changed explicitly on creation), so that's consistent.

If you query openPMD attributes such as axisLabels now directly as an attribute it will be in the opposite order in a C api as the low-level index order (and low-level data .shape), yes (before it was the same). Nevertheless, the confusion added by an additional inversion with dataOrder was larger than the benefit you mention.

We are designing a openPMD object aware API where we can return or directly map it consistently to make things easier.

[ax3l]

For dataOrder removal in 2.0, please see https://github.com/openPMD/openPMD-standard/issues/235#issuecomment-2130075297

We generally allow in upcoming 2.0 arbitrary axis labels on meshes, e.g., the laser envelope extension uses x,y,t.