BP3 in ADIOS2: Reads Fail on Some Attributes

[ax3l]

Describe the bug I seams there are problems reading a BP3 file from ADIOS1 with the ADIOS2 backend, with regards to reading the /openPMD base path string attribute: https://github.com/ComputationalRadiationPhysics/picongpu/issues/3223#issuecomment-612605933

To Reproduce

Create BP3 files with

export OPENPMD_BP_BACKEND="ADIOS1"

and then try to read generated files with ADIOS2:

# ADIOS2 bpls
$ bpls -l -A ./samples/thetaMode_00200.bp
# ...
  string    /data/200/meshes/E/axisLabels          attr   = {"r", "z"}
  string    /data/200/meshes/E/dataOrder           attr   = "C"
  string    /data/200/meshes/E/geometry            attr   = "thetaMode"
  string    /data/200/meshes/E/geometryParameters  attr   = "m=4;imag=+"
  double    /data/200/meshes/E/gridGlobalOffset    attr   = {0, 0}
  double    /data/200/meshes/E/gridSpacing         attr   = {1, 1}
# ...
  string    /openPMD                               attr   = "1.1.0"
  uint32_t  /openPMDextension                      attr   = 0
  string    /software                              attr   = "openPMD-api"
  string    /softwareVersion                       attr   = "0.12.0-dev"

So far so good, but /data/200/meshes/E/gridUnitSI is missing (the unit test that writes this data even sets it explicitly.)

If I write the same data set with ADIOS2, the /data/200/meshes/E/gridUnitSI is present.

Python read

export OPENPMD_BP_BACKEND="ADIOS2"
export PYTHONPATH=$PWD/lib/python3.7/site-packages/:$PYTHONPATH

then

import openpmd_api as io

s = io.Series("samples/thetaMode_%T.bp", io.Access_Type.read_only)

Warning: ADIOS2 is not configured in the JSON configuration. Running with default settings.
[ADIOS2] Warning: Attribute with name /data/200/meshes/E/gridUnitSI has no type in backend.
[ADIOS2] Warning: Attribute with name /data/200/meshes/E/gridUnitSI has no type in backend.
[~ADIOS2IOHandler] An error occurred: [ADIOS2] Requested attribute (/data/200/meshes/E/gridUnitSI) not found in backend.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: [ADIOS2] Requested attribute (/data/200/meshes/E/gridUnitSI) not found in backend.

2nd Bug: With PIConGPU ADIOS1 files ... (need to create new ones)

import openpmd_api as io

s = io.Series("simData_%T.bp", io.Access_Type.read_only)

[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
terminate called after throwing an instance of 'std::runtime_error'
  what(): [ADIOS2] Requested attribute (/openPMD) not found in backend.

Expected behavior Should be readible via the ADIOS2 BP3 engine.

Software Environment

• version of openPMD-api: 0.11.0 and dev as of d7a16a171873b435aca24b1a6a09a3dab46dd812
• installed openPMD-api via: from source (and user report below)
• operating system: Ubuntu 18.04
• machine: local laptop
• name and version of Python implementation: CPython 3.7.7
• version of HDF5: 1.10.6
• version of ADIOS1: 1.13.1
• version of ADIOS2: 2.5.0
• name and version of MPI: MPICH 3.3.2

Additional context First seen in https://github.com/ComputationalRadiationPhysics/picongpu/issues/3223#issuecomment-612605933 with PIConGPU data:

[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
terminate called after throwing an instance of 'std::runtime_error'
  what(): [ADIOS2] Requested attribute (/openPMD) not found in backend.

P.S.: should we really print a warning

Warning: ADIOS2 is not configured in the JSON configuration. Running with default settings.

?

[ax3l]

@franzpoeschel there might be two bugs here: identifying some BP3 C-string attributes in our ADIOS2 backend (maybe they are returned as array of char?) and writing the gridUnitSI attribute default in the ADIOS1 backend.

Furthermore, the ADIOS2 backend warning for not adding options via JSON seams too verbose to me, passing no options/defaults is fine and done in all our examples.

The ADIOS1 write issue has lower priority, but reading of existing (PIConGPU) BP3 files in ADIOS2 should be made robust.

[guj]

@ax3l does ADIOS1's bpls show that attribute?

[ax3l]

Oh yes, excellent question! It surprisingly does show it in ADIOS 1.13.1's bpls on files in samples/thetaMode_*.bp.

$ bpls -A samples/thetaMode_00200.bp 
  string              /meshesPath                            attr
  string              /openPMD                               attr
  string              /iterationEncoding                     attr
  string              /date                                  attr
  string              /basePath                              attr
  string              /softwareVersion                       attr
  string              /iterationFormat                       attr
  double              /data/200/timeUnitSI                   attr
  double              /data/200/time                         attr
  double              /data/200/dt                           attr
  double              /data/200/meshes/E/unitDimension       attr
  double              /data/200/meshes/E/timeOffset          attr
  double              /data/200/meshes/E/t/unitSI            attr
  double              /data/200/meshes/E/r/position          attr
  unsigned long long  /data/200/meshes/E/z/shape             attr
  double              /data/200/meshes/E/z/position          attr
  double              /data/200/meshes/E/z/unitSI            attr
  double              /data/200/meshes/E/r/unitSI            attr
  char*[]             /data/200/meshes/E/axisLabels          attr
  string              /data/200/meshes/E/geometryParameters  attr
  string              /data/200/meshes/E/dataOrder           attr
  string              /software                              attr
  string              /data/200/meshes/E/geometry            attr
  real                /data/200/meshes/E/z/value             attr
  double              /data/200/meshes/E/t/position          attr
  double              /data/200/meshes/E/gridSpacing         attr
  double              /data/200/meshes/E/gridGlobalOffset    attr
  unsigned integer    /openPMDextension                      attr
  double              /data/200/meshes/E/gridUnitSI          attr

$ $(which bpls)
lrwxrwxrwx 1 axel axel 121 Apr 21 16:56 /home/axel/src/spack/var/spack/environments/openpmd-api-dev/.spack-env/view/bin/bpls -> /home/axel/src/spack/opt/spack/linux-ubuntu18.04-skylake/gcc-8.3.0/adios-1.13.1-c6iyu7kfbn4hgzljkim4id3sm6c34fqa/bin/bpls*

Will open that part as upstream ADIOS2 issue: https://github.com/ornladios/ADIOS2/issues/2152

[guj]

@axel, so to double check, a bp1 file is created by openPMD, and adios1/bpls shows all attributes, and adios2/bpls missed one?

will you put the file on Cori's scratch and give us permission so we can all try it out?

Thanks,

[ax3l]

Absolutely, File uploaded here: https://github.com/ornladios/ADIOS2/issues/2152

[franzpoeschel]

P.S.: should we really print a warning

Warning: ADIOS2 is not configured in the JSON configuration. Running with default settings.

?

From a user's side, it only makes sense to print that warning if the user actually passed a configuration. Since in the implementation we don't distinguish that case and instead just set config = "{}" as a default argument, it's probably better to just get rid of that, yes. It's effectively a debugging statement that survived into production.

[guj]

@ax3l @franzpoeschel Norbert fixed it. I checked on my Mac and it works. So Please update ADIOS2.

[ax3l]

Thank you all, that was super quick! I just tried with the latest ADIOS2 master after https://github.com/ornladios/ADIOS2/pull/2153 was merged and can confirm this fixes it :)

[cbontoiu]

Hello,

I used ADIOS1 for some while in Python with conda and today installed adios2 and upgraded the openpmd-api but the same syntax does not work. I tried

import adios2
import openpmd_api as io
import os
#os.environ['OPENPMD_BP_BACKEND'] = 'ADIOS1' 
os.environ['OPENPMD_BP_BACKEND'] = 'ADIOS2' 

before opening the data series as:

print("> loading time series .........................................................")
ts = io.Series(filePath + '/simOutput/bp/simData_%T.bp', io.Access_Type.read_only);    
print("> finished loading time series .........................................................")

but obtained the error:

> loading time series .........................................................
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [ADIOS2] Requested attribute (/openPMD) not found in backend.
Aborted (core dumped)

[franzpoeschel]

Hello, this does not have to do with BP3 vs. BP4, but with the fact that the native ADIOS plugin in PIConGPU writes its dumps in a slightly different format than the openPMD API would understand:

  string    //author                                                      attr     
  string    //basePath                                                    attr        
  string    //date                                                        attr
  string    //iterationEncoding                                           attr
  string    //iterationFormat                                             attr
  string    //meshesPath                                                  attr
  string    //openPMD                                                     attr
  uint32_t  //openPMDextension                                            attr
  string    //particlesPath                                               attr
  string    //software                                                    attr
  string    //softwareVersion                                             attr

Due to the double slashes, the openPMD does not find these attributes. Since the ADIOS plugin in PIConGPU is soon-to-be deprecated and the openPMD plugin has at least been merged on the dev branch, the easiest fix for now would be switching to the openPMD plugin in PIConGPU. @ax3l should we support reading attributes/variables with multiple slashes? This would conform with the Unix way where a path separator is any non-zero number of slashes.

[ax3l]

Urgh, maybe let us rather fix this in the PIConGPU ADIOS1 plugin? @franzpoeschel @PrometheusPi This should be mainly in include/picongpu/plugins/adios/WriteMeta.hpp.

You can try to work-around this in openPMD-api as well, but the PIConGPU API write calls here are clearly off.

@franzpoeschel Do we actually handle this ourselves in openPMD-api for the ADIOS1 backend or does ADIOS1 handle this for us? In the latter case, we could open a feature request in ADIOS2 for the BP3 engine.

[franzpoeschel]

Has this been fixed already? Current dev branch of PIConGPU writes these attributes just fine and the openPMD API succeeds in opening the files.

[ax3l]

I cannot say, don't see a change here: https://github.com/ComputationalRadiationPhysics/picongpu/tree/dev/include/picongpu/plugins/adios

[ax3l]

@cbontoiu what is the PIConGPU .cfg script you used to write the file?

What is the openPMD-api version & adios1 & 2 version in the conda environment you use for post-processing?

I tried ...

os.environ['OPENPMD_BP_BACKEND'] = 'ADIOS1' 

In your post above, ADIOS1 was commented out (but it should be the one that is active).

[cbontoiu]

@ax3l Since that time I changed my configuration and I cannot remember. I will try to reproduce the error with the new setup and let you know. Thank you.