openPMD-api

openPMD-api copied to clipboard

Reading failures should ideally affect only the innermost scope possible. When one record component fails parsing due to whatever error, the neighboring record components should remain readable. When one iteration fails parsing, skip it and go on to the next. Current behavior: Parsing fails and the Series cannot be opened

TODO

• [x] Introduce ReadError error type that backends can throw to tell the frontend about recoverable failures
• [x] Throw ReadError inside ADIOS1 and ADIOS2 attribute reading routines
• [x] Frontend: Recover from ReadErrors ~~(partially implemented so far)~~
• [ ] ~~Other backends,~~ also consider dataset reading, ... UPDATE: Catching wrong attribute reads now possible in all backends, dataset reading and further things can come in a later update
• [ ] Testing: ~~file-based, group-based, varialbe-based, deferred/eager~~, opening single iterations of a file-based series, ~~streaming API, skipping iteration~~
• [ ] Merge #949 first to avoid merge conflicts
• [ ] Correctly recover from deferred actions in ADIOS2
• [x] ADIOS2 / variable-based encoding: Backend reports wrong value for snapshot in iteration 10
• [x] ~~bp4_steps test seems to not write snapshot attribute~~ only written in new ADIOS2 schema as it should be
• [x] Avoid const_cast by using the parsing state

This whole PR is a bit of a stretch-goal as there are many different types and sources of errors to be recovered from. @ax3l Can you name some specific error pattern (or even give me a test file) that the openPMD-api should handle more robustly?

I think that a backend failure should mean that the entire call to AbstractIOHandler::flush() has failed and the queue is cleared. For finer granularity error handling, the frontend must issue single flush() calls. So, I've changed error recovery procedures to clear the entire IO queue upon failure.

(not actually ready for review, just wanting to run the CI)

This pull request introduces 1 alert when merging 1985a704253a6e6654c3f15701f2a723dcf706b3 into 3c57afbb1e0ce4a597a9977961a43f66c6a5119f - view on LGTM.com

new alerts:

• 1 for Comparison result is always the same

This pull request introduces 1 alert when merging e4e00b3ae9442938beeccb9ddc602ab2af332662 into 8b4a58a8c4604a84f2d17d5df11a24ded5a0c069 - view on LGTM.com

new alerts:

• 1 for Comparison result is always the same

Can you name some specific error pattern (or even give me a test file) that the openPMD-api should handle more robustly?

The most important pattern I have it reading a series (any encoding) from a sim that is currently running. The latest time step is usually incomplete. I think we could create test files by aborting sims mid-dump somehow...?

This pull request introduces 1 alert when merging 91eed899e9bd2929d85fbb905e9a9da7d4794994 into 29bb474837288ace5d6d6bf68854a2d96624795d - view on LGTM.com

new alerts:

• 1 for Comparison result is always the same

This pull request introduces 1 alert when merging 124b92344c13797048aa8dbe2ff984c80f5ce196 into 29bb474837288ace5d6d6bf68854a2d96624795d - view on LGTM.com

new alerts:

• 1 for Comparison result is always the same

This pull request introduces 1 alert when merging 47ec9950038c46e8efc992d662b6efab4cbb185c into 9c4173a8b5d8ee9485b0df834ae5b0ee6433cbd9 - view on LGTM.com

new alerts:

• 1 for Comparison result is always the same

This pull request introduces 1 alert when merging b64988ff4c60842123489f9ba0984447cac3433e into 9c4173a8b5d8ee9485b0df834ae5b0ee6433cbd9 - view on LGTM.com

new alerts:

• 1 for Comparison result is always the same

This pull request introduces 1 alert when merging 9345ee5108847eed818bb9683ccf9aa444351047 into 1f467a3b7724e730e8625be0b5451970e466fcdb - view on LGTM.com

new alerts:

• 1 for Comparison result is always the same

I've now reproduced the only remaining CI failure locally. It looks like for some reason Clang6 does not catch an exception despite there being a fitting catch block. Not sure what to do with this yet.

If I add catch (...) at the offending places, everything works fine again. For some reason, clang-6 loses information about exception types across static libraries.

This same workaround is not necessary for the other backends.

It looks like Clang with libc++ has trouble catching exceptions: https://stackoverflow.com/questions/51673343/clang-with-libc-exceptions This only affects ADIOS1 in our case, the rest is working. I think we should either disable that test in this CI workflow, or disable ADIOS1 for that workflow.

Relevant:

I think I was now able to avoid the incompatibility issues by removing Error.hpp from the ADIOS1 backend targets entirely, replacing them with functions that throw the errors, e.g.:

    void throwReadError(
        AffectedObject affectedObject,
        Reason reason,
        std::optional<std::string> backend,
        std::string description)
    {
        throw ReadError(
            affectedObject, reason, std::move(backend), std::move(description));
    }

This way, libopenPMD.ADIOS1.(Serial|Parallel) don't define the error symbols at all.

Thanks a lot!

I did a first test reading partially written simulation data, i.e., the /openPMD attribute is not yet written for each iteration. I tested it in fileBased encoding. Currently, incomplete iterations (snapshots) are not yet leniently skipped:

openpmd-ls ~/src/warpx/build/bin/diags/diag_btd/openpmd_%T.bp
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
[~ADIOS2IOHandler] An error occurred: [ADIOS2] Requested attribute (/openPMD) not found in backend.
An error occurred while opening the specified openPMD series!
[ADIOS2] Requested attribute (/openPMD) not found in backend.

partial_btd.tar.gz

This is the output that I get upon openpmd-ls openpmd_%T.bp with your dataset:

[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '0' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '1' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '2' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '3' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '4' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '5' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '6' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '7' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '8' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '9' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '10' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '11' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '12' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '13' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '14' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '15' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '16' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '17' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '18' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '19' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '20' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '21' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '22' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '23' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '24' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '25' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '26' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '27' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '28' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '29' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '30' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '31' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '32' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '33' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '34' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '35' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '36' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '37' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '38' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '39' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '40' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '41' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '42' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '43' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '44' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '45' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '46' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '47' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '48' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '49' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '50' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '51' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '52' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '53' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '54' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '55' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '56' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '57' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '58' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '59' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '60' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '61' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '62' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '63' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '64' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '65' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '66' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '67' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '68' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '69' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '70' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '71' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '72' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '73' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '74' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '75' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '76' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '77' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '78' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '79' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '80' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '81' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '82' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '83' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '84' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '85' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '86' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '87' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '88' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '89' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '90' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '91' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '92' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '93' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '94' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '95' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '96' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '97' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '98' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[ADIOS2] Warning: Attribute with name /openPMD has no type in backend.
Cannot read iteration '99' and will skip it due to read error:
Read Error in backend ADIOS2
Object type:	Attribute
Error type:	NotFound
Further description:	/openPMD
[fwk394:1588514:0:1588514] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x6)
==== backtrace (tid:1588514) ====
 0 0x000000000003dbf0 __GI___sigaction()  ???:0
 1 0x000000000046559c std::__1::__sort<openPMD::ADIOS2IOHandlerImpl::~ADIOS2IOHandlerImpl()::$_2&, std::__1::unique_ptr<openPMD::detail::BufferedActions, std::__1::default_delete<openPMD::detail::BufferedActions> >*>()  ADIOS2IOHandler.cpp:0
 2 0x0000000000465475 std::__1::__sort<openPMD::ADIOS2IOHandlerImpl::~ADIOS2IOHandlerImpl()::$_2&, std::__1::unique_ptr<openPMD::detail::BufferedActions, std::__1::default_delete<openPMD::detail::BufferedActions> >*>()  ADIOS2IOHandler.cpp:0
 3 0x0000000000465475 std::__1::__sort<openPMD::ADIOS2IOHandlerImpl::~ADIOS2IOHandlerImpl()::$_2&, std::__1::unique_ptr<openPMD::detail::BufferedActions, std::__1::default_delete<openPMD::detail::BufferedActions> >*>()  ADIOS2IOHandler.cpp:0
 4 0x000000000045202e openPMD::ADIOS2IOHandlerImpl::~ADIOS2IOHandlerImpl()  ???:0
 5 0x000000000046fc76 openPMD::ADIOS2IOHandler::~ADIOS2IOHandler()  ???:0
 6 0x000000000046fde9 openPMD::ADIOS2IOHandler::~ADIOS2IOHandler()  ???:0
 7 0x00000000002f2cce openPMD::internal::SeriesData::close()  ???:0
 8 0x00000000002f282e openPMD::internal::SeriesData::~SeriesData()  ???:0
 9 0x00000000002f2d69 openPMD::internal::SeriesData::~SeriesData()  ???:0
10 0x0000000000287bd5 openPMD::Attributable::~Attributable()  ???:0
11 0x00000000002f3e2b openPMD::Series::Series()  ???:0
12 0x00000000000c7a0e pybind11::error_already_set::m_fetched_error_deleter()  ???:0
13 0x000000000002f95a pybind11::error_already_set::m_fetched_error_deleter()  ???:0
14 0x0000000000129fc3 cfunction_call()  ???:0
15 0x00000000000bcc84 _PyObject_MakeTpCall.localalias()  ???:0
16 0x000000000006f41c _PyEval_EvalFrameDefault()  ???:0
17 0x00000000001eb2cf _PyEval_Vector()  ???:0
18 0x000000000006d68f _PyEval_EvalFrameDefault()  ???:0
19 0x00000000001eb2cf _PyEval_Vector()  ???:0
20 0x00000000001eb938 PyEval_EvalCode()  ???:0
21 0x000000000026f8ad run_mod()  ???:0
22 0x000000000027c382 _PyRun_SimpleFileObject.localalias()  ???:0
23 0x000000000027c95b _PyRun_AnyFileObject()  ???:0
24 0x0000000000280a1f Py_RunMain.localalias()  ???:0
25 0x0000000000281245 Py_BytesMain()  ???:0
26 0x000000000002924e __libc_start_call_main()  ???:0
27 0x0000000000029309 __libc_start_main_alias_2()  ???:0
28 0x0000000000401075 _start()  ???:0
=================================

Weirdly enough, the segfault only appears when redirecting stderr into a file....

update: we use std::stable_sort not in the destructor, even though there are no duplicate files.