[BUG] Serialization should fail with a readable error when required memory cannot be allocated

[nitbharambe]

Describe the bug

Serialization fails with an difficult to interpret error message. Likely cause as per the msgpack error meassage is that required memory cannot be allocated.

To Reproduce

A big output dataset was attempted to serialize to file using json and msgpack serializer.

Expected behavior

Some understandable error message like PowerGridSerializationError would make it clear.

Screenshots

json_serialize_to_file gives out following:

same data with msgpack export it gives out bad allocation message.

[TonyXiang8787]

@nitbharambe, what is the use case to export large dataset to json?

[nitbharambe]

The dataset is only relatively huge. Intention was to store intermediary results of an analysis. (Although there can be ways around it.)

@nitbharambe, what is the use case to export large dataset to json?

[mgovers]

a couple observations:

• [ ] a good starting point might be to determine how many mega/gigabytes the data to be allocated was (e.g.: less or greater than 4GB due to page size). Depending on the specific parameters provided to the serializer, this may be calculated from the above dataset size
• [x] the difference in behaviour most likely comes from the fact that the default result value is char const*{} which is NULL. For the JSON serializer, this is interpreted as null terminated string for JSON (illegal for null, hence the stacktrace) and to fixed-size string for msgpack
• [ ] it might be a good idea to make a separate PGM_serialize_to_file functionality that directly outputs the result data to the filestream instead of first to python. If the std::bad_alloc came from e.g. the page size, then directly outputting to file might actually fix the issue.