PowerNetworkMatrices.jl icon indicating copy to clipboard operation
PowerNetworkMatrices.jl copied to clipboard

[DNMY] Serialize NetworkReduction Data

Open jd-lara opened this issue 1 month ago • 2 comments

This commit implements comprehensive serialization and deserialization of NetworkReductionData within PTDF HDF5 files, addressing the issue where network reduction maps were lost during serialization, rendering the PowerNetworkMatrices less useful.

Changes:

  • Extended to_hdf5() to serialize NetworkReductionData fields
  • Extended from_hdf5() to deserialize NetworkReductionData
  • Added JSON3 import for handling complex nested data structures
  • Implemented helper functions for serializing:
    • Simple sets and dictionaries (bus maps, removed buses/arcs)
    • Complex dictionaries with tuple keys and complex values
    • Bus reduction maps and reverse search maps
    • Added branch and admittance maps
    • Branch metadata (direct, parallel, series, transformer3W)
    • ReductionContainer with all reduction types

Implementation Details:

  • Uses HDF5 groups to organize network reduction data hierarchically
  • Employs JSON3 for serializing complex nested structures
  • Preserves network topology information (bus/arc mappings)
  • Stores metadata about PSY objects (names, types) where full object serialization is not feasible
  • Maintains backward compatibility with existing HDF5 files
  • Empty NetworkReductionData is handled efficiently

Limitations:

  • PSY object references (ACTransmission, ThreeWindingTransformer) are not fully serialized, only their metadata
  • Reverse maps (reverse_direct_branch_map, etc.) are preserved as empty since they contain PSY objects as keys
  • Derived fields (all_branch_maps_by_type, name_to_arc_map) are not serialized and should be repopulated if needed

Testing:

  • Added comprehensive test suite in test_network_reduction_serialization.jl
  • Tests cover RadialReduction, DegreeTwoReduction, and combined reductions
  • Verifies topology preservation, compression compatibility, and backward compatibility

Fixes issue where PowerNetworkMatrices became less useful after deserialization due to lost network reduction information.

Thanks for opening a PR to PowerNetworkMatrices.jl, please take note of the following when making a PR:

Check the contributor guidelines

jd-lara avatar Nov 09 '25 23:11 jd-lara

@copilot review the code errors in the getter function that is failing in the tests and make a correction to the tests. Check the logs from the GitHub actions.

jd-lara avatar Nov 10 '25 00:11 jd-lara

@jd-lara I've opened a new pull request, #209, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot avatar Nov 10 '25 00:11 Copilot