pygef icon indicating copy to clipboard operation
pygef copied to clipboard

136-feature: Allow import/export from JSON

Open maarten-betman opened this issue 2 years ago • 2 comments

Adding support for exporting and import data from json.

As discussed in #136, try to achieve this using Pydantic

maarten-betman avatar Jun 13 '23 12:06 maarten-betman

Waited for the release of pydantic v2, which is now implemented in Rust and is significantly faster than v1.x I turned the dataclasses to Pydantic BaseModels, but am getting some validation errors since some fields appear to be optional and this is not reflected in the model's type annotations.

eg, below throws a validation error since date is not optional.

class CPTData(BaseModel):
    ....
    research_report_date: date
    ...


# shim.py:

def gef_cpt_to_cpt_data(gef_cpt: _GefCpt) -> CPTData:
    ....
    kwargs["research_report_date"] = None
    ....
    return CPTData(**kwargs)

maarten-betman avatar Jul 04 '23 02:07 maarten-betman

Waited for the release of pydantic v2, which is now implemented in Rust and is significantly faster than v1.x I turned the dataclasses to Pydantic BaseModels, but am getting some validation errors since some fields appear to be optional and this is not reflected in the model's type annotations.

eg, below throws a validation error since date is not optional.

class CPTData(BaseModel):
    ....
    research_report_date: date
    ...


# shim.py:

def gef_cpt_to_cpt_data(gef_cpt: _GefCpt) -> CPTData:
    ....
    kwargs["research_report_date"] = None
    ....
    return CPTData(**kwargs)

We can set the validation errors to None, looking at the minimal available attributes of the GEF and XML definitions there are not mandatory. -> https://publicwiki.deltares.nl/download/attachments/102204318/GEF-Bore.pdf?version=1&modificationDate=1409732017000&api=v2 -> https://publicwiki.deltares.nl/download/attachments/102204318/GEF-CPT.pdf?version=1&modificationDate=1409732008000&api=v2 -> https://docs.geostandaarden.nl/bro/def-im-CPT-20220901/ -> https://docs.geostandaarden.nl/bro/def-im-BHR-GT-20220901/

RDWimmers avatar Jul 07 '23 10:07 RDWimmers