Fiona icon indicating copy to clipboard operation
Fiona copied to clipboard

Write failures with date and str field types in the Schema, in 1.10.x

Open pwheimann opened this issue 8 months ago • 1 comments

I have run into an issue on upgrading from 1.9.x to 1.10.x...

Expected vs Actual behavior

I have a data normalization and validation pipeline that leverages Fiona to write from an input GPKG to an output GPKG. After upgrading to 1.10.x I started to encounter failures to write to a new GPKG when the Schema contained both a date type field and a str type field. It appears as though when a date type is present in a Schema all the str type fields are treated as date types, and fail on calls to rfc3339.py: parse_date accordingly.

I was able to recreate this issue using GeoJSON, so it doesn't appear to be an issue only w/ GPKG(s).

Again, using 1.9.x it works as expected, with 1.10.x I get the error(s).

To Recreate

import fiona  # type: ignore
import logging

logging.basicConfig(
    level=logging.DEBUG,
)


schema = {
    "geometry": "Point",
    "properties": {
        "date": "date",
        "text": "str",
    },
}

feature = {
    "geometry": {
        "type": "Point",
        "coordinates": [-78.40253, 42.82548],
    },
    "properties": {
        "date": "2022-09-01",
        "text": "n",
    },
}

with fiona.open("out.geojson", "w", schema=schema) as ds:
    ds.write(feature)

Running the above produces the following output...

DEBUG:fiona.ogrext:Transaction supported: 0
DEBUG:fiona.ogrext:Setting feature property: key='date', value='2022-09-01', i=0, setter=<fiona.ogrext.DateField object at 0x10ee6d580>
DEBUG:Fiona:Match groups: ('2022', '-', '09', '-', '01')
DEBUG:fiona.ogrext:Setting feature property: key='text', value='n', i=1, setter=<fiona.ogrext.DateField object at 0x10ee6d580>
DEBUG:fiona.ogrext:Flushed data source cache
DEBUG:fiona.collection:Flushed buffer
DEBUG:fiona.collection:Stopped session
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "xxx/Projects/test/fiona-ops/src/fo/main.py", line 29, in <module>
    ds.write(feature)
    ~~~~~~~~~^^^^^^^^^
  File "xxx/Projects/test/fiona-ops/.venv/lib/python3.13/site-packages/fiona/collection.py", line 551, in write
    self.writerecords([record])
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "xxx/Projects/test/fiona-ops/.venv/lib/python3.13/site-packages/fiona/collection.py", line 541, in writerecords
    self.session.writerecs(records, self)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "fiona/ogrext.pyx", line 1666, in fiona.ogrext.WritingSession.writerecs
  File "fiona/ogrext.pyx", line 788, in fiona.ogrext.OGRFeatureBuilder.build
  File "fiona/ogrext.pyx", line 417, in fiona.ogrext.DateField.set
  File "xxx/Projects/test/fiona-ops/.venv/lib/python3.13/site-packages/fiona/rfc3339.py", line 79, in parse_date
    raise ValueError(f"Time data '{text}' does not match pattern")
ValueError: Time data 'n' does not match pattern

Changing the Schema date type to str succeeds and produces the following...

DEBUG:fiona.ogrext:Transaction supported: 0
DEBUG:fiona.ogrext:Setting feature property: key='date', value='2022-09-01', i=0, setter=<fiona.ogrext.StringField object at 0x105d71200>
DEBUG:fiona.ogrext:Setting feature property: key='text', value='n', i=1, setter=<fiona.ogrext.StringField object at 0x105d71200>
DEBUG:fiona.ogrext:Flushed data source cache
DEBUG:fiona.collection:Flushed buffer
DEBUG:fiona.collection:Stopped session

pwheimann avatar Mar 14 '25 14:03 pwheimann

@pwheimann thanks for the report! I'll modify the test at https://github.com/Toblerity/Fiona/blob/main/tests/test_datetime.py#L832 to try to reproduce this.

sgillies avatar Mar 14 '25 15:03 sgillies