Fiona icon indicating copy to clipboard operation
Fiona copied to clipboard

tests/test_drvsupport.py::test_no_append_driver_cannot_append[FlatGeobuf] segfault with GDAL 3.5.0

Open sgillies opened this issue 3 years ago • 4 comments

See https://github.com/OSGeo/gdal/issues/5739

sgillies avatar May 16 '22 22:05 sgillies

We can apply the patch from https://github.com/OSGeo/gdal/pull/5740 when we build 1.9 pre-release wheels.

sgillies avatar May 17 '22 01:05 sgillies

I am able to reproduce with GDAL 3.4.0 (#1114) and 3.4.3:

>>> import fiona
>>> fiona.show_versions()
Fiona version: 1.9a2
GDAL version: 3.4.3
PROJ version: 9.0.0

OS: Linux 5.13.0-44-generic
Python: 3.9.13

snowman2 avatar Jun 07 '22 01:06 snowman2

With GDAL 3.5.1 I get

_______________ test_no_append_driver_cannot_append[FlatGeobuf] ________________

tmpdir = local('/build/pytest-of-nixbld/pytest-0/test_no_append_driver_cannot_a1')
driver = 'FlatGeobuf'
testdata_generator = <function testdata_generator.<locals>._testdata_generator at 0x7fffc4fc5000>
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fffc4fe69b0>

    @pytest.mark.parametrize(
        "driver",
        [
            driver
            for driver, raw in supported_drivers.items()
            if "w" in raw and "a" not in raw
        ],
    )
    def test_no_append_driver_cannot_append(
        tmpdir, driver, testdata_generator, monkeypatch
    ):
        """
        Test if a driver that supports write and not append cannot also append

        If this test fails, it should be considered to enable append support for the respective driver in drvsupport.py.

        """

        monkeypatch.setitem(fiona.drvsupport.supported_drivers, driver, "raw")

        if driver == "BNA" and GDALVersion.runtime() < GDALVersion(2, 0):
            pytest.skip("BNA driver segfaults with gdal 1.11")

        path = str(tmpdir.join(get_temp_filename(driver)))
        schema, crs, records1, records2, test_equal, create_kwargs = testdata_generator(
            driver, range(0, 5), range(5, 10)
        )

        # If driver is not able to write, we cannot test append
        if driver in driver_mode_mingdal[
            "w"
        ] and get_gdal_version_num() < calc_gdal_version_num(
            *driver_mode_mingdal["w"][driver]
        ):
            return

        # Create test file to append to
        with fiona.open(
            path, "w", driver=driver, crs=crs, schema=schema, **create_kwargs
        ) as c:

            c.writerecords(records1)

        try:
            with fiona.open(path, "a", driver=driver) as c:
                c.writerecords(records2)
        except Exception as exc:
            log.exception("Caught exception in trying to append.")
            return

        if driver in {"FileGDB", "OpenFileGDB"}:
            open_driver = driver
        else:
            open_driver = None

        with fiona.open(path, driver=open_driver) as collection:
            assert collection.driver == driver
>           assert len(list(collection)) == len(records1)
E           AssertionError: assert 10 == 5
E            +  where 10 = len([{'geometry': {'coordinates': (0.0, 0.0), 'type': 'Point'}, 'id': '0', 'properties': OrderedDict([('position', 0)]), '...nates': (0.0, 5.0), 'type': 'Point'}, 'id': '5', 'properties': OrderedDict([('position', 5)]), 'type': 'Feature'}, ...])
E            +    where [{'geometry': {'coordinates': (0.0, 0.0), 'type': 'Point'}, 'id': '0', 'properties': OrderedDict([('position', 0)]), '...nates': (0.0, 5.0), 'type': 'Point'}, 'id': '5', 'properties': OrderedDict([('position', 5)]), 'type': 'Feature'}, ...] = list(<open Collection '/build/pytest-of-nixbld/pytest-0/test_no_append_driver_cannot_a1/foo.fgb:foo', mode 'r' at 0x7fffc4fbe770>)
E            +  and   5 = len([{'geometry': {'coordinates': (0.0, 0.0), 'type': 'Point'}, 'properties': {'position': 0}}, {'geometry': {'coordinates...operties': {'position': 3}}, {'geometry': {'coordinates': (0.0, 4.0), 'type': 'Point'}, 'properties': {'position': 4}}])

tests/test_drvsupport.py:323: AssertionError
------------------------------ Captured log call -------------------------------

dotlambda avatar Jul 06 '22 09:07 dotlambda

@dotlambda Fiona maintains a list of driver capabilities (read, write, append). This test suggest, that with GDAL 3.5.1 FlatGeobuf gained support for append.

rbuffat avatar Jul 06 '22 13:07 rbuffat