gdal2tiles.py : detected memory leak

[jmckenna]

Expected behavior and actual behavior.

Executing gdal2tiles throws a memory leak warning (but the command works successfully) :

"swig/python detected a memory leak of type 'OSRSpatialReferenceShadow *', no destructor found.

No other utility (gdal_merge, etc) throws this warning.

Steps to reproduce the problem.

Simply calling the script will throw the warning, but a full command will run successfully, with the warning (twice) at end:

   Generating Base Tiles:
   0...10...20...30...40...50...60...70...80...90...100
   Generating Overview Tiles:
   0...10...20...30...40...50...60...70...80...90...100

  swig/python detected a memory leak of type 'OSRSpatialReferenceShadow *', no destructor found.
  swig/python detected a memory leak of type 'OSRSpatialReferenceShadow *', no destructor found.

Operating system

Windows

GDAL version and provenance

• GDAL 3.4.0
• SWIG-master
• Python 3.10.0

Discussion

In the case that this is marked as invalid, it will still be good to have on record, so other packagers find this ticket when they hit this wall in the next year.

[rouault]

Hi Jeff, I've tried to replicate as close with GDAL master (shouldn't be different of 3.4.0 in this regard) and Python 3.10.0 on Windows with the following commands, but I didn't get any warning:

gdal_translate autotest/gcore/data/byte.tif input.tif -outsize 1024 0
python gdal2tiles.py input.tif outdir

where byte.tif is https://github.com/OSGeo/gdal/blob/master/autotest/gcore/data/byte.tif Could you try that ? If it doesn't throw warnings for you, then it is dataset dependent, and it would be good if you could provide how to replicate.

SWIG-master

Just to be sure, does this mean you have regenerated the swig/python/extensions/.cpp and swig/python/osgeo/.py with it ? (anyway I've tried with regenerated bindings, or with the ones in git, and it didn't make a difference)

[jmckenna]

I've regenerated all SWIG/Python bindings in GDAL, correct, with SWIG-master and Python 3.10.0

I can trigger the warning message with just gdal2tiles --help so I don't think this has to do with any dataset, but I could be wrong (that's why I don't think a test case is needed, as it just happens with the usage call).

Hmm yeah I knew this one would be very difficult to reproduce. (not many out there who run SWIG-master, with latest GDAL and latest Python).

[rouault]

I've regenerated all SWIG/Python bindings in GDAL, correct, with SWIG-master and Python 3.10.0

ok, and what if you don't regenerate them but use the ones in git (which are at SWIG 3.0.12) ? For reference, attached the regenerated bindings I get with SWIG master: bindings_swig_master.zip

I can trigger the warning message with just gdal2tiles --help

Hum, this smells to me like there's an issue with the regenerated bindings

[jmckenna]

The results from the byte.tif test:

Generating Base Tiles:
0...10...20...30...40...50...60...70...80...90...100
Generating Overview Tiles:
0...10...20...30...40...50...60...70...80...90...100
swig/python detected a memory leak of type 'OSRSpatialReferenceShadow *', no destructor found.
swig/python detected a memory leak of type 'OSRSpatialReferenceShadow *', no destructor found.

Issue with regenerated bindings: indeed, and this is why I struggled even to file this ticket, as I knew it would be invalid (yet sometimes these invalid ones are like gold to packagers and users who hit this later). That, and most won't hit this until next year when packages catch up to these versions.

I'm honestly fine with setting this as invalid, it was inevitable ha.

Thanks for your feedback,

[rouault]

Did you compare your regenerated bindings with the ones I attached ? (could you attach yours?) It would be good to understand why there's a difference. I regenerated mine from Linux with "make generate" from swig/python and the swig binary of SWIG master in the PATH

[rouault]

Closing. Please reopen if you can provide more elements on how to reproduce

[coroa]

I am seeing exactly these messages when creating a conda-forge based gdal install of version 3.7.0.

ie.

❯ mamba create -n test gdal==3.7.0
❯ mamba activate test
❯ gdal2tiles.py --help     
Usage: gdal2tiles.py [options] input_file [output]

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
[...]
  WEBP options:
    Options for WEBP tiledriver

    --webp-quality=WEBP_QUALITY
                        quality of webp image, integer between 1 and 100,
                        default is 75
    --webp-lossless     use lossless compression for the webp image
swig/python detected a memory leak of type 'OSRSpatialReferenceShadow *', no destructor found.
swig/python detected a memory leak of type 'OSRSpatialReferenceShadow *', no destructor found.

Does that help for the reproduction?

[rouault]

ok, so it seems this is dependent of the SWIG version. I don't get the error message with SWIG 4.0.2, but I do get it with 4.1.1

[HansBrende]

I am also getting exactly the same issue on conda forge v 3.7.0, python 3.11.

from osgeo_utils.gdal2tiles import main as gdal2tiles

if __name__ == '__main__':
    gdal2tiles(['--profile=mercator'])

warnings.warn( Usage: gdal2tiles.py [options] input_file [output] gdal2tiles.py: error: You need to specify at least an input file as argument to the script swig/python detected a memory leak of type 'OSRSpatialReferenceShadow *', no destructor found. swig/python detected a memory leak of type 'OSRSpatialReferenceShadow *', no destructor found.

[rouault]

"fix" in https://github.com/OSGeo/gdal/pull/7951.

I guess we should keep this issue opens to remind revisiting the status of https://github.com/swig/swig/issues/2638 and potentially removing the workaround as it is far from ideal

[jmckenna]

@rouault I've confirmed this fix/workaround with master and SWIG 4.1.1 on Windows. thanks!

[rouault]

For reference, another potential workaround, less invasive, was mentioned in https://github.com/swig/swig/issues/2638#issuecomment-2344819313

[rouault]

closing as an upstream issue , and superseded by by gdal raster tile