ConfigurePythonLogging clashing with UseExceptions

[rcoup]

Expected behavior and actual behavior.

In 2023, I'd hope everyone using the Python bindings is automatically doing:

from osgeo import gdal
gdal.UseExceptions()

# ... everything else...

But that seems to clash with ConfigurePythonLogging(), where messages don't get sent to the logger. I'm not sure if this is a result of #7475 or something subsequently. test_misc_14 presumably runs in a not-UseExceptions environment so doesn't encounter the issue.

Steps to reproduce the problem.

bug.py

import logging
import sys
from osgeo import gdal

logging.basicConfig(format='@PY[%(levelname)s]: %(message)s', level=logging.DEBUG)

if len(sys.argv) >= 2 and sys.argv[1] == "no":
    print("Not calling UseExceptions...")
else:
    print("Calling UseExceptions...")
    gdal.UseExceptions()

gdal.SetConfigOption("CPL_DEBUG", "ON")

gdal.ConfigurePythonLogging(enable_debug=True)
gdal.Debug("log2", "testing testing 123")
gdal.Error(gdal.CE_None, gdal.CPLE_AppDefined, "testing testing 456")

Then

$ python3 bug.py
Calling UseExceptions...
CPL: CPLSetErrorHandler() called with an error handler on the local stack.  New error handler will not be used immediately.
log2: testing testing 123
ERROR 1: testing testing 456

$ python3 bug.py no
Not calling UseExceptions...
@PY[DEBUG]: log2: testing testing 123
@PY[INFO]: AppDefined: testing testing 456

Note that the logged messages in the first one are coming from CPL/stderr, not via the Python logger.

Operating system

macOS 13.6.2/arm64, Python3.12

GDAL version and provenance

GDAL 3.9.0dev-88102bde6d, released 2023/11/28 (debug build)

[rouault]

@rcoup Aren't you the maintainer of gdal.ConfigurePythonLogging() ;-) ? If so, I'm happy to defer to you. But yeah its likely that "recent" changes have interfered with that. Making GDAL error system works smoothly with Python is a nightmare

[rcoup]

@rouault I'm happy to have a deeper look. Anything relevant you can recall beyond #7475 (where it seems to have been working/considered)?

[rouault]

Perhaps: b67ab290f4811e4e6f38da66ff92d0561d937520 : "Python bindings: make sure that CPL_DEBUG=ON and gdal.UseExceptions() work fine during gdal.VectorTranslate() (and similar) (fixes #8552); accumulate CE_Failure messages during a GDAL call when gdal.UseExceptions() (fixes #8551)"