Configuration settings provided by gdalrc or `GDAL_CONFIG_FILE` should be linted to logs

[hobu]

Feature description

It's very easy to fail to realize you are having global GDAL configuration values like GDAL_CACHEMAX or GDAL_SWATH_SIZE silently preferred over CLI-specified equivalents. While the logic of GDAL_CONFIG_FILE is quite clear, GDAL users don't always control their environment (especially when GDAL is embedded in a larger application). A colleague ran into this situation which was very frustrating to figure out, and he would have had a clue if his CLI logs simply reported:

GDAL: GDAL_CACHEMAX specified via environment variable is overridden by GDAL_CONFIG_FILE setting provided in /users/home/hobu/.gdal/gdalrc

A simple bit of logging would have saved a bunch of debugging time.

Additional context

No response

[rouault]

A colleague ran into this situation which was very frustrating to figure out, and he would have had a clue if his CLI logs simply reported:

does the config file had ignore-env-vars=yes ? otherwise env variables have precedence over it

[msmitherdc]

it did not. Are there some software packages that silently set that to true internally?

[rouault]

it did not. Are there some software packages that silently set that to true internally?

no they can't set it. But I now realize my statement was wrong. When ignore-env-vars=yes is not set, config options have precedence over environment variables . That applies whatever way is used to set them: through the config file or through --config or through CPLSetConfigOption(). Noting that the 2 former options end up calling CPLSetConfigOption()).

So, mostly a note for self: the debug message suggested by @hobu would imply we have like a CPLSetConfigOptionWithOrigin(key, value, origin) function to keep track how a config option was set to be able to emit the debug message appropriately

[hobu]

So, mostly a note for self: the debug message suggested by @hobu would imply we have like a CPLSetConfigOptionWithOrigin(key, value, origin) function to keep track how a config option was set to be able to emit the debug message appropriately

Maybe it's simply enough to log the fact that environment variables are being set from GDAL_CONFIG_FILE with the path to the actively used file.

[rouault]

When ignore-env-vars=yes is not set, config options have precedence over environment variables

Doh, I was actually wrong again and my initial statement "env variables have precedence over it" is actually true. So I'm afraid we miss something in the situation of the original issue and that a proper reproducer should be provided.

The following experiment shows that options set as environment variables or --config do override the value set in the GDAL configuration file, wheras the issue implies the other way round.

$ cat config_file.ini 
[configoptions]
GDAL_CACHEMAX=1GB

$ GDAL_CONFIG_FILE=config_file.ini gdalinfo byte.tif -checksum --debug CPL,GDAL --config GDAL_CACHEMAX=2GB >/dev/null
CPL: Loading configuration from config_file.ini
GDAL: GDAL_CACHEMAX = 2048 MB

$ GDAL_CONFIG_FILE=config_file.ini GDAL_CACHEMAX=2GB gdalinfo byte.tif -checksum --debug CPL,GDAL >/dev/null
CPL: Loading configuration from config_file.ini
GDAL: GDAL_CACHEMAX = 2048 MB

[rouault]

@hobu @msmitherdc Given my experiment in the above note (https://github.com/OSGeo/gdal/issues/13061#issuecomment-3281339790) I can't replicate the issue as reported.