OSGeo
[ Issue/FR ] Paths of source files in VRTs
Expected behavior and actual behavior.
We have a process that pulls various GeoTIFF files from remote locations and creates a VRT for the user. I'm using gdal.BuildVRT() in the Python bindings. We give an option of relative or absolute source paths inside the VRT to the user. Relative source paths gives the user the opportunity to zip the GeoTIFFs and the VRTs together and send it to someone else.
The way that paths for sources are set inside the VRT file seems to be based on a set of conditions that result in particular behaviors listed here https://lists.osgeo.org/pipermail/gdal-dev/2009-October/022358.html
The post is from 2009 but still seems to hold true from my testing. It will set the relativeToVRT field and modify the paths depending on the conditions when BuildVRT() was called.
We adhered to the various conditions to get the behavior we want, namely by changing the working directory while inside the process, and it seems to work as expected. However, we recently got a report from a user that it did not. The paths remained relative as it was originally passed in, but the relativeToVRT field was set to 0. The incorrect pathing resulted in an error being thrown later in the process when BuildOverviews() was called on the VRT because the source files could not be found.
The user was using GDAL 3.7.2. Apologies, I don't have any more information to reproduce the problem and am failing to reproduce it myself in multiple environments.
As a request, I was wondering if its possible to have more explicit control over the paths within the function call? Changing the working directory, which is what we're currently doing, causes some potential issues with multithreading in Python because it changes the working directory for all threads. It would simplify things If there was an option to just set the relativeToVRT value within BuildVRT() args and for source paths to not be changed from how they are input in that case. I also might just be completely off base in my interpretation of this process so appreciate any information.
GDAL version and provenance
3.7.2
Without a reproducer, I don't think any action is possible on our side, as the explicit option might not work. Changing the current directory should not be necessary: if both the output VRT filename and your source filenames are specified as full paths and that the source filenames are "under" the VRT in the file hiearchy, the VRT driver will automatically write them as relative paths with relativeToVRT=1.
Without a reproducer, I don't think any action is possible on our side, as the explicit option might not work.
Understandable. Thanks.
Changing the current directory should not be necessary: if both the output VRT filename and your source filenames are specified as full paths and that the source filenames are "under" the VRT in the file hiearchy, the VRT driver will automatically write them as relative paths with relativeToVRT=1.
In the scenario where its not under the VRT, I believe we have to change the current working directory right?
The filetree looks like this from our downloader. We didn't want to nest the GeoTIFFs down too far because some people use the downloader only for the source GeoTIFFs, and the VRT generation is a separate option. Although maybe changing the file structure is the best option.
Package-Downloader/
├─ GeoTIFFs/
│ ├─ UTM17/
│ │ ├─ Example.tiff
│ │ ├─ Example.tiff.aux.xml
├─ VRTs/
│ ├─ Primary_VRT_UTM17.vrt.ovr
│ ├─ Primary_VRT_UTM17.vrt
│ ├─ Subregion_VRTS/
│ │ ├─ Subregion_VRT_1A.vrt
│ │ ├─ Subregion_VRT_1A.vrt.ovr
I was able to get relativeToVRT=1 with this file tree structure by changing the working directory and passing in relative paths ex. "../../GeoTIFFs/UTM17/Example.tiff". It did not modify the path when I did it this way (it modified it in other cases).
The filetree looks like this from our downloader.
ok, in that situation, the logic that computes relative paths from absolute paths will not work (would be extra complication, and kind of purpose, at least in a default mode). And indeed you have to cd to the output directory and provide that paths in a relative way
The filetree looks like this from our downloader.
ok, in that situation, the logic that computes relative paths from absolute paths will not work (would be extra complication, and kind of purpose, at least in a default mode). And indeed you have to cd to the output directory and provide that paths in a relative way
Thanks for confirming. You can close. I can see explicit option having maybe a small use case consisting of just me.
Can continue changing the process working directory and maybe just parse the text myself if need be. Thanks.