OSGeo
XYZ grids: inverted Y value, failed to load
The attached XYZ grid raster fails to load in qgis through gdal. Error as above. The same file is loaded correctly in SAGA. File produced in Surfer. 1S_AsBuilt_KP_9.084_DTM_0.10.zip
The error from gdalinfo is
ERROR 1: Ungridded dataset: At line 104671, change of Y direction
On line 104671 Y value is less than on line 104670. Until that Y value has been increasing.
1691920.150 4100511.850 -76.440
1691920.250 4100488.250 -75.690
Documentation says:
The spacing between each cell must be constant and no missing value is supported. Cells with same Y coordinates must be placed on consecutive lines. For a same Y coordinate value, the lines in the dataset must be organized by increasing X values. The value of the Y coordinate can increase or decrease however.
It seems that XYZ driver works as documented but the file created by Surfer does not have data by X ascending, Y ascending OR Y descending, but in an uniform way throughout the whole file.
I do not know how Surfer file is supposed to be organized. Perhaps this page https://support.goldensoftware.com/hc/en-us/articles/115005530868-What-File-Format-Should-I-Export-from-Surfer-#datafiles could help with finding a common format for Surfer and GDAL. As a workaround you can sort your file again by X and Y columns.
thanks @jratike80 for the prompt reply. I understand this, but apparently this type of files are very widespread, produced not only by Surfer but also by other mainstream software, and are opened correctly at least by SAGA, among free sw. Would it be reasonable for GDAL to be tolerant of these incoherences?
Would it be reasonable for GDAL to be tolerant of these incoherences?
Could make sense. The driver, as currently written due to the hypothesis taken on the file structure, requires a tiny amount of RAM even for large datsets and has a reasonable processing speed since it doesn't need to ingest the full dataset in memory and can directly seek to the line of interest. For 'randomly ordered' files as those one, one would need to allocate the full grid in memory (or create a temporary on-disk GTiff file if that's too big)
I believe that sorted XYZ is even more widespread but it does not feel extraordinary hard to improve the driver to pre-process and sort if needed. It just does not happen without someone working on it.
Surfer seems to be able to write ESRI style ASC files as well, they should work out-of-the-box.
I think also that this discussion would have been of more value for other GDAL users on gdal-dev mailing list, even if it may yield a feature request.
We use PDAL to sort XYZ files, and it does load the whole file in memory for processing and it consumes 5 times RAM compared to file size. Having XYZ sorting in GDAL could be useful, but we need a flexible algorithm with RAM usage settings (I suppose XYZ file could be split into parts that can be sorted separately and then merged back to one file)
Hi. Has there been any progress on supporting ungridded XYZ files in GDAL?
