grass icon indicating copy to clipboard operation
grass copied to clipboard
verified publisher icon OSGeo

[Bug] grass7:v.rast.stats fails with QGIS 3.34.10 or 3.34.11 on Windows 10

Open a-benini-2 opened this issue 1 year ago • 10 comments

Describe the bug

After updating QGIS LTR from 3.34.9 to 3.34.10 grass7:v.rast.stats does not calculate univariate statistics from a raster map based on vector polygons. Instead of uploading statistics to new attribute columns, grass7:v.rast.stats returns these new attributes with only NULL s as sole value. Further updating to QGIS LTR 3.34.11 hasn’t helped to overcome this issue. While running grass7:v.rast.stats with a parallel installed QGIS version 3.34.9 works fine.

My operating system is Windows 10.

I’ve raised this as a QGIS-issue and I’ve been told that this issue is not due to QGIS itself (s. here). Furthermore, the response to my QGIS-issue states, that this issue also occurs even directly using GRASS-GIS 8.4.0 installed by the OSGeo4W installer for QGIS 3.34.11 on Windows 10. And thus I’ve been advised to report this issue as bug here.

To reproduce

  1. Go to https://github.com/qgis/QGIS-Sample-Data
  2. Download QGIS-Sample-Data-master.zip on the computer where you have QGIS 3.34.11 installed
  3. unzip QGIS-Sample-Data-master.zip --> QGIS-Sample-Data-master
  4. Open QGIS 3.34.11 and start a new project
  5. Go within folder QGIS-Sample-Data-master to ~\qgis_sample_data\shapefiles\lakes.shp and ~\qgis_sample_data\raster\SR_50M_alaska_nad.tif. Drag and drop these two files / layers into the new QGIS project you just started
  6. Open the Processing Toolbox and search for v.rast.stats
  7. Open v.rast.stats set lakes as vector input and SR_50M_alaska_nad as raster input, specify column prefix, optionally select 1 or more methods and change percentile. Leave Advanced Parameters as they are. grafik
  8. Press the Run button and wait until grass7:v.rast.stats has finished. The log should reveal some error: log_file_v.rast.stats_QGIS_3.34.11.txt
log_file_v rast stats_QGIS_3 34 11 
Importing raster map <rast_66feb3cbb3a5b3>...
0..3..6..9..12..15..18..21..24..27..30..33..36..39..42..45..48..51..54..57..60..63..66..69..72..75..78..81..84..87..90..93..96..99..100
c:\Users\loc-bia3\Documents>g.region n=9275122.968681416 5=-735684.661767209 e=6363148.437637258 w=-6232946.672697669 res=7181.354110795283
c: \Users\loc-bia3\Documents>v.rast.stats -c map=vector_66feb3cbb3a5b2 raster=rast_
_66feb3cbb3a5b3 column_prefix="rs" method="maximum, percentile"
percentile=80 --overwrite
Processing input data (15 categories)...
WARNING: LZ4 decompression error
WARNING: LZ4 decompression error
2. WARNING: ZSTD compression error -10: Unknown frame descriptor
ERROR: Error uncompressing null data for row 8 of <rast_66feb3cbb3a5b3>
WARNING: ZSTD compression error -72: Src size is incorrect
Updating the database
WARNING: Concurrent mapset locking is not supported on Windows iden
C: \Users\loc-bia3\Documents>chcp 1252 1>NUL Warning 1: Layer 'vector 66feb3cbb3a5b2' has been declared with non-z geometry type Polygon, but it does contain geometries with Z. Setting the
z=2 hint into gpkg_geometry_columns
WARNING: Vector map <vector_66feb3cbb3a5b2> is 3D. Use format specific layer creation options (parameter 'Ico') to export in 3D rather than 2D (default).
X. 12 18 024. 305. 30 Ye 54 20 66 72 78. 84.90..96..100 WARNING: 68 features without category were skipped. Features without category are written only when -c flag is given.
v.out.ogf complete. 15 features (Polygon type) written to ‹vector_66feb3cbb3a5b2> (GPKG format) .
c: \Users\loc-bia3\Documents>exit Execution completed in 2.55 Sekunden
Ergebnisse:
¡'output': <gsProcessingOutputLayerDefinition ('sink':TEMPORARY_OUTPUT, 'createoptions': ('fileEncoding': 'windows-1252'})>}
Lade Ergebnis Layer
Algorithmus 'v.rast.stats' beendet
  1. Inspect attribute table of grass7:v.rast.stats ' s output Rast stats . The new statistic-attributes only consist of NULL s:

attribute_table_2

Expected behavior

s. above To reproduce step 8. & 9.

Screenshots

s. above To reproduce

System description

  • Operating System: [Windows 10]
  • GRASS GIS version: [8.4.0]
  • for details about the involved QGIS version look at the Versions section in the raised QGIS-issue
  • details about further software components: the log returned when running grass7:v.rast.stats with QGIS 3.34.11 (s. above To reproduce step 8.) list in the frist lines these software components: QGIS-Version: 3.34.11-Prizren QGIS-Codeversion: 2904bcec Qt-Version: 5.15.13 Python-Version: 3.12.6 GDAL-Version: 3.9.2 GEOS-Version: 3.12.2-CAPI-1.18.2 PROJ-Version: Rel. 9.4.0, March 1st, 2024 PDAL-Version: 2.6.3 (git-version: b5523a) GRASS-Version: 8.4.0

Additional context

When running grass7:v.rast.stats on QGIS 3.34.10 or QGIS 3.34.11 (on Windows 10) with different inputs I’ve seen varying error messages returned by the log. But the one thing that was consistent, was that returned statistics attributes only included NULL s.

a-benini-2 avatar Oct 04 '24 06:10 a-benini-2

The issue occurs on Windows 10 directly using either GRASS-GIS 8.4.0 or GRASS-GIS 8.5.0dev (f0a997ca8) provided by OSGeo4W. The issue didn't occur using GRASS-GIS 8.3.2 provided by OSGeo4W on the same system.

agiudiceandrea avatar Oct 04 '24 08:10 agiudiceandrea

Thanks for the report @a-benini-2! (Small advice: to make the log content searchable, if there will be reason for another report in the future, please post the log as text).

This looks very similar to #4284, fixed by #4297. In fact v.rast.stats internally calls r.univar so this is hopefully fixed with next GRASS release (8.4.1).

nilason avatar Oct 04 '24 08:10 nilason

The issue occurs on Windows 10 directly using either GRASS-GIS 8.4.0 or GRASS-GIS 8.5.0dev (f0a997c) provided by OSGeo4W. The issue didn't occur using GRASS-GIS 8.3.2 on the same system.

However, if it fails also with GRASS-GIS 8.5.0dev f0a997c then I have to revise my above statement.

nilason avatar Oct 04 '24 08:10 nilason

I've also verified that the issue doesn't occur using GRASS-GIS 8.4.0 from WinGRASS, while it does occur using GRASS-GIS 8.4.0 from OSGeo4W on the same Windows 10 system. @jef-n, may you please have a look a this issue?

agiudiceandrea avatar Oct 04 '24 11:10 agiudiceandrea

@a-benini-2 I had the same issue with r.univar and v.rast.stats from OSGeo4W installed GRASS 8.4: I tried it now from the Windows installer and it works, as @agiudiceandrea says.

mazingaro avatar Oct 04 '24 17:10 mazingaro

If I understand this correctly, the standalone installer for Windows works, while the OSGeo4W distribution does not. Given that is the case, I assume this is caused by conflicting dependencies. OSGeo4W currently installs both libgomp-1.dll and libomp.dll, whereas the standalone only installs libomp.dll. The standalone installer still does suffer from OpenMP issues. The PR #4121 is aiming to streamline the two distributions and as part of that fixing the OpenMP dependency issue(s).

nilason avatar Oct 07 '24 08:10 nilason

If I understand this correctly, the standalone installer for Windows works, while the OSGeo4W distribution does not.

@nilason, the standalone WinGRASS GRASS-GIS 8.4.0 works, while GRASS-GIS 8.4.0 from OSGeo4W doesn't work. Anyway the standalone WinGRASS GRASS-GIS 8.5.0dev doesn't work either.

Given that is the case, I assume this is caused by conflicting dependencies. OSGeo4W currently installs both libgomp-1.dll and libomp.dll, whereas the standalone only installs libomp.dll

In fact it seems the case:

  • WinGRASS GRASS-GIS 8.4.0 works: only libomp.dll
  • GRASS-GIS 8.4.0 from OSGeo4W doesn't work: both libomp.dll and libgomp-1.dll
  • WinGRASS GRASS-GIS 8.5.0dev doesn't work: both libomp.dll and libgomp-1.dll

agiudiceandrea avatar Oct 07 '24 09:10 agiudiceandrea

If I understand this correctly, the standalone installer for Windows works, while the OSGeo4W distribution does not.

@nilason, the standalone WinGRASS GRASS-GIS 8.4.0 works, while GRASS-GIS 8.4.0 from OSGeo4W doesn't work. Anyway the standalone WinGRASS GRASS-GIS 8.5.0dev doesn't work either.

Given that is the case, I assume this is caused by conflicting dependencies. OSGeo4W currently installs both libgomp-1.dll and libomp.dll, whereas the standalone only installs libomp.dll

In fact it seems the case:

  • WinGRASS GRASS-GIS 8.4.0 works: only libomp.dll
  • GRASS-GIS 8.4.0 from OSGeo4W doesn't work: both libomp.dll and libgomp-1.dll
  • WinGRASS GRASS-GIS 8.5.0dev doesn't work: both libomp.dll and libgomp-1.dll

Thanks! That was very useful information. I kind of figured that was case.

nilason avatar Oct 07 '24 09:10 nilason

@nilason there was a typo: WinGRASS GRASS-GIS 8.4.0 installs only libgomp-1.dll (not libomp.dll as previously incorrectly written).

Anyway, I've also checked now that GRASS-GIS 8.3.2 from OSGeo4W, which works, does install both libgomp-1.dll and libomp.dll

agiudiceandrea avatar Oct 07 '24 10:10 agiudiceandrea

@nilason there was a typo: WinGRASS GRASS-GIS 8.4.0 installs only libgomp-1.dll (not libomp.dll as previously incorrectly written).

Anyway, I've also checked now that GRASS-GIS 8.3.2 from OSGeo4W, which works, does install both libgomp-1.dll and libomp.dll

Thanks!

The OSGeo4W installs in fact both the GCC provided OpenMP library libgomp-1.dll which comes with mingw-w64-x86_64-gcc, and the LLVM libomp.dll from mingw-w64-x86_64-openmp. On the OSGeo4W side the probable solution would be to drop the mingw-w64-x86_64-openmp dependency, and the three related patches [^1]. I can't test this myself, so I refrain to make a PR on this, but that seems to me the likely solution.

[^1]: https://github.com/jef-n/OSGeo4W/blob/86989f9092cdffedc3b69fa1665b6fff2307e86c/src/grass-dev/osgeo4w/patch#L9-L10, https://github.com/jef-n/OSGeo4W/blob/86989f9092cdffedc3b69fa1665b6fff2307e86c/src/grass-dev/osgeo4w/patch#L22-L23 and https://github.com/jef-n/OSGeo4W/blob/86989f9092cdffedc3b69fa1665b6fff2307e86c/src/grass-dev/osgeo4w/patch#L79

nilason avatar Oct 07 '24 12:10 nilason