Different behaviour when rasterizing on Windows laptop vs. Linux VM

[khyll]

Code Sample

import geopandas as gpd
import geodatasets as gds
from functools import partial
from rasterio.enums import MergeAlg
from geocube.rasterize import rasterize_image
from geocube.api.core import make_geocube
gdf = gpd.read_file(gds.get_path('nybb'))
N = gdf.shape[0]
gdf['meas'] = np.arange(0, N)
stats_fun = partial(rasterize_image, merge_alg=MergeAlg.add, all_touched=True)
raster = make_geocube(vector_data=gdf, measurements=['meas'], resolution=(1000,-1000), 
          crs=3006, rasterize_function=stats_fun, fill=-1)
meas = raster['meas']
meas = meas.where(meas != -1)  # Set background values to NaN

Problem description

On my windows laptop, the maximum value of the resulting raster (see image below) is 13, which is expected. When I run the same code on my Linux VM (through Visual Studio Code via a SSH-tunnel), the maximum value is 745. Something seems wrong with the Linux rasterization.

Expected Output

Result to the left in image.

Environment Information

For windows:

geocube v0.5.0

GDAL deps: fiona: 1.9.5 GDAL[fiona]: 3.8.3 rasterio: 1.3.9 GDAL[rasterio]: 3.8.3

Python deps: appdirs: 1.4.4 click: 8.1.7 geopandas: 0.14.3 odc_geo: 0.4.2 rioxarray: 0.15.1 pyproj: 3.6.1 xarray: 2024.1.1

System: python: 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:27:34) [MSC v.1937 64 bit (AMD64)] executable: C:\Program Files\Miniconda3\envs\rot_env\python.exe machine: Windows-10-10.0.19045-SP0

For Linux VM:

geocube v0.5.0

GDAL deps: fiona: 1.9.5 GDAL[fiona]: 3.7.2 rasterio: 1.3.9 GDAL[rasterio]: 3.7.2

Python deps: appdirs: 1.4.4 click: 8.1.7 geopandas: 0.14.2 odc_geo: 0.4.2 rioxarray: 0.15.1 pyproj: 3.6.1 xarray: 2023.6.0

System: python: 3.10.9 | packaged by conda-forge | (main, Feb 2 2023, 20:20:04) [GCC 11.3.0] executable: /home/kari.hyll/anaconda3/envs/forcore-env/bin/python3.10 machine: Linux-6.5.0-1015-azure-x86_64-with-glibc2.35

Installation method

• conda

Conda environment information (if you installed with conda):

Environment - Windows

``` # Name Version Build Channel fiona 1.9.5 py310h618e506_3 conda-forge gdal 3.8.3 py310h66fcd94_0 conda-forge libgdal 3.8.3 h576f4c1_0 conda-forge rasterio 1.3.9 py310hb35c73f_2 conda-forge rioxarray 0.15.1 pyhd8ed1ab_0 conda-forge scipy 1.12.0 py310hf667824_2 conda-forge xarray 2024.1.1 pyhd8ed1ab_0 conda-forge ```

Environment - Linux

``` fiona 1.9.5 py310h7631d76_0 conda-forge gdal 3.7.2 py310h5c4b078_5 conda-forge libgdal 3.7.2 h6238fc3_5 conda-forge rasterio 1.3.9 py310h6a913dc_0 conda-forge rioxarray 0.15.1 pyhd8ed1ab_0 conda-forge scipy 1.11.4 py310heeff2f4_0 xarray 2023.6.0 py310h06a4308_0 ```

Details about conda and system ( conda info ):

Windows

``` active environment : rot_env active env location : C:\Program Files\Miniconda3\envs\rot_env shell level : 2 user config file : C:\Users\kahy\.condarc populated config files : C:\Users\kahy\.condarc conda version : 22.9.0 conda-build version : 3.21.8 python version : 3.9.12.final.0 virtual packages : __cuda=12.0=0 __win=0=0 __archspec=1=x86_64 base environment : C:\Program Files\Miniconda3 (writable) conda av data dir : C:\Program Files\Miniconda3\etc\conda conda av metadata url : None channel URLs : https://conda.anaconda.org/conda-forge/win-64 https://conda.anaconda.org/conda-forge/noarch https://repo.anaconda.com/pkgs/main/win-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/win-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/msys2/win-64 https://repo.anaconda.com/pkgs/msys2/noarch https://conda.anaconda.org/open3d-admin/win-64 https://conda.anaconda.org/open3d-admin/noarch package cache : C:\Program Files\Miniconda3\pkgs C:\Users\kahy\.conda\pkgs C:\Users\kahy\AppData\Local\conda\conda\pkgs envs directories : C:\Program Files\Miniconda3\envs C:\Users\kahy\.conda\envs C:\Users\kahy\AppData\Local\conda\conda\envs platform : win-64 user-agent : conda/22.9.0 requests/2.27.1 CPython/3.9.12 Windows/10 Windows/10.0.19045 administrator : False netrc file : None offline mode : False ´´´

Linux

´´´ active environment : forcore-env active env location : /home/kari.hyll/anaconda3/envs/forcore-env shell level : 2 user config file : /home/kari.hyll/.condarc populated config files : /home/kari.hyll/.condarc conda version : 23.11.0 conda-build version : 3.28.3 python version : 3.11.5.final.0 solver : libmamba (default) virtual packages : __archspec=1=zen2 __conda=23.11.0=0 __cuda=12.2=0 __glibc=2.35=0 __linux=6.5.0=0 __unix=0=0 base environment : /home/kari.hyll/anaconda3 (writable) conda av data dir : /home/kari.hyll/anaconda3/etc/conda conda av metadata url : None channel URLs : https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch package cache : /home/kari.hyll/anaconda3/pkgs /home/kari.hyll/.conda/pkgs envs directories : /home/kari.hyll/anaconda3/envs /home/kari.hyll/.conda/envs platform : linux-64 user-agent : conda/23.11.0 requests/2.31.0 CPython/3.11.5 Linux/6.5.0-1015-azure ubuntu/22.04.3 glibc/2.35 solver/libmamba conda-libmamba-solver/23.12.0 libmambapy/1.5.6 aau/0.4.2 c/2MtWx04_zMqb0dUChxV0WQ s/tZxdIUgWEAtKRRnFxwouxQ e/6ZnDx99xhj25DGrgjGelpA UID:GID : 10832710:10832710 netrc file : None offline mode : False ´´´

[snowman2]

Note: You have different versions of GDAL on each system. Please make sure that you have the same versions of GDAL and other libraries when making the comparison.

[khyll]

Upgrading the Linux gdal to 3.8.3 actually seems to solve the issue. A bit worrisome that the other gdal version gave flawed result, but good to have a solution.