Init script needs updating?

[JimShady]

The script here says that if UbuntuGIS flag is used, then the GDAL installation will be 3.4.3

https://github.com/databrickslabs/mosaic/blob/main/scripts/mosaic-gdal-init.sh .

However UbuntuGIS actually now provides 3.6.4

https://launchpad.net/~ubuntugis/+archive/ubuntu/ppa

I think that using the UbuntuGIS option means that Mosaic does not install GDAL properly / links to the wrong GDAL version.

@mjohns-databricks ^

[mjohns-databricks]

Thanks for pointing that out, @JimShady. It is good that UbuntuGIS udated to 3.6.4 but Mosaic is standardized to the GDAL 3.4 JNI shared objects and JAR to reflect default Jammy GDAL. For clarity, we should most likely remove the option from the install script and advise users to only use the Jammy defaults. This is for the current series of Mosaic (0.4) which is for DBR 13.3. There is a new project in the works, called Spatial-Utils, that will be fore DBR 14+, so we will have to see what is preferred for handling GDAL versions in that project.

[JimShady]

I attempted PR to do this @mjohns-databricks

https://github.com/databrickslabs/mosaic/pull/555

[mjohns-databricks]

There is the setup_gdal and setup_fuse functions as well. The default is to use jammy, so shouldn't be pressing.

[noah-pitts]

Thanks for pointing that out, @JimShady. It is good that UbuntuGIS udated to 3.6.4 but Mosaic is standardized to the GDAL 3.4 JNI shared objects and JAR to reflect default Jammy GDAL. For clarity, we should most likely remove the option from the install script and advise users to only use the Jammy defaults. This is for the current series of Mosaic (0.4) which is for DBR 13.3. There is a new project in the works, called Spatial-Utils, that will be fore DBR 14+, so we will have to see what is preferred for handling GDAL versions in that project.

Exciting stuff here! Any information you could share in terms of a timeline we should expect with the transition from Mosaic to Spatial-Utils?

[mjohns-databricks]

@noah-pitts I don't anticipate heavy-weight raster DE in the first wave of what makes it into spatial-utils. You will want to focus on Mosaic 0.4 series in DBR 13.3 to meet that need for the next quarter or so. We can get into the details more at our upcoming sync if you like.

[RickLeite]

Thanks for pointing that out, @JimShady. It is good that UbuntuGIS udated to 3.6.4 but Mosaic is standardized to the GDAL 3.4 JNI shared objects and JAR to reflect default Jammy GDAL. For clarity, we should most likely remove the option from the install script and advise users to only use the Jammy defaults. This is for the current series of Mosaic (0.4) which is for DBR 13.3. There is a new project in the works, called Spatial-Utils, that will be fore DBR 14+, so we will have to see what is preferred for handling GDAL versions in that project.

Exciting to know more about this project! 😀

[mjohns-databricks]

Fix coming with 0.4.2, hopefully in a week or so, included with checkpoint_0.4.2 branch, soon to be a PR, more in branch release notes.

[mjohns-databricks]

This issue is addressed with recent release of 0.4.2.