Bump 9.6-3.1 to GEOS 3.9.1

[TomPohys]

Hi all,

I would like to ask if there is a plan to update GEOS to 3.9.1 (https://github.com/libgeos/geos/releases)?

As I thought wrongly, update HASH for master still leads to GEOS 3.7.1 (https://github.com/postgis/docker-postgis/commit/c58090124ccd7f54b39bb5bcf2f79b7011e94964) and not upgrade the GEOS version to 3.9.1.

I am trying to resolve a GEOS bump in OpenMapTiles.

Thanks for comment

[nyurik]

I think it would be great to upgrade all 3.1 tags to the latest official release of geo libraries, not just 12-master and 13-master, while keeping postgres & postgis to the designated versions. Older Postgis versions might not be not as important for the upgrade. An alternative would be to create another set of all tags, e.g. 9.6-3.1-3.9.1, but that seem too much, and adds another dimension to the variants (might be ok though)

[ImreSamu]

my personal opinion - for brainstorming ( I am still thinking :thinking: )

@TomPohys:

I would like to ask if there is a plan to update GEOS to 3.9.1

IMHO: this is mostly an upstream ( Debian packaging) issue. As I know

• for the current Debian version - no Geos update expected
• for the next Debian (bullseye) - now: geos 3.9.0 ( and not 3.9.1 )

see the related e-mail thread

IMHO: no easy way to "Bump 9.6-3.1 to GEOS 3.9.1"

• We need to build the geos 3.9.1
• and we need to rebuild the PostGIS lib
  • search for "Conditional Compilation" #if POSTGIS_GEOS_VERSION < 39
    • https://github.com/postgis/postgis/search?q=%22%23if+POSTGIS_GEOS_VERSION%22
• and we need to preserve the "compatibility" for the other users

@nyurik:

I think it would be great to upgrade all 3.1 tags to the latest official release of geo libraries, not just 12-master and 13-master, while keeping postgres & postgis to the designated versions.

my personal opinion/suggestions:

1.) adding more "build time" customizations to the "master Dockerfile" (base image, versions)

• adding ARG instructions
  • to create ~ any combinations for the "local" customized build.
• keeping the current compatibility - with the "-master"

so in theory you need to run this script:

# ... draft example - for your use cases  ...
rm -rf docker-postgis
git clone --quiet --depth 1 https://github.com/postgis/docker-postgis.git
cd docker-postgis/13-master

# customized local build 
docker build --pull \
  \
  --build-arg BASE_PG_IMAGE=postgres:9.6 \
  \
  --build-arg SFCGAL_VERSION=v1.3.9 \
  --build-arg SFCGAL_GIT_HASH=823db7a318b8841f8296e80036ef993ddf19ebf5 \
  --build-arg PROJ_VERSION=8.0.0 \
  --build-arg PROJ_GIT_HASH=12e7a4395784d2dfb45cc0b719d37cd7e534fe91 \
  --build-arg GEOS_VERSION=3.9.1 \
  --build-arg GEOS_GIT_HASH=1b2f4bd3e19001c37846687c765595aa8c28b90f \
  --build-arg GDAL_VERSION=v3.2.2 \
  --build-arg GDAL_GIT_HASH=a33784291d19015217ea2604988e53d448e14a07 \
  --build-arg POSTGIS_VERSION=3.1.1 \
  --build-arg POSTGIS_GIT_HASH=aaf4c79a4906f11727fe6b9bccb12051a52acfe3 \
  \
  -t postgis:pg9.6-pgis3.1.1-geos3.9.1-proj8.0.0-gdal3.2.2  .

# final step:
# -->  Push a new image to your registry ( https://hub.docker.com/u/openmaptiles ) 

IMHO: I am deeply conservative - what we add to the postgis docker-hub

• the PG 9.6 Final release = "November 11, 2021"
• the debian11(bullseye) - is near.
• and lot of use cases .. it is easy to add;
  • on the other hand hard to remove / change / support
    • so need a deep discussion

IMHO: Later: if the -master script is more mature

• We can add a ~13-latest; 14-latest version only for the last 2 Postgres version;

2.) adding Debian:bullseye based PostGIS images as early as we can

• Now I am testing the Debian "11/testing/bullseye" based - postgis3.1+Geos3.9.0 version
  • Debian 11: expected this calendar year, it is now in a "Hard Freeze" status;
  • IMHO: usable ( --> I am using in my pet projects )
  • my test postgis:13-3.1-bullseye
    • docker size ( ~ 567MB )
    • new LLVM(for a posgres JIT!)/Geos3.9/Proj7.2/Gdal3.2/...
• maybe we can add this version in the next months; with an extra -bullseye tag
  • The biggest problem - no upstream postgres-bullseye versions yet
    • we need to ask.
    • or temporary we need to build ourselves.
• For a long time - there will be the same problem with a locked Geos3.9/Proj7.2/Gdal3.2 version.

[ImreSamu]

related:

the method: upgrade GEOS - without re-building PostGIS

• has a "known" side effect, the 3.9+ only functions not available.

see this e-mail question:

• Q: https://lists.osgeo.org/pipermail/postgis-users/2021-April/044845.html
• A: https://lists.osgeo.org/pipermail/postgis-users/2021-April/044846.html

[nyurik]

Thanks @ImreSamu for the in-depth reply! I wonder if we can utilize something like INCLUDE+ dockerfile command (extension). This works with the recent dockers - I tried it with 20.10, Linux containers only. To build, one must use DOCKER_BUILDKIT=1 docker build ... to enable buildkit mode. The rest will be done automatically by the docker build service when it sees the # syntax = edrevo/dockerfile-plus comment in the dockerfile.

Essentially, the idea would be to make current templates files into "common" dockerfiles that can be included from specific versions, but also to make those templates usable directly by downstream projects. I don't know if all such features would be supported, needs some testing.

[nyurik]

See also a relevant question at https://github.com/edrevo/dockerfile-plus/issues/6

[ImreSamu]

@nyurik

Essentially, the idea would be to make current templates files into "common" dockerfiles that can be included from specific versions, but also to make those templates usable directly by downstream projects. I don't know if all such features would be supported, needs some testing.

Please: give more info; What is the problem you want to solve?

• a current practical example - with code fragments.

As I understand - you want to split the Dockerfile.master.template; How? and Which part is important for you?

[nyurik]

@ImreSamu above you gave a draft example of custom build with many build args. I wonder if it would make sense to do it purely in dockerfile rather than with extra bash/makefile commands. It might be more trouble than its worth though, just an idea.

[ImreSamu]

I wonder if it would make sense to do it purely in dockerfile rather than with extra bash/makefile commands. It might be more trouble than its worth though, just an idea.

I just want to add a few ARG to the Dockerfile.master.template

• With keeping the current compatibility
• With no change in the current workflow. ( ~ keeping the current "bash/makefile commands" )
• With no other extra complexity.

IMHO: Changing the basic Dockerfile syntax .. adds extra complexity; and need a Discussions

[ImreSamu]

the good news: Debian Bullseye expected in the next months ( May/June/... )

"Finding a tentative bullseye release date" : https://lists.debian.org/debian-release/2021/04/msg00187.html "The Release Team believes that the state of bullseye is pretty good." "... We propose to aim for a release date in May."

[ImreSamu]

Status: "Debian Installer Bullseye RC 1 release" ( Fri, 23 Apr 2021 07:21:43 +0200 ) https://lists.debian.org/debian-boot/2021/04/msg00192.html

13-3.1-bullseye version :

select postgis_full_version();
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                            postgis_full_version                                                                            |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| POSTGIS="3.1.1 aaf4c79" [EXTENSION] PGSQL="130" GEOS="3.9.0-CAPI-1.16.2" PROJ="7.2.1" LIBXML="2.9.10" LIBJSON="0.15" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" TOPOLOGY |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
(1 row)

[ImreSamu]

• Debian bullseye expected on 2021-07-31 http://debiantracker.herokuapp.com/
• My proposal WIP : https://github.com/postgis/docker-postgis/pull/247 ( This is only a proposal; "merge" is not guaranteed )

[TomPohys]

Great! Thank you for pushing the next steps!

[ImreSamu]

I think it would be great to upgrade all 3.1 tags to the latest official release of geo libraries,

The https://github.com/postgis/docker-postgis/pull/257 has been merged; so now

• all PostGIS3.1 images are Bullseye based now. ( ---> GEOS="3.9.0" )
• And docker pull postgis/postgis:14rc1-3.1 is available for testing.
  • https://github.com/postgis/docker-postgis/blob/1cf67643bb955faadf0ec24e70aeb94ee262e912/14rc1-3.1/Dockerfile

postgis/postgis:14rc1-3.1

psql (14rc1 (Debian 14~rc1-1.pgdg110+1))
Type "help" for help.

postgres=# select postgis_full_version();
                                                                            postgis_full_version                                                                            
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POSTGIS="3.1.3 008d2db" [EXTENSION] PGSQL="140" GEOS="3.9.0-CAPI-1.16.2" PROJ="7.2.1" LIBXML="2.9.10" LIBJSON="0.15" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" TOPOLOGY
(1 row)

postgres=# select version() ;
                                                            version                                                             
--------------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 14rc1 (Debian 14~rc1-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
(1 row)

postgis/postgis:13-3.1

psql (13.4 (Debian 13.4-1.pgdg110+1))
Type "help" for help.

postgres=# select postgis_full_version();
                                                                            postgis_full_version                                                                            
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POSTGIS="3.1.4 ded6c34" [EXTENSION] PGSQL="130" GEOS="3.9.0-CAPI-1.16.2" PROJ="7.2.1" LIBXML="2.9.10" LIBJSON="0.15" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" TOPOLOGY
(1 row)

postgres=# select version() ;
                                                           version                                                           
-----------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 13.4 (Debian 13.4-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
(1 row)

[nyurik]

@ImreSamu hi, any updates on this? I think the most stable way for OpenMapTiles would be to use your #247 with custom ARG params. This way this repo will maintain HOW to build a docker image with all dependencies, and OMT repo will specify WHAT exact versions of each component to use. This way we can have PG14+GEOS3.9.1+... as the latest PG image.

One other thought - is there a way to factor out Postgis+deps building into a separate script that Dockerfile calls during the build phase? For the heavy-duty production systems we currently install PG directly on GCP VM, and install the needed code using a startup script. It would be great if PG installation script could be centrally maintained, and users would just use it for their setup?

Thanks for all your hard work in this repo!

[ImreSamu]

@nyurik

with custom ARG params. ... any updates on this?

Still working on this.

• I want to add default package support
• the new SFCGAL requires CGAL >= 5.3 ; and bullseye has CGAL 5.2
• etc;

This way we can have PG14+GEOS3.9.1+... as the latest PG image.

If It is urgent : Temporary you can use the current https://github.com/postgis/docker-postgis/blob/master/Dockerfile.master.template just you have to replace the parameters (%%PROJ_GIT_HASH%%,%%GEOS_GIT_HASH%%,%%GDAL_GIT_HASH%%,%%POSTGIS_GIT_HASH%%, etc ) with a simple script. ( sed? https://github.com/postgis/docker-postgis/blob/master/update.sh#L95 )

• e357e0a7381a8c2f36c62375380acbf800da697d --> %%POSTGIS_GIT_HASH%% -> and you have: 3.2.0alpha1

And later you can upgrade to the new API.

... we currently install PG directly on GCP VM, ... It would be great if PG installation script could be centrally maintained, and users would just use it for their setup?

I don't have a quick answer. so please post your request to the: https://github.com/postgis/docker-postgis/discussions

IMHO:

• If full consistency is important, I think you should convert the Dockerfile to a shell script - with a tool. ( probably need some research )

[ImreSamu]

Since this issue is quite old and Postgis 9.6 is no longer supported, I am closing this issue. If you have any new information related to this, please let me know, and I will reopen it. Alternatively, feel free to create a new issue.