pygeoapi icon indicating copy to clipboard operation
pygeoapi copied to clipboard

Published 20 hours ago • verified publisher icon geopython

[WIP] upgrade Elasticsearch support to 8

Open tomkralidis opened this issue 2 years ago • 2 comments

Overview

This PR upgrades pygeoapi Elasticsearch support to version 8, and drops support for ES 7 and earlier.

Related Issue / Discussion

None

Additional Information

None

Contributions and Licensing

(as per https://github.com/geopython/pygeoapi/blob/master/CONTRIBUTING.md#contributions-and-licensing)

  • [x] I'd like to contribute [feature X|bugfix Y|docs|something else] to pygeoapi. I confirm that my contributions to pygeoapi will be compatible with the pygeoapi license guidelines at the time of contribution.
  • [x] I have already previously agreed to the pygeoapi Contributions and Licensing Guidelines

tomkralidis avatar Oct 16 '22 20:10 tomkralidis

@KoalaGeo any idea why the mainline CI did not run here?

tomkralidis avatar Oct 16 '22 20:10 tomkralidis

@KoalaGeo any idea why the mainline CI did not run here?

'lint workflow runs on all push and PR to master build workflow only runs on the master branch containers only runs when build completes

KoalaGeo avatar Oct 17 '22 08:10 KoalaGeo

https://github.com/elastic/elasticsearch-dsl-py/issues/1569

webb-ben avatar Oct 18 '22 16:10 webb-ben

FYI this PR is now ready. I am keeping as WIP to sort out the following issues with Python package dependencies:

  • elasticsearch-dsl is still bound to <8 while elasticsearch supports 8+. Installing elasticsearch-dsl after elasticsearch will downgrade the latter to <8
  • PostgreSQL tests fail given SQLAlchemy 2 which is installed via pygeofilter

As a result, in CI, I have done the following as workarounds AFTER the main requirements-*txt files are installed in .github/workflows/main.yml:

  • upgrade elasticsearch
  • downgrade SQLAlchemy

Questions:

  • @francbartoli is elasticsearch-dsl something we will support long term? See https://github.com/elastic/elasticsearch-dsl-py/issues/1569
  • @KoalaGeo should we update the PostgreSQL plugin (in another PR)?

cc @kalxas

tomkralidis avatar Jan 28 '23 15:01 tomkralidis

Proposing the following:

  • merge PR on approval
  • open an issue on elasticsearch-dsl support
  • open an issue on SQLAlchemy 2 support

Thoughts @francbartoli @kalxas @doublebyte1 @webb-ben ?

tomkralidis avatar Jan 31 '23 20:01 tomkralidis

Issues filed:

  • https://github.com/geopython/pygeoapi/issues/1123
  • https://github.com/geopython/pygeoapi/issues/1124

tomkralidis avatar Jan 31 '23 20:01 tomkralidis

I agree with @francbartoli

kalxas avatar Feb 01 '23 08:02 kalxas

Maybe we could cut a release before merging this, what do you think @kalxas?

francbartoli avatar Feb 01 '23 10:02 francbartoli

Thinking more, the safest bet is cutting a release prior to merging this PR. We can determine what kind of documentation is needed at that point depending on elasticsearch-dsl status and our use of same.

tomkralidis avatar Feb 01 '23 16:02 tomkralidis

ok, a new release is now done. Also the SQLAlchemy issue is fixed on master

kalxas avatar Feb 03 '23 13:02 kalxas