Deprecated usage of python package in user.py

[mmacata]

The usage of recent versions of itsdangerous leads to following error:

[2022-04-28 11:54:36,723] ERROR     : gunicorn.error.glogging   -Exception in worker process [in /usr/lib/python3.9/site-packages/gunicorn/glogging.py:270]
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/lib/python3.9/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
    super().init_process()
  File "/usr/lib/python3.9/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/usr/lib/python3.9/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python3.9/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/usr/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python3.9/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/src/actinia_core/src/actinia_core/main.py", line 30, in <module>
    from .endpoints import create_endpoints
  File "/src/actinia_core/src/actinia_core/endpoints.py", line 34, in <module>
    from actinia_core.rest.location_management import \
  File "/src/actinia_core/src/actinia_core/rest/location_management.py", line 41, in <module>
    from actinia_core.rest.base.user_auth import very_admin_role
  File "/src/actinia_core/src/actinia_core/rest/base/user_auth.py", line 34, in <module>
    from actinia_core.core.common.user import ActiniaUser
  File "/src/actinia_core/src/actinia_core/core/common/user.py", line 31, in <module>
    from itsdangerous import (TimedJSONWebSignatureSerializer,
ImportError: cannot import name 'TimedJSONWebSignatureSerializer' from 'itsdangerous' (/usr/lib/python3.9/site-packages/itsdangerous/__init__.py)

As mentioned TimedJSONWebSignatureSerializer is deprecated from v2.0.0 but Flask requires itsdangerous>=2 also from v2.0.0, until this deprecated usage in actinia is fixed, the Flask version must be pinned to a version lower than that. Eg Flask==1.1.4. This then leads to:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/lib/python3.9/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
    super().init_process()
  File "/usr/lib/python3.9/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/usr/lib/python3.9/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/lib/python3.9/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/usr/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/lib/python3.9/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/src/actinia_core/src/actinia_core/main.py", line 30, in <module>
    from .endpoints import create_endpoints
  File "/src/actinia_core/src/actinia_core/endpoints.py", line 31, in <module>
    from actinia_core.core.common.app import flask_api
  File "/src/actinia_core/src/actinia_core/core/common/app.py", line 95, in <module>
    from flask_httpauth import HTTPBasicAuth
  File "/usr/lib/python3.9/site-packages/flask_httpauth.py", line 15, in <module>
    from flask import request, make_response, session, g, Response, current_app
  File "/usr/lib/python3.9/site-packages/flask/__init__.py", line 14, in <module>
    from jinja2 import escape
  File "/usr/lib/python3.9/site-packages/jinja2/__init__.py", line 12, in <module>
    from .environment import Environment
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 25, in <module>
    from .defaults import BLOCK_END_STRING
  File "/usr/lib/python3.9/site-packages/jinja2/defaults.py", line 3, in <module>
    from .filters import FILTERS as DEFAULT_FILTERS  # noqa: F401
  File "/usr/lib/python3.9/site-packages/jinja2/filters.py", line 13, in <module>
    from markupsafe import soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/lib/python3.9/site-packages/markupsafe/__init__.py)

Which can be fixed by pinning MarkupSafe==2.0.1

When the deprecated usage is removed, pinning of these versions can be undone (Flask>=0.12.3, no spcific mention of MarkupSafe)

[mmacata]

The version pinning of Flask leads to a version conflict with click when the actinia-stac-plugin is installed:

{'message': '(click 7.1.2 (/usr/lib/python3.9/site-packages), '
            "Requirement.parse('click>=8.0.0'), {'stac-validator'})",
 'traceback': '[\'  File "/src/actinia_core/src/actinia_core/endpoints.py", '
              "line 272, in create_endpoints\\n    check_import_plugins()\\n', "
              '\'  File "/src/actinia_core/src/actinia_core/endpoints.py", '
              'line 266, in check_import_plugins\\n    '
              'exec(import_run_str)\\n\', \'  File "<string>", line 1, in '
              "<module>\\n', '  File "
              '"/usr/lib/python3.9/site-packages/actinia_stac_plugin/endpoints.py", '
              'line 33, in <module>\\n    from actinia_stac_plugin.api.stac '
              "import Stac\\n', '  File "
              '"/usr/lib/python3.9/site-packages/actinia_stac_plugin/api/stac.py", '
              'line 32, in <module>\\n    from actinia_stac_plugin.core.stac '
              "import createStacItemList\\n', '  File "
              '"/usr/lib/python3.9/site-packages/actinia_stac_plugin/core/stac.py", '
              'line 28, in <module>\\n    from actinia_stac_plugin.core.common '
              "import connectRedis, defaultInstance\\n', '  File "
              '"/usr/lib/python3.9/site-packages/actinia_stac_plugin/core/common.py", '
              'line 30, in <module>\\n    from stac_validator import '
              "stac_validator\\n', '  File "
              '"/usr/lib/python3.9/site-packages/stac_validator/stac_validator.py", '
              'line 64, in <module>\\n    '
              '@click.version_option(version=pkg_resources.require("stac-validator")[0].version)\\n\', '
              "'  File "
              '"/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", '
              'line 886, in require\\n    needed = '
              "self.resolve(parse_requirements(requirements))\\n', '  File "
              '"/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", '
              'line 777, in resolve\\n    raise VersionConflict(dist, '
              "req).with_context(dependent_req)\\n']",
 'type': "<class 'pkg_resources.ContextualVersionConflict'>"}

This makes this issue more urgent.

As downpinning of Flask is only needed for TimedJSONWebSignatureSerializer and JSONWebSignatureSerializer and these again are only needed for API key and token generation and validation, possible solutions are:

• Minimize authentication to basic auth with credentials only when keycloak integration is finished
• Refactoring of user.py to move API key and token handling to a separate plugin which cannot be used together with stac-plugin then (not very sustainable)
• Rewrite API key and token handling with new tools, best using exact same hash algorithm / signature as current one to not invalidate current API keys
• dirty hack which shouldn't be used:

pip3 install --upgrade flask click \
   && sed -i "s+TimedJSONWebSignatureSerializer+BadSignature+g" /usr/lib/python3.9/site-packages/actinia_core/core/common/user.py \
   && sed -i "s+JSONWebSignatureSerializer+SignatureExpired+g" /usr/lib/python3.9/site-packages/actinia_core/core/common/user.py \
   && sed -i "s+user = ActiniaUser.verify_auth_token(username_or_token)+user = False+g" /usr/lib/python3.9/site-packages/actinia_core/rest/base/user_auth.py \
   && sed -i "s+user = ActiniaUser.verify_api_key(username_or_token)+user = False+g" /usr/lib/python3.9/site-packages/actinia_core/rest/base/user_auth.py

[mmacata]

Decided solution:

Minimize authentication to basic auth with credentials only when keycloak integration is finished

Then actinia API will be upgraded to v4, actinia.mundialis.de v3 + v4 will be deployed in parallel.