unable to edit postgres_default connection

[hyw]

i'm getting an invalidToken error when attempting to edit the default postgres_default connection. i suspect this is related to setting FERNET_KEY but i can't seem to figure out what i'm doing wrong. i have FERNET_KEY and AIRFLOW__CORE__FERNET_KEY set to the same values for the webserver, scheduler, and worker containers.

i've also tried docker stop to destroy all containers and rebuilding them with FERNET_KEY and AIRFLOW__CORE__FERNET_KEY set as environment variables in my docker-compose.yml. i'm using CeleryExecutor btw.

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.7/site-packages/flask_admin/base.py", line 69, in inner
    return self._run_view(f, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_admin/base.py", line 368, in _run_view
    return fn(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_admin/model/base.py", line 2125, in edit_view
    form = self.edit_form(obj=model)
  File "/usr/local/lib/python3.7/site-packages/flask_admin/model/base.py", line 1340, in edit_form
    return self._edit_form_class(get_form_data(), obj=obj)
  File "/usr/local/lib/python3.7/site-packages/wtforms/form.py", line 212, in __call__
    return type.__call__(cls, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_admin/form/__init__.py", line 16, in __init__
    super(BaseForm, self).__init__(formdata=formdata, obj=obj, prefix=prefix, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/wtforms/form.py", line 278, in __init__
    self.process(formdata, obj, data=data, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/wtforms/form.py", line 127, in process
    if obj is not None and hasattr(obj, name):
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 353, in __get__
    retval = self.descriptor.__get__(instance, owner)
  File "/usr/local/lib/python3.7/site-packages/airflow/models/connection.py", line 155, in get_password
    return fernet.decrypt(bytes(self._password, 'utf-8')).decode()
  File "/usr/local/lib/python3.7/site-packages/cryptography/fernet.py", line 171, in decrypt
    raise InvalidToken
cryptography.fernet.InvalidToken

[hyw]

may be related to #320 and #387

[hyw]

additionally, i'm able to create new connections with encrypted fields and view/edit them. but i can't view any of the *_default connections that include encrypted fields.

[gimbo]

Just as a reference for future travellers who google their way here as I did: #320 does seem to have answer, which is setting the FERNET_KEY env var... I'd created a stack based on the example docker-compose-LocalExecutor.yml, without explicitly defining FERNET_KEY; I didn't try to edit the postgres_default connection until I'd spun the stack down and up a few times, and I guess that meant a stored (and autogenerated) fernet key didn't match the one (also autogenerated, but different) currently in use by the container. Even when I explicitly set FERNET_KEY in my docker-compose.yml, it still didn't seem to work, so just nuked my DB and started again, and then all was well: no key was autogenerated, it just used my key from the start, and the error went away. Hope this helps someone else.

[mmain10]

@gimbo - When you say "Nuked my DB", what do you mean by this in the context of your Docker services? Did you execute a complete restart of the DB directly on the container, or did you just swap out for a new container? Did you nuke the Persistent Data volume? It might be helpful for the Googler's like me to get more specifics to your solution, since FERNET_KEY explicit setting hasn't been resolved in either of the open issues in #320 and #387.

[gimbo]

@mmain10 Sorry for the delayed reply... it was a while ago now but I think the answer is that yes, I deleted the persistent data volume.

[digichap28]

This might work for you...

1) Before building/setting up the image based on puckel's docker file, generate a fernet key using your CLI with the line below:

docker run puckel/docker-airflow python -c "from cryptography.fernet import Fernet; FERNET_KEY = Fernet.generate_key().decode(); print(FERNET_KEY)"

That will pull puckel's image, spin up a container, and generate a random fernet key from with in the container.

2) Copy the Fernet Key Generated and insert in your docker-compose file as an environment variable.

webserver:
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.10 
      dockerfile: Dockerfile
    restart: always
    environment:
      - LOAD_EX=n
      - EXECUTOR=Local
      - AIRFLOW__CORE__FERNET_KEY=<INSERT THE FERNET KEY GENERATED HERE>
...

3) Remove the container generated on the step 2 (Optional).

4) Run the docker-compose file and enjoy.

Hope that works for you! Be safe!