Kinto does not start when auth policy is not available

[SimonKlausLudwig]

I create a docker-compose file with Keycloak as the Identity Provider and Keycloak. As Keycloak takes some time to start, Kinto does not start and exists with this message:

<class 'json.decoder.JSONDecodeError'>: Expecting value: line 1 column 1 (char 0)

Im pretty sure this is as the issuer does not provide a valid json file. In Previous versions of docker-compose it was possible to make the kinto container depended. But this is no longer recommended.

"There's been a move away from specifying container dependencies in compose. They're only valid at startup time and don't work when dependent containers are restarted at run time. Instead, each container should include mechanism to retry to reconnect to dependent services when the connection is dropped. Many libraries to connect to databases or REST API services have configurable built-in retries. I'd look into that. It is needed for production code anyway."

What is the best way to handle this ?

[leplatrem]

Could you please share the stacktrace? I'd like to understand what code path leads to a call to authentication on startup!

[SimonKlausLudwig]

kinto_1 | Traceback (most recent call last): kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/config/actions.py", line 307, in execute_actions kinto_1 | callable(*args, **kw) kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid_multiauth/init.py", line 280, in grab_policies kinto_1 | policy = factory(**kwds) kinto_1 | File "/app/kinto/plugins/openid/init.py", line 26, in init kinto_1 | self.oid_config = fetch_openid_config(issuer) kinto_1 | File "/app/kinto/plugins/openid/utils.py", line 11, in fetch_openid_config kinto_1 | _configs[issuer] = resp.json() kinto_1 | File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 900, in json kinto_1 | return complexjson.loads(self.text, **kwargs) kinto_1 | File "/usr/local/lib/python3.7/json/init.py", line 348, in loads kinto_1 | return _default_decoder.decode(s) kinto_1 | File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode kinto_1 | obj, end = self.raw_decode(s, idx=_w(s, 0).end()) kinto_1 | File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode kinto_1 | raise JSONDecodeError("Expecting value", s, err.value) from None kinto_1 | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) kinto_1 | kinto_1 | During handling of the above exception, another exception occurred: kinto_1 | kinto_1 | Traceback (most recent call last): kinto_1 | File "/usr/local/bin/kinto", line 33, in kinto_1 | sys.exit(load_entry_point('kinto', 'console_scripts', 'kinto')()) kinto_1 | File "/app/kinto/main.py", line 216, in main kinto_1 | env = bootstrap(config_file, options={"command": "migrate"}) kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/paster.py", line 117, in bootstrap kinto_1 | app = get_app(config_uri, options=options) kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/paster.py", line 30, in get_app kinto_1 | return loader.get_wsgi_app(name, options) kinto_1 | File "/usr/local/lib/python3.7/site-packages/plaster_pastedeploy/init.py", line 129, in get_wsgi_app kinto_1 | global_conf=defaults, kinto_1 | File "/usr/local/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 253, in loadapp kinto_1 | return loadobj(APP, uri, name=name, **kw) kinto_1 | File "/usr/local/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 278, in loadobj kinto_1 | return context.create() kinto_1 | File "/usr/local/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 715, in create kinto_1 | return self.object_type.invoke(self) kinto_1 | File "/usr/local/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 152, in invoke kinto_1 | return fix_call(context.object, context.global_conf, **context.local_conf) kinto_1 | File "/usr/local/lib/python3.7/site-packages/paste/deploy/util.py", line 55, in fix_call kinto_1 | val = callable(*args, **kw) kinto_1 | File "/app/kinto/init.py", line 51, in main kinto_1 | kinto.core.initialize(config, version=version, default_settings=DEFAULT_SETTINGS) kinto_1 | File "/app/kinto/core/initialization.py", line 581, in initialize kinto_1 | config.include("kinto.core", route_prefix=api_version) kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/config/init.py", line 666, in include kinto_1 | c(configurator) kinto_1 | File "/app/kinto/core/init.py", line 207, in includeme kinto_1 | config.commit() kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/config/actions.py", line 151, in commit kinto_1 | self.action_state.execute_actions(introspector=self.introspector) kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/config/actions.py", line 314, in execute_actions kinto_1 | tb, kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/util.py", line 732, in reraise kinto_1 | raise value.with_traceback(tb) kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid/config/actions.py", line 307, in execute_actions kinto_1 | callable(*args, **kw) kinto_1 | File "/usr/local/lib/python3.7/site-packages/pyramid_multiauth/init.py", line 280, in grab_policies kinto_1 | policy = factory(**kwds) kinto_1 | File "/app/kinto/plugins/openid/init.py", line 26, in init kinto_1 | self.oid_config = fetch_openid_config(issuer) kinto_1 | File "/app/kinto/plugins/openid/utils.py", line 11, in fetch_openid_config kinto_1 | _configs[issuer] = resp.json() kinto_1 | File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 900, in json kinto_1 | return complexjson.loads(self.text, **kwargs) kinto_1 | File "/usr/local/lib/python3.7/json/init.py", line 348, in loads kinto_1 | return _default_decoder.decode(s) kinto_1 | File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode kinto_1 | obj, end = self.raw_decode(s, idx=_w(s, 0).end()) kinto_1 | File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode kinto_1 | raise JSONDecodeError("Expecting value", s, err.value) from None kinto_1 | pyramid.exceptions.ConfigurationExecutionError: <class 'json.decoder.JSONDecodeError'>: Expecting value: line 1 column 1 (char 0) kinto_1 | in: kinto_1 | Line 0 of file None:

[leplatrem]

The reason why we fetch issuer information during init is because we expose its information (endpoints etc.) in the root URL response.

A possible fix to that would consist in offering the possibility for plugins (like openid here) to register their metadata lazily.

Instead of calling

   config.add_api_capability(
        "openid",
        description="OpenID connect support.",
        url="http://kinto.readthedocs.io/en/stable/api/1.x/authentication.html",
        providers=providers_infos,
    )

we could have:

def get_infos(config):
     # fetch issuer infos
     return infos

config.add_lazy_api_capability("openid", get_infos)

And in the root URL code, when the content of a capability is a function we just call it before serving the response.