erlang
Simplified SSL option value for `cacerts`?
When one wants to use system default trusted certs to verify peer. There seems to be only two options:
- Extract the trusted CA certificates in a file, and provide
cacertfileoption - Trust ALL default ca certs from the OS:
{cacerts, public_key:cacerts_get()}
I'd like to have option 2 simplified as {cacerts, system_defaults} or similar so OTP's ssl lib can call public_key:cacerts_get() for me.
This is necessary because public_key:cacerts_get() is often a quite bloated term.
And SSL options are usually passed around and stored in various process states.
When process crashes or when exceptions with SSL options in the context are caught and dumped to logs,
the options are printed to the logs which in turn bloats the logs quite much.
A further enhancement is probably not to turn system_defaults into public_key:cacerts_get(),
but use it as an indication to search for certs in certificate store which is periodically refreshed thus to ensure certificate renewal is smooth.
Thank you for you input. We definitely will consider doing changes in this direction. We also have our own thoughts of a "caching mechanism" for cert provided as DER directly. Can not give you any estimates of when it can be prioritized.
@IngelaAndin thank you. If the "caching mechanism" means passing a reference but not a large term, that's great. I am unsure why it's limited to DER format, but it could be a good start. Thanks.
Another point I added was the support for root CA certificates renewal (without having to restart).
The current cacertfile solution has support for it (ssl lib periodically reloads it), but public_key:cacerts_get() is a immutable term.
If the OS trusted certifcates can be loaded into a "cache", and then application can use a reference for it, I would wish this cache to be periodically refreshed.
The result of public_key:cacerts_get() is stored on the literal area so it never is a bloated term, until it is printed that is.
But until then it doesn't cost any memory and is already cached between users.
Well I guess we have three things, if certs are provided as DER in options we want to create a cache. In public_key:get_cacerts case we want the cache to be refreshable as the PEM cache is. We also want https://github.com/erlang/otp/pull/8741 so that public_key:get_cacerts will not be printed by SASL progress reports.
@zmstone As I said #8741 should fix you log problem, cacerts are not not printed in error reports already today so it is just the progress reports. For certs provided by cacerts that is not public_key:cacerts_get() but raw DER is the caching improvement I was thinking about. For PEM-files we have a really not disruptive cache refresh mechanism, but for system defaults as @dgud mentions is implemented differently. And maybe we need to think about if this should be handled by user application by calling public_key:cacerts_clear/0 instead? I am not sure about the tradeoffs here!?
It's often more than SASL progress which may print the certs. When a config is constructed at application level, and passed down the abstraction layers, each layer may have its own idea about error handling and logging.
If one uses a third party HTTP client (e.g. Hackney) this problem is not solved by #8741. For example, when Hackney is used in combination with tls_certificate_check and progress reporting is enabled in Logger, the following get printed into Common Test logs:
*** System report during my_test_SUITE:init_per_suite/1 2025-01-15 09:14:28.079 ***[🔗](file:///path/to/project/_build/test/logs/[email protected]_09.12.47/lib.my_app.logs/run.2025-01-15_09.12.48/my_test_suite.init_per_suite.html#e-5389)
=PROGRESS REPORT==== 15-Jan-2025::09:14:28.034467 ===
supervisor: {local,hackney_sup}
started: [{pid,<0.15886.0>},
{id,<<"localhost">>},
{mfargs,
{hackney_pool,start_link,
[<<"localhost">>,
[{name,<<"localhost">>},
with_body,
{pool,<<"localhost">>},
{ssl_options,
[{verify,verify_peer},
{depth,100},
{cacerts,
[<<48,130,5,52,48,130,3,30,160,3,2,1,2,2,1,1,
48,11,6,9,42,134,72,134,247,13,1,1,11,48,
45,49,11,48,9,6,3,85,4,6,19,2,83,69,49,17,
48,15,6,3,85,4,10,19,8,75,105,118,114,97,
...
Which is then followed by ~4 Mb of "pretty printed" certificate binaries.
It would be nice if it would be possible to pass in a persistent term of that certificate list or something smaller, that represents the full list. Something like {cacerts, {persistent_term, my_certs}} that would then make the SSL library read that persistent term.
