Replacing SSOWat (but why?™)

[selfhoster1312]

The idea to replace SSOWat entirely was brought up again in #676, talking about Authelia as a potential replacement. Here's the tracking issue for that idea.

Meanwhile, @alexAubin has started work on a new portal API and portal frontend (separate components), as tracked here. In those branches, SSOWat as a lua script still exists but has reduced responsibilities, with more power to a Python backend split across moulinette (login/logout) and yunohost (portal-api) packages.

More candidates were identified here. Out of them, only Kanidm and Authelia have the feature set to replace SSOWat, and not just act as an additional provider for eg. OpenID Connect. However, neither have armv6 (in debian parlance: armel) packages available on their repositories for Raspberry Pi support.

Adding to the original comparison:

Name	Debian package	Architectures	Backend	Frontend	Groups/permissions	nginx ACL	Password reset	Multi-factor auth	Invitations/Registrations
portal-api	yunohost repo	everything	LDAP	HTTP headers	yes	access_by_lua_file	no	no	no
Authelia	external repo	amd64,arm64,armhf	LDAP/HTTP headers	OpenID Connect	yes	yes	yes	yes	no?
Kanidm	external repo	amd64,arm64 (with modern instructions)	Own database,LDAP	HTTP headers/OpenID Connect	yes	not directly	yes	yes	no?
lemonLDAP-ng	yes	everything	LDAP, own database, others	OpenID Connect, SAML, HTTP headers	yes	yes	?	yes	?
shibboleth	yes	everything	?	OpenID Connect, HTTP Headers	?	?	?	?	?

[BRGustavoRibeiro]

As replied in #676, although I really like Authelia, Kanidm is worth a look. It has an amazing performance, it's stable, safe and kinda flexible. Looks nice.

[selfhoster1312]

I've been following Kanidm for a while and it looks great! The main downside is they have strict hardware requirements. Because an authentication platform is central to many things and has to run fast, they have chosen to only support CPUs with modern instruction sets, which excludes a lot of systems Yunohost runs on.

Actually i see they support different CPU profiles in their builds so maybe with CpuOptLevel::none it could work on anything. Still, at the moment rust only targets glibc 2.17 on arm-unknown-linux-gnueabihf (raspberry pi or armel in Debian terms) and a lot of crates require a more recent version. If you can cross-compile it and run it on a Raspberry Pi, it's definitely worth looking into. I may be wrong about this, but i get a feeling Authelia will be much easier to cross-compile for all the Yunohost platforms.

[alexAubin]

I'd also try to check https://git.autistici.org/id/go-sso and https://git.autistici.org/ai/sso though i have not checked the exact set of features, but it's used by some inspiring italian folks

[hex-m]

Shibboleth supports multiple backends but Database and LDAP are probably the relevant ones. It's only the authentication part of the InCommon-Stack. Group management would be done in Grouper. Multi-factor-auth is not supported natively but there is a plug-in to make it play nice with privacyIDEA.

The whole project is geared towards the needs of big organizations so it's probably not ideal for small-scale self-hosting purposes.

[selfhoster1312]

I'd also try to check https://git.autistici.org/id/go-sso

Love autistici! The actual daemons are split between sso-server and auth server, but i'm not sure it exactly fits the bill. From the README:

Also note that the authenticated identity is not passed along to the backend

The expectation is sso-server (or rather sso-proxy) acts as a reverse proxy but lets backend application handle their own sessions, via /sso_login and /sso_logout routes defined in their sso protocol. To be clear i think it's great that their protocol requires backend applications to implement proper login/logout workflow, but that would require specific implementation for every Yunohost app.

Also, not providing auth headers to the backend means it's not backwards-compatible with the existing SSOWat architecture, although if it is the only blocking point it should be fairly easy to implement in sso-server then using nginx auth_request.

[Shibboleth] is geared towards the needs of big organizations so it's probably not ideal for small-scale self-hosting purposes.

Thanks for those links, i still can't find a complete admin manual. If you believe Shibboleth is too complex for Yunohost usecase, that's fine, but if you find the manual feel free to post it here.

[selfhoster1312]

So about the idea to replace SSOWat, i've been running very unscientific benchmarks. I've tried to disable authentication entirely (commenting out access_by_lua_file), the current SSOWat implementation, the SSOWat implementation from portal-api branch, some patches i applied on top of that, and a custom minimalist SSOWat implementation in rust.

Here are the results for 5000 req/s with fail2ban disabled:

Authentication	Success (%)	Min	Mean	50	90	95	99	Max
None	99.98%	107.486µs	1.653ms	682.756µs	3.913ms	8.395ms	15.106ms	34.706ms
SSOWat	99.84%	110.139µs	11.207ms	9.919ms	21.693ms	26.165ms	47.438ms	88.249ms
portal-api	34.43%	130.044µs	24.463s	30.031s	36.934s	38.734s	47.061s	57.936s
portal-api+patches	99.66%	421.631µs	25.981ms	33.233ms	36.978ms	40.572ms	57.215ms	83.917ms
ssowat-rs	100%	492.241µs	2.519ms	963.978µs	7.72ms	12.894ms	18.437ms	42.876ms

Experimental notes:

• the machines used in the measurement were used for other purposes at the same time, so the results may not be reproduced accurately ; this is not scientific benchmarking!
• the ssowat-rs daemon was run under a single (async) thread to measure how well it would fare irrespective of how many CPU cores are available ; the connection with nginx was handled by a unix socket not an open port
• this benchmark was run on a beefy machine with each core running at 2.93MHz which is not representative of real hardware running Yunohost
• nginx is running with 4 threads, as configured by Yunohost
• the results 50/90/95/99 correspond to the Nth percentile of the results
• the benchmarks are run for a single logged in user and only a handful of installed apps ; the results may change dramatically with more apps to go through to authorize requests, or with more user sessions opened/invalidated

Interpretation:

• the variance in success rate (except for the portal API branch) are probably just noise in the measurement due to server/network load unrelated to the experiment
• no authentication is always the fastest
• nginx lua is doing really well at 5000 req/s ; i was expecting a lot worse
• the current portal-api branch is doing REAL bad by dropping caching entirely ; this PR makes it better, but not as good as previous SSOWat
• even with those optimizations, the portal-api branch is at least two times slower than previous SSOWat on average
• ssowat-rs is doing good... somehow i was expecting a lot better (due to pre-compilation) or a lot worse (due to making an extra HTTP request/processing for every incoming request) ; it's 4 times faster than SSOWat on average and at least twice as fast in the worst cases, although in the best case it's slower by 0.4ms
• ssowat-rs only adds 1ms compared to no authentication in the average case, 4ms in the 99th percentile, and 8ms at maximum

Recommendations:

• these benchmarks should be run again on a single-core VM and a raspberry pi to get results that are more representative of real Yunohost hardware ; i'll come up with a way to run those tests in an automated fashion
• no authentication is always fastest, so public routes should probably disable authentication entirely in the nginx config to avoid unnecessary load... this is not how yunohost apps are architectured so far so no easy change
• Rust is a good candidate to write readable code that performs well ; i faced some trouble cross-compiling for raspberry pi (although i'll try again soon) but if this can be solved it's definitely worth looking into
• nginx auth_request is very fast so it's actually worth it to try SSOWat alternatives and benchmark them too (Authelia, LemonLDAP-ng...)

[zamentur]

kanidm is the choice of selfprivacy.org