freeradius-server
freeradius-server copied to clipboard
FreeRADIUS
Implement async rlm_sql and drivers
Need to verify which SQL backends offer access to the FD and provide support async querying.
@pwdng investigated.
The Good
- PostgreSQL has full, native async I/O support
- https://www.postgresql.org/docs/9.6/static/libpq-async.html
- The original MySQL client driver does not support async connections, but the Maria DB client does, and is likely protocol compatible.
- https://mariadb.com/kb/en/mariadb/non-blocking-api-reference/
- https://github.com/chkpk/mysql-async-example/blob/master/mysql_async_example.c
- Oracle Instaclient also has async I/O support
- ODBC support is dependent on the connector.
The Bad
- Firebird (we can drop this as firebird provides an ODBC interface).
- FreeTDS (we probably need to keep this).
- DB2 (very small number of users, can probably drop this).
- SQLite (we need to keep this)
My $0.02 is to create an rlm_sql_async, and go from there. The idea is that we can then prune it down to (e.g.) Postgres, and work on that code / API. Once that's done, add MySQL, etc.
We're probably OK with SQLite mostly as-is. File IO is blocking, but there isn't much you can do about that. And SQLite has file locking primitives so that only one thread at a time is accessing the DB.
Just need to markup that rlm_sql instance as blocking when we're either running with a blocking ODBC driver, or using FreeTDS or SQLite.
That means we can pass it off those calls to the blocking I/O threads or however we end up doing that.
Regarding copying the code. We could do. I think @pwdng is reviewing the code, so i guess it depends on whether there's breaking changes required.
For ODBC & Oracle, it turns out the only non blocking option available is to use polling. With previous versions of Oracle, it was possible to get the underlying file descriptor but this doesn't seem to be the case anymore and they don't recommend using the non blocking API anymore as polling is bad. As for ODBC, there is a notification based system but it isn't available in either iODBC or UnixODBC. The remaining option here is to use polling for non blocking calls.
So true event driven non blocking will only be available for Postgres & MySQL.
Both MySQL and Postgres can only handle a single concurrent request per connection to the DB. The initial thought of having a single connection per worker thread is therefore impossible. We therefore need to keep a connection pool.
We can probably stick to a similar architecture to what's been done for rlm_rest and have a pool per thread.
Sure, that's simple enough. Looks like it is worth writing that two tiered connection pool stuff, i'll see if I can find time tomorrow.
Here's ongoing work to make rlm_sql async. https://github.com/pwdng/freeradius-server/commits/async_sql
Ignore changes to postgres driver. Goal here is to make the module compatible with non blocking drivers. mod_authorize has been migrated.
I have made some changes to my original commit and broken it up a bit to make it more readable.