channels icon indicating copy to clipboard operation
channels copied to clipboard

Use Django async-native APIs where possible

Open bigfootjon opened this issue 1 year ago • 9 comments

As discussed on the Django forumns Carlton pointed out that one area we'd like to invest in in is Channels support for the new async functionality that has been added over the last several releases (refer to the above thread for details).

I've scanned the channels repo by searching for database_sync_to_async and observed a few areas that could benefit from these new APIs:

  1. Sessions
  2. Auth

(this list is the same as Carlton pointed out on that thread, something tells me he might be some kind of expert!)

The other calls to database_sync_to_async are from wrappers for intentionally sync code, or tests (neither of which are pertinent).

I'd like to propose using these native APIs. The one hitch I see is that these features are too new to be used presently:

  1. async sessions has not yet been released (scheduled for 5.1)
  2. async auth was released in 5.0

Considering that Channels currently supports 4.2 (and definitely doesn't yet support unreleased 5.1) we might have to do some feature-checking or wait until Channels bumps its minimum supported version. Should I start now with writing some polyfills/feature-checks or set this aside until minimum versions are bumped?

bigfootjon avatar Apr 14 '24 01:04 bigfootjon

Hey @bigfootjon — thanks for this.

So, yes, we should definitely leverage the new APIs where we can.

The standard approach would be to try and import the new code (or branch on Django version number), use it if it's available, otherwise fallback to our existing code.

We won't drop 4.2 support until 5.2 is released (and maybe a teeny bit after that) but that's no reason we can't get folks on the latest versions updated.

We test against Django main branch, which is already 5.1 (although pre-alpha), so no-reason not to already begin with the code for the upcoming version.

carltongibson avatar Apr 14 '24 05:04 carltongibson

Perhaps separate pull requests for sessions and auth to simplify testing and code review and also to provide more flexibility in deploying the PRs.

cclauss avatar Apr 14 '24 09:04 cclauss

Sounds good, I'll put up 2 different PRs and import testing to see if the new features exist.

While figuring out what would be needed to do that I noticed some bits in the docs that need to be updated for async django ORM support: https://github.com/django/channels/pull/2090

bigfootjon avatar Apr 15 '24 01:04 bigfootjon

Here we go:

  1. Sessions PR: https://github.com/django/channels/pull/2092
  2. Auth PR: https://github.com/django/channels/pull/2093

I also noticed an out-of-date fallback import in https://github.com/django/channels/pull/2091

bigfootjon avatar Apr 15 '24 02:04 bigfootjon

The usage of django async method is very useful, also because it is easier to write tests with django built-in testcases.

My concern is: currently there is nothing in the flow of channels which performs database connection cleanup while django uses request_started/request_finished signals to perform such task:

https://github.com/django/django/blob/43cdfa8b20e567a801b7d0a09ec67ddd062d5ea4/django/db/init.py#L62-L63

IMO the only good part left in database_sync_to_async is to invoke the close_old_connections, without it how are we going to clean-up those connections?

sevdog avatar Aug 14 '24 14:08 sevdog

@sevdog thats actually changed recently. Take a look at this PR: https://github.com/django/channels/pull/2090

bigfootjon avatar Aug 14 '24 14:08 bigfootjon

Thank you @bigfootjon, I missed that PR. Nice job.

sevdog avatar Aug 14 '24 14:08 sevdog

And now we just need to push forward these last few bits and get the new version out.

@bigfootjon I'm fairly sure I'm blocking something. Just let me know and I'll get to it 🫡

carltongibson avatar Aug 15 '24 06:08 carltongibson

Actually we’re blocked on me having a bit of time to review other PRs! The auth PR will have to wait for Django 5.2 (since the underlying change it depends on in core make the 5.1 cut)

I’ve got a few other changes I’m trying to fit into this release, I’ll tag you on them when they’re ready for a final review!

bigfootjon avatar Aug 15 '24 14:08 bigfootjon