WIP: Initial code for aiosqlite and base async backend

[Andrew-Chen-Wang]

Adding async backend Base and aiosqlite -- someone else can pick it up if I become inactive.

• Adds aatomic and an async atomic
• Adds alru_cache
• Internal django test cases can skip if it's an async or sync backend.

aatomic probably shouldn't have been implemented in this PR, but I also couldn't think of another time it could be implemented without a Base async backend.

There is no associated ticket as of yet. This was made a month ago, but because I could never pick this up due to classes, I made a PR for others to take advantage of. I should have time after February 20th to get back to this.

To write transaction test cases, it requires the async ORM.

[Andrew-Chen-Wang]

Just some further thoughts from my post in the developer group thread for DATABASES setting for async usage:

ping @andrewgodwin if you have any thoughts on this (or if you're confused about what I'm saying).

I'm thinking that for our purposes:

1. Because several Django projects need a migration from sync to async which requires time and resources, sync engines can specify a new key called ASYNC_DATABASE_ALIAS whose value is a string that is a database alias in the DATABASES dict. The purpose of this is two-fold: A) ease Django project migrations to async and B) allow for testing on Django internal test suite and third-party packages. This also helps with backwards compatibility for the templating system such that when we call Model.objects.acreate, because of that a prefix, we can implicitly use the async engine.
2. Projects that start off as async or have their default engine as async need an option called SYNC_DATABASE_ALIAS that is exactly like the ASYNC_DATABASE_ALIAS option. The purpose for this is to fill in the gap of DEP 9 by having a migration and schema engine changer since async engines aren't supposed to. In the database wrapper, we would grab the sync database's database wrapper's Schema class.

It'll look like this:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.aiosqlite',
        'SYNC_DATABASE_ALIAS': 'sync'
    },
    'sync': {
        'ENGINE': 'django.db.backends.sqlite3',
    },
}

or

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'ASYNC_DATABASE_ALIAS': 'async'
    },
    'async': {
        'ENGINE': 'django.db.backends.aiosqlite',
    },
}

I'm not sure if either of these options are possible due to sqlite being tested in memory though.

Original post

Hi, I'm intrigued in helping develop the async ORM engines (link to random PR with aiosqlite and base async engine). The biggest road block is having test cases for async engines only; for instance, introspection/test_async.py can't run because the test suite needs to first migrate using a sync engine as the default database connection then switch the default alias to using an async engine; additionally, for most of the test suite, it uses the sync engine. The current methods I'm thinking of:

1. Each async engine would get its own CI worker. I think it's a waste of resources and because it wouldn't help Django users who also need to write test cases with both an async and sync (for migrations) engine.
2. Async and sync engines for the same database are run at the same time. Every async engine requires a parallel sync engine, an alias to be specified in the engine's DATABASES options. This way, while testing, migrations can be performed with a designated sync engine. This is great for the end user who may be async-centric; this doesn't really resolve the current problem. Most of the Django test suite is designed for synchronous db engines, so the default alias database engine will switch around a bunch of times.
3. Implement a test decorator that switches the default alias connection.

Lemme know if that's confusing.

Thanks, Andrew

[Andrew-Chen-Wang]

Can't seem to figure out why the socket is not being closed (assuming this is the async side):

Exception ignored in: <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 58768), raddr=('127.0.0.1', 58766)>
ResourceWarning: unclosed <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 58768), raddr=('127.0.0.1', 58766)>

[Shankar-Balaji-U]

from channels.db import database_sync_to_async

@database_sync_to_async
def get_user(mobile_no):
	return MyModel.objects.get(mobile_no=mobile_no)


user = get_user("9999999999")

it's works fine!!!

[felixxm]

Closing due to inactivity.