Is it really necessary to delete atomically in `clear_expired` tokens?

[adityakrgupta25]

Currently in clear_expired function we have:

    with transaction.atomic():
        if refresh_expire_at:
            refresh_token_model.objects.filter(revoked__lt=refresh_expire_at).delete()
            refresh_token_model.objects.filter(access_token__expires__lt=refresh_expire_at).delete()
        access_token_model.objects.filter(refresh_token__isnull=True, expires__lt=now).delete()
        grant_model.objects.filter(expires__lt=now).delete()

Somewhere down in the past year, this function started timing out for us and we accumulated big debt of the tokens to be deleted. The clear_expired kept causing Operational Timeout due to the sheer volume of the tokens and a reinforcing loop was set up, accumulating more and more tokens.

To counter this, we have been deleting the tokens in batches (of around 1000) with our own code, but without making the refresh token and access token deletion token atomic. Can somebody point out what kind of potential inconsistent states could we look at if not doing deletion atomically?

Also, if it is safe, can django-oauth-toolkit consider making deletion non-atomic and batched?

[auvipy]

may be you can check this https://github.com/jazzband/django-oauth-toolkit/pull/969

[dopry]

@adityakrgupta25 it looks like this was fixed in #969, please re-open with more detail if this is still an issue for you.