Inconsistent results when working with unitest/pytest

[agusmakmun]

Here is my settings.py:

# Django Defender to protect API login from brute force
DEFENDER_LOGIN_FAILURE_LIMIT = 3
DEFENDER_COOLOFF_TIME = 300
DEFENDER_BEHIND_REVERSE_PROXY = False
DEFENDER_LOCK_OUT_BY_IP_AND_USERNAME = False
DEFENDER_ACCESS_ATTEMPT_EXPIRATION = 24
DEFENDER_REDIS_URL = "redis://redis:6379/0"
DEFENDER_USE_CELERY = True

and here is my test:

import pytest

from django.urls import reverse
from defender.models import AccessAttempt

pytestmark = pytest.mark.django_db


@pytest.mark.django_db
def test_brute_force_login(settings, client):
    # settings.DEFENDER_LOGIN_FAILURE_LIMIT = 3
    # settings.DEFENDER_ACCESS_ATTEMPT_EXPIRATION = 5
    # settings.STORE_ACCESS_ATTEMPTS = True
    # settings.USE_CELERY = False

    # to clean up the database before test
    AccessAttempt.objects.all().delete()

    login_url = reverse("api:auth_login")
    response1 = client.post(login_url, {
        "username": "[email protected]",
        "password": "wrong_pass1"
    })
    print("response1:", response1.data)

    response2 = client.post(login_url, {
        "username": "[email protected]",
        "password": "wrong_pass2"
    })
    print("response2:", response2.data)

    response3 = client.post(login_url, {
        "username": "[email protected]",
        "password": "wrong_pass3"
    })
    print("response3:", response3.data)

    response4 = client.post(login_url, {
        "username": "[email protected]",
        "password": "wrong_pass4"
    })
    print("response4:", response4.data)

    # to clean up the database after test
    AccessAttempt.objects.all().delete()

    assert False  # just for raising the traceback to get the above prints

And the results of my test is:

First run

response1: {'result': {}, 'message': 'Unable to log in with provided credentials.', 'status': 400, 'success': False}
response2: {'result': {}, 'message': 'Unable to log in with provided credentials.', 'status': 400, 'success': False}
response3: {'result': {}, 'message': 'Unable to log in with provided credentials.', 'status': 400, 'success': False}
response4: {'result': {}, 'message': 'Unable to log in with provided credentials.', 'status': 400, 'success': False}

Second run

response1: {'result': {}, 'message': 'You have attempted to login 3 times, with no success. Your account is locked for 300 seconds.', 'status': 400, 'success': False}
response2: {'result': {}, 'message': 'You have attempted to login 3 times, with no success. Your account is locked for 300 seconds.', 'status': 400, 'success': False}
response3: {'result': {}, 'message': 'You have attempted to login 3 times, with no success. Your account is locked for 300 seconds.', 'status': 400, 'success': False}
response4: {'result': {}, 'message': 'You have attempted to login 3 times, with no success. Your account is locked for 300 seconds.', 'status': 400, 'success': False}

[ericls]

Does flushing redis after each run help?