fakeredis
fakeredis copied to clipboard
jamesls
Implementing "tick" to avoid using sleep
Great work on this library. Just wondering, have you considered adding a tick method or some sort of frozen time mode to FakeRedis that allows programmatic advancing of time to avoid using an actual sleep in the test code?
freezegun can be used for this purpose and works like a charm with fakeredis because the later uses the stdlib datetime module to determine if keys are expired. Feels like it's not something that should be implemented in this package.
Thanks, I haven't come across freezegun before. From a quick look at the docs it appears to mock the functions that get time, but not time.sleep - in which case it would presumably cause an infinite loop since the timeout would never be reached?
@bmerry DataBase.time is retrieved from time.time() on command processing
https://github.com/jamesls/fakeredis/blob/e04fc6e24baaaceb582e95a3d61b63d34a9e634d/fakeredis/_server.py#L818-L820
This method is mocked by freezegun which means tests such as
https://github.com/jamesls/fakeredis/blob/dcf0c8933fff7b3e0f08879a9905dea4f67ca750/test/test_fakeredis.py#L3994-L4001
Could be rewritten as
@freeze_time(as_kwarg='frozen_time')
def test_expireat_should_expire_key_by_timestamp(r, frozen_time):
r.set('foo', 'bar')
assert r.get('foo') == b'bar'
r.expireat('foo', int(time() + 1))
frozen_time.tick(1.5)
assert r.get('foo') is None
assert r.expire('bar', 1) is False
In the case of internal usages of time.sleep
https://github.com/jamesls/fakeredis/blob/dcf0c8933fff7b3e0f08879a9905dea4f67ca750/fakeredis/_server.py#L2592
They can be mocked with mock.patch('time.sleep', frozen_time.tick). There even seems to be a solution for asyncio.sleep https://github.com/spulec/freezegun/issues/290
I think one problem is that the tests run against both real redis and fake redis to ensure that the tests have correct expectations, and of course freezegun won't work with real redis. But I can probably build a pytest fixture to do the mocking in the appropriate cases, which would at least halve the sleep time in tests. I'll take a look at some point when I have free time - thanks for the suggestion.
FYI I built a faster alternative to freezegun called time-machine. See https://adamj.eu/tech/2020/06/03/introducing-time-machine/ .
@adamchainz sadly I'm trying this right now and does not seem to work 😢
def test_cache_expiration(self,...):
frozen_time = pytz.UTC.localize(datetime.datetime(2023, 11, 23, 14, 52))
with time_machine.travel(frozen_time, tick=False) as travel_time:
print(datetime.datetime.now())
print(cache.has_key('XXX'))
print(cache.pttl('XXX'))
...
# some caching with django cache and fakeredis
# some asserts...
...
before_expire_delta = datetime.timedelta(seconds=(CACHE_TIEMOUT - 10))
travel_time.shift(before_expire_delta)
print(datetime.datetime.now())
print(cache.has_key('XXX'))
print(cache.pttl('XXX'))
...
# some asserts
...
after_expire_delta = datetime.timedelta(seconds=(CACHE_TIEMOUT + 10))
travel_time.shift(after_expire_delta)
print(datetime.datetime.now())
print(cache.has_key('XXX'))
print(cache.pttl('XXX'))
2023-11-23 14:52:00
False
0
...
2023-11-23 14:56:50
True
299999
...
2023-11-23 15:02:00
True
299991
...
@slothyrulez Hmm I am afraid I can't see why exactly. It looks like fakeredis (the new maintained repo) calls time.time() on each command: https://github.com/cunla/fakeredis-py/blob/04d4703d6bf7bc7c9d98d2d128cd206de80787b3/fakeredis/_basefakesocket.py#L271
time-machine definitely mocks time.time() just fine. If you add time.time() calls in your test they should show updates just like datetime.now().
Maybe the above code path isn't running, so the server time variable is not updating correctly. Calling the time command would probably be instructive.
Ummmmm, should I open a discussion on time-machine or in fakeredis ?
Definitively, thanks for your response @adamchainz
I think this is an issue with how fakeredis reads the time. I’d only want a time-machine issue if you can show there’s an underlying cause that needs a fix.
@slothyrulez Hmm I am afraid I can't see why exactly. It looks like fakeredis (the new maintained repo) calls
time.time()on each command: https://github.com/cunla/fakeredis-py/blob/04d4703d6bf7bc7c9d98d2d128cd206de80787b3/fakeredis/_basefakesocket.py#L271time-machine definitely mocks
time.time()just fine. If you addtime.time()calls in your test they should show updates just likedatetime.now().Maybe the above code path isn't running, so the server
timevariable is not updating correctly. Calling thetimecommand would probably be instructive.
BEFORE time_machine.travel --
datetime.datetime.now()=datetime.datetime(2023, 11, 24, 6, 1, 23, 8122)
time.time()=1700805683.0081432
cache.client.get_client().time()=(1700805683, 8477)
cache.has_key("XXXX")=False
cache.pttl("XXXX")=0
AFTER time_machine.travel --
datetime.datetime.now()=datetime.datetime(2023, 11, 23, 14, 52)
time.time()=1700751120.0
cache.client.get_client().time()=(1700805683, 16603)
cache.has_key("XXXX")=False
cache.pttl("XXXX")=0
--
CACHED DURING 300 secs (5min.)
--
TRAVEL 4m50s to the future
datetime.datetime.now()=datetime.datetime(2023, 11, 23, 14, 56, 50)
time.time()=1700751410.0
cache.client.get_client().time()=(1700805683, 38205)
cache.has_key("XXXX")=True
cache.pttl("XXXX")=299999
--
TRAVEL 15s to the future
datetime.datetime.now()=datetime.datetime(2023, 11, 23, 14, 57, 5)
time.time()=1700751425.0
cache.client.get_client().time()=(1700805683, 47826)
cache.has_key("XXXX")=True
cache.pttl("XXXX")=299989
Cross-posting here for future visibility https://github.com/cunla/fakeredis-py/issues/253 Thanks @jamesls and sorry for the noise