expiringdict
expiringdict copied to clipboard
RuntimeError with Python3.5
I am seeing a very annoying bug when using this library with Python 3.5:
In [1]: import sys
...:
...: # Python 3.5 has a broken 'optimised' version of OrderedDict which can corrupt itself at high load.
...: if sys.version_info.major == 3 and sys.version_info.minor == 5:
...: from test import support
...:
...: py_coll = support.import_fresh_module('collections', blocked=['_collections'])
...: OrderedDict = py_coll.OrderedDict
...: else:
...: from collections import OrderedDict
...:
In [2]: from expiringdict import ExpiringDict
In [3]: ExpiringDict(max_len=60, max_age_seconds=4)
Out[3]: ExpiringDict()
In [4]: d=ExpiringDict(max_len=60, max_age_seconds=4)
In [5]: d["soc3"] = {"soc":14}
In [6]: d["soc2"] = {"soc":11}
In [7]: d
Out[7]: ---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
~/.virtualenvs/hagenv2g/lib/python3.5/site-packages/IPython/core/formatters.py in __call__(self, obj)
670 type_pprinters=self.type_printers,
671 deferred_pprinters=self.deferred_printers)
--> 672 printer.pretty(obj)
673 printer.flush()
674 return stream.getvalue()
~/.virtualenvs/hagenv2g/lib/python3.5/site-packages/IPython/lib/pretty.py in pretty(self, obj)
366 if cls in self.type_pprinters:
367 # printer registered in self.type_pprinters
--> 368 return self.type_pprinters[cls](obj, self, cycle)
369 else:
370 # deferred printer
~/.virtualenvs/hagenv2g/lib/python3.5/site-packages/IPython/lib/pretty.py in _ordereddict_pprint(obj, p, cycle)
829 p.text('...')
830 elif len(obj):
--> 831 p.pretty(list(obj.items()))
832
833 def _deque_pprint(obj, p, cycle):
~/.virtualenvs/hagenv2g/lib/python3.5/site-packages/expiringdict/__init__.py in items(self)
111 """ Return a copy of the dictionary's list of (key, value) pairs. """
112 r = []
--> 113 for key in self:
114 try:
115 r.append((key, self[key]))
RuntimeError: OrderedDict mutated during iteration
In items
and values
, the loop is like:
for key in self:
try:
r.append(self[key])
self[key] may trigger expiration--delete while iterating. Solution is to iterate over the copy of all keys:
for key in list(self.keys()):
try:
r.append(self[key])
@kophy Because this package seemed abandoned, I forked it and modified it. I also added some things to a similar package called ttldict. You can find updated packages in pypi:
https://pypi.python.org/pypi/ttldict/
The code is tested on all Python3 versions.
feel free to star the project in https://github.com/mobilityhouse/ttldict.
And BTW, sorry for posting here so late about the work done in parallel...