I'm using streamparse==3.16.0 and storm 1.1.3. When I try to run using python3.8.5 I get a RuntimeError with the following exception trace:
(VENV3) francis@francis-VirtualBox:~/Documents/repos/wordcount$ sparse run
Traceback (most recent call last):
File "/home/francis/Documents/repos/platform/VENV3/bin/sparse", line 8, in
sys.exit(main())
File "/home/francis/Documents/repos/platform/VENV3/lib/python3.8/site-packages/streamparse/cli/sparse.py", line 87, in main
args.func(args)
File "/home/francis/Documents/repos/platform/VENV3/lib/python3.8/site-packages/streamparse/cli/run.py", line 127, in main
run_local_topology(
File "/home/francis/Documents/repos/platform/VENV3/lib/python3.8/site-packages/streamparse/cli/run.py", line 46, in run_local_topology
topology_class = get_topology_from_file(topology_file)
File "/home/francis/Documents/repos/platform/VENV3/lib/python3.8/site-packages/streamparse/util.py", line 573, in get_topology_from_file
mod = importlib.import_module(mod_name)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 783, in exec_module
File "", line 219, in _call_with_frames_removed
File "topologies/wordcount.py", line 11, in
class WordCount(Topology):
File "topologies/wordcount.py", line 13, in WordCount
count_bolt = WordCountBolt.spec(inputs={word_spout: Grouping.fields("word")}, par=2)
File "/home/francis/Documents/repos/platform/VENV3/lib/python3.8/site-packages/streamparse/storm/bolt.py", line 192, in spec
return ShellBoltSpec(
File "/home/francis/Documents/repos/platform/VENV3/lib/python3.8/site-packages/streamparse/dsl/bolt.py", line 23, in init
super(ShellBoltSpec, self).init(
File "/home/francis/Documents/repos/platform/VENV3/lib/python3.8/site-packages/streamparse/dsl/component.py", line 267, in init
super(ShellComponentSpec, self).init(
File "/home/francis/Documents/repos/platform/VENV3/lib/python3.8/site-packages/streamparse/dsl/component.py", line 41, in init
self.inputs = self._sanitize_inputs(inputs)
File "/home/francis/Documents/repos/platform/VENV3/lib/python3.8/site-packages/streamparse/dsl/component.py", line 84, in _sanitize_inputs
for key, val in inputs.items():
RuntimeError: dictionary keys changed during iteration
Hi @frank-switchdin
The problem is you cannot iterate through a dictionary while it's changing during for loop. Casting the dictionary items to a list creates a list of its items, so you can iterate over it and avoid the RuntimeError.
I think this issue should be fixed with this commit 3368d61. You can try using the latest version 4.0.0 to avoid this error.
Thanks.
pystorm
