mongo-connector icon indicating copy to clipboard operation
mongo-connector copied to clipboard

StopIteration Runtime Exception with Python 3.8

Open jazzdup opened this issue 3 years ago • 0 comments

`2021-04-22 07:36:48,231 [CRITICAL] mongo_connector.oplog_manager:713 - Exception during collection dump Traceback (most recent call last): File "/home/al/dev/zapper/projects/etl2/lib/python3.8/site-packages/mongo_connector/doc_managers/solr_doc_manager.py", line 292, in batch = list(next(cleaned) for i in range(self.chunk_size)) StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/al/dev/zapper/projects/etl2/lib/python3.8/site-packages/mongo_connector/oplog_manager.py", line 668, in do_dump upsert_all(dm) File "/home/al/dev/zapper/projects/etl2/lib/python3.8/site-packages/mongo_connector/oplog_manager.py", line 651, in upsert_all dm.bulk_upsert(docs_to_dump(from_coll), mapped_ns, long_ts) File "/home/al/dev/zapper/projects/etl2/lib/python3.8/site-packages/mongo_connector/util.py", line 33, in wrapped return f(*args, **kwargs) File "/home/al/dev/zapper/projects/etl2/lib/python3.8/site-packages/mongo_connector/doc_managers/solr_doc_manager.py", line 292, in bulk_upsert batch = list(next(cleaned) for i in range(self.chunk_size)) RuntimeError: generator raised StopIteration 2021-04-22 07:36:48,232 [ERROR] mongo_connector.oplog_manager:723 - OplogThread: Failed during dump collection cannot recover!`

code fix is in solr_doc_manager.py, replace 2x instances of: batch = list(next(cleaned) for i in range(self.chunk_size)) with batch = [] for i in range(self.chunk_size): for x in cleaned: batch.append(x)

jazzdup avatar Apr 22 '21 09:04 jazzdup