petastorm
petastorm copied to clipboard
Call stack is not reported if an error occurs on a worker thread (when using workers_pool.ThreadPool)
For example, this is a call stack that does not indicate where ValueError is actually occurring.
petastorm/reader.pyc in next(self)
588
589 def next(self):
--> 590 return self.__next__()
591
592 # Functions needed to treat reader as a context manager
petastorm/reader.pyc in __next__(self)
582 def __next__(self):
583 try:
--> 584 return self._results_queue_reader.read_next(self._workers_pool, self.schema, self.ngram)
585 except StopIteration:
586 self.last_row_consumed = True
petastorm/py_dict_reader_worker.pyc in read_next(self, workers_pool, schema, ngram)
58 # Reverse order, so we can pop from the end of the list in O(1) while maintaining
59 # order the items are returned from the worker
---> 60 rows_as_dict = list(reversed(workers_pool.get_results()))
61
62 if ngram:
petastorm/workers_pool/thread_pool.pyc in get_results(self)
170 self.stop()
171 self.join()
--> 172 raise result
173 else:
174 return result
ValueError: min() arg is an empty sequence
Claiming this issue (assuming it's still open/valid).