petastorm icon indicating copy to clipboard operation
petastorm copied to clipboard

Call stack is not reported if an error occurs on a worker thread (when using workers_pool.ThreadPool)

Open selitvin opened this issue 6 years ago • 1 comments

For example, this is a call stack that does not indicate where ValueError is actually occurring.

petastorm/reader.pyc in next(self)
    588
    589     def next(self):
--> 590         return self.__next__()
    591
    592     # Functions needed to treat reader as a context manager

petastorm/reader.pyc in __next__(self)
    582     def __next__(self):
    583         try:
--> 584             return self._results_queue_reader.read_next(self._workers_pool, self.schema, self.ngram)
    585         except StopIteration:
    586             self.last_row_consumed = True

petastorm/py_dict_reader_worker.pyc in read_next(self, workers_pool, schema, ngram)
     58                 # Reverse order, so we can pop from the end of the list in O(1) while maintaining
     59                 # order the items are returned from the worker
---> 60                 rows_as_dict = list(reversed(workers_pool.get_results()))
     61
     62                 if ngram:

petastorm/workers_pool/thread_pool.pyc in get_results(self)
    170                     self.stop()
    171                     self.join()
--> 172                     raise result
    173                 else:
    174                     return result

ValueError: min() arg is an empty sequence

selitvin avatar Dec 18 '18 01:12 selitvin

Claiming this issue (assuming it's still open/valid).

jsgoller1 avatar Jun 17 '20 22:06 jsgoller1