datachain icon indicating copy to clipboard operation
datachain copied to clipboard

Fix pickle/unpickle FileError exception

Open dreadatour opened this issue 7 months ago • 2 comments

Fix for https://github.com/iterative/datachain/issues/1125

  File "/Users/vlad/.virtualenvs/datachain/lib/python3.13/site-packages/multiprocess/queues.py", line 138, in get_nowait
    return self.get(False)
           ~~~~~~~~^^^^^^^
  File "/Users/vlad/.virtualenvs/datachain/lib/python3.13/site-packages/multiprocess/queues.py", line 125, in get
    return _ForkingPickler.loads(res)
           ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/Users/vlad/.virtualenvs/datachain/lib/python3.13/site-packages/dill/_dill.py", line 303, in loads
    return load(file, ignore, **kwds)
  File "/Users/vlad/.virtualenvs/datachain/lib/python3.13/site-packages/dill/_dill.py", line 289, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Users/vlad/.virtualenvs/datachain/lib/python3.13/site-packages/dill/_dill.py", line 444, in load
    obj = StockUnpickler.load(self)
TypeError: FileError.__init__() missing 1 required positional argument: 'message'

dreadatour avatar May 29 '25 16:05 dreadatour

Deploying datachain-documentation with  Cloudflare Pages  Cloudflare Pages

Latest commit: a1806dc
Status: ✅  Deploy successful!
Preview URL: https://2d4dce7b.datachain-documentation.pages.dev
Branch Preview URL: https://fix-file-error-pickling.datachain-documentation.pages.dev

View logs

Codecov Report

Attention: Patch coverage is 41.37931% with 17 lines in your changes missing coverage. Please review.

Project coverage is 88.61%. Comparing base (648cf44) to head (a1806dc). Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
src/datachain/lib/file.py 36.36% 14 Missing :warning:
src/datachain/lib/video.py 25.00% 3 Missing :warning:
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1126      +/-   ##
==========================================
- Coverage   88.65%   88.61%   -0.04%     
==========================================
  Files         148      148              
  Lines       12853    12862       +9     
  Branches     1810     1810              
==========================================
+ Hits        11395    11398       +3     
- Misses       1033     1039       +6     
  Partials      425      425              
Flag Coverage Δ
datachain 88.54% <41.37%> (-0.04%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/datachain/lib/image.py 76.92% <100.00%> (ø)
src/datachain/lib/utils.py 91.66% <100.00%> (-0.44%) :arrow_down:
src/datachain/lib/video.py 79.76% <25.00%> (ø)
src/datachain/lib/file.py 90.49% <36.36%> (-1.21%) :arrow_down:
:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar May 29 '25 16:05 codecov[bot]

@dreadatour is it good to go? do we need a test?

shcheklein avatar Jun 02 '25 19:06 shcheklein