acquire icon indicating copy to clipboard operation
acquire copied to clipboard

Prevent .tar file corruption by patching short reads

Open joost-j opened this issue 4 months ago • 2 comments

Fixes an issue where a .tar output file would contain inconsistencies with regards to expected and actual file size of the included files.

In some cases, a file on disk can report a size of X bytes, but at the time of actually reading X bytes from the file, less than X bytes are actually available in the file (a short read). Acquire would report these issues as OSError in the resulting Acquisition log file, because the Python stdlib tarfile.py handles it that way. Data may however already be written to the destination archive at that point.

Afterwards, Acquire continues to add new files to the archive. When trying to untar the file using tar -xvf <FILE> this would show as a tar: Skipping to next header error and finally, the process exists with a nonzero exit code.

Included a test case which simulates a file that actually returns less bytes than its reported size, to test this case.

joost-j avatar Aug 12 '25 14:08 joost-j

Codecov Report

:x: Patch coverage is 79.41176% with 7 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 45.20%. Comparing base (c243af8) to head (00a211c). :warning: Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
acquire/outputs/tar.py 79.41% 7 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #261      +/-   ##
==========================================
+ Coverage   44.93%   45.20%   +0.26%     
==========================================
  Files          26       26              
  Lines        3527     3568      +41     
==========================================
+ Hits         1585     1613      +28     
- Misses       1942     1955      +13     
Flag Coverage Δ
unittests 45.20% <79.41%> (+0.26%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Aug 12 '25 14:08 codecov[bot]

Any idea when this is getting fixed? It's affecting me as well. Let me know if there's anything I can do to help!

qmadev avatar Sep 09 '25 18:09 qmadev