Prevent .tar file corruption by patching short reads
Fixes an issue where a .tar output file would contain inconsistencies with regards to expected and actual file size of the included files.
In some cases, a file on disk can report a size of X bytes, but at the time of actually reading X bytes from the file, less than X bytes are actually available in the file (a short read). Acquire would report these issues as OSError in the resulting Acquisition log file, because the Python stdlib tarfile.py handles it that way. Data may however already be written to the destination archive at that point.
Afterwards, Acquire continues to add new files to the archive. When trying to untar the file using tar -xvf <FILE> this would show as a tar: Skipping to next header error and finally, the process exists with a nonzero exit code.
Included a test case which simulates a file that actually returns less bytes than its reported size, to test this case.
Codecov Report
:x: Patch coverage is 79.41176% with 7 lines in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.20%. Comparing base (c243af8) to head (00a211c).
:warning: Report is 2 commits behind head on main.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| acquire/outputs/tar.py | 79.41% | 7 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #261 +/- ##
==========================================
+ Coverage 44.93% 45.20% +0.26%
==========================================
Files 26 26
Lines 3527 3568 +41
==========================================
+ Hits 1585 1613 +28
- Misses 1942 1955 +13
| Flag | Coverage Δ | |
|---|---|---|
| unittests | 45.20% <79.41%> (+0.26%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
Any idea when this is getting fixed? It's affecting me as well. Let me know if there's anything I can do to help!