ArcticDB
ArcticDB copied to clipboard
Empty type decoding: move the segment buffer forward by the compressed data size for empty types
Reference Issues/PRs
There is a compatibility issue with columns containing empty types written in 4.0.x or earlier. For version 4.0.x, the buffer was remaining behind the correct position which was causing an lz4 decoding error.
What does this implement or fix?
Any other comments?
Checklist
Checklist for code changes...
- [ ] Have you updated the relevant docstrings, documentation and copyright notice?
- [ ] Is this contribution tested against all ArcticDB's features?
- [ ] Do all exceptions introduced raise appropriate error messages?
- [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes?
We should add a test that reproduces the original problem (ie writes with 4.0.x and updates with HEAD).
We should add a test that reproduces the original problem (ie writes with 4.0.x and updates with HEAD).
Test should look something like this:
# version 4.0.3
df = pd.DataFrame({'c1': [None, None, None], 'c2': [1.2, None, 1.3]}, index=pd.date_range('1/1/2018', end='1/03/2018'))
lib.write("mysym", df)
# version latest master
df = pd.DataFrame({'c1': [None, None, None], 'c2': [1.1, None, 1.4]}, index=pd.date_range('1/1/2018', end='1/03/2018'))
lib.update("mysym", df)
...
arcticdb_ext.exceptions.InternalException: E_INVALID_ARGUMENT Error while decoding with lz4 at address 7eacd8d4efea with size 21. Code -4 // this error shows failure in decoding something written with 4.0.3
Superseded by another PR