feat: various compression fixes for 2.1, add support for per-value block compression

[westonpace]

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 10.76923% with 116 lines in your changes missing coverage. Please review.

Project coverage is 78.75%. Comparing base (8a61b69) to head (5aa84ff). Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...-encoding/src/encodings/physical/block_compress.rs	0.00%	104 Missing :warning:
rust/lance-encoding/src/format.rs	0.00%	7 Missing :warning:
rust/lance-encoding/src/decoder.rs	0.00%	2 Missing :warning:
rust/lance-encoding/src/encoder.rs	85.71%	1 Missing and 1 partial :warning:
rust/lance-encoding/src/encodings/physical.rs	0.00%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3448      +/-   ##
==========================================
- Coverage   78.93%   78.75%   -0.19%     
==========================================
  Files         251      251              
  Lines       92267    92951     +684     
  Branches    92267    92951     +684     
==========================================
+ Hits        72833    73202     +369     
- Misses      16463    16773     +310     
- Partials     2971     2976       +5     

Flag	Coverage Δ
unittests	78.75% <10.76%> (-0.19%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[westonpace]

Superceded by https://github.com/lancedb/lance/pull/3488