parquet2 icon indicating copy to clipboard operation
parquet2 copied to clipboard

Error running tests with `--all-features` (cannot find function `lz4_decompress_to_buffer`)

Open aldanor opened this issue 1 year ago • 1 comments

Here's just cloning the repo and trying to run the tests (note that the same error occurs if you try to run all tests for arrow2):

   Compiling parquet2 v0.17.0
error[E0425]: cannot find function `lz4_decompress_to_buffer` in this scope
   --> src/compression.rs:196:13
    |
196 |             lz4_decompress_to_buffer(input_buf, Some(output_buf.len() as i32), output_buf)
    |             ^^^^^^^^^^^^^^^^^^^^^^^^ not found in this scope

error[E0425]: cannot find function `lz4_decompress_to_buffer` in this scope
   --> src/compression.rs:265:33
    |
265 |         let decompressed_size = lz4_decompress_to_buffer(
    |                                 ^^^^^^^^^^^^^^^^^^^^^^^^ not found in this scope

I understand this has to do with lz4_flex and inconsistent feature sets, but having --all-features cause invalid feature sets is not very nice if it can be avoided.

lz4_decompress_to_buffer() is only defined for lz4 && !lz4_flex or for !lz4_flex && lz4, however that's not what the match checks. Perhaps lz4_flex && lz4 should be treated as lz4_flex for all intents and purposes? (so that --all-features would make sense)

Here's what the match checks:

        #[cfg(all(feature = "lz4_flex", not(feature = "lz4")))]
        Compression::Lz4Raw => lz4_flex::block::decompress_into(...),
        #[cfg(feature = "lz4")]
        Compression::Lz4Raw => lz4::block::decompress_to_buffer(...),
        #[cfg(all(not(feature = "lz4"), not(feature = "lz4_flex")))]
        Compression::Lz4Raw => Err(Error::FeatureNotActive(...)),
        #[cfg(any(feature = "lz4_flex", feature = "lz4"))]
        Compression::Lz4 => try_decompress_hadoop(...),
        // ^ lz4_decompress_to_buffer() also used here, error!
        #[cfg(all(not(feature = "lz4_flex"), not(feature = "lz4")))]
        Compression::Lz4 => Err(Error::FeatureNotActive(...)),

(I know it's a breaking change but, just saying, another option is converting a feature, like lz4_flex, into a 'negative' feature, so that --all-features wouldn't enable it by default)

aldanor avatar Dec 24 '22 10:12 aldanor