parquet-go icon indicating copy to clipboard operation
parquet-go copied to clipboard

Use generics to generalize int32/int64/float32/float64 to number encoder/decoder/store

Open akrennmair opened this issue 2 years ago • 2 comments

This PR introduces Go 1.18 generics to generalize the separate int32/int64/float32/float64 implementation into a single number type with small specializations. This deduplicates a lot of similar code across these types.

Since this required a switch to Go 1.18, I also replaced interface{} with the new any type. I also migrated the existing fuzz tests for gofuzz with Go 1.18 fuzz tests.

And finally, I removed the vendor subdirectory as it caused issues and doesn't really make sense for a library.

akrennmair avatar Mar 20 '22 12:03 akrennmair

Looks like golangci-lint isn't quite mature enough to lint this code. Converting to draft for now...

akrennmair avatar Mar 20 '22 13:03 akrennmair

Codecov Report

Merging #83 (d41620e) into master (e6b70a8) will increase coverage by 0.53%. The diff coverage is 92.47%.

@@            Coverage Diff             @@
##           master      #83      +/-   ##
==========================================
+ Coverage   86.73%   87.27%   +0.53%     
==========================================
  Files          48       46       -2     
  Lines        9777     9444     -333     
==========================================
- Hits         8480     8242     -238     
+ Misses        939      884      -55     
+ Partials      358      318      -40     
Impacted Files Coverage Δ
cmd/parquet-tool/cmds/readfile.go 0.00% <0.00%> (ø)
floor/interfaces/unmarshaller.go 75.93% <75.00%> (ø)
helpers.go 70.73% <75.00%> (+0.49%) :arrow_up:
deltabp_decoder.go 65.85% <76.19%> (+6.94%) :arrow_up:
type_dict.go 73.25% <77.77%> (ø)
deltabp_encoder.go 64.17% <78.57%> (-8.67%) :arrow_down:
floor/interfaces/marshaller.go 88.57% <85.71%> (ø)
schema.go 74.58% <85.71%> (ø)
type_int96.go 78.18% <87.50%> (ø)
chunk_writer.go 84.09% <90.00%> (-0.79%) :arrow_down:
... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update e6b70a8...d41620e. Read the comment docs.

codecov-commenter avatar Mar 20 '22 20:03 codecov-commenter