BSON.jl icon indicating copy to clipboard operation
BSON.jl copied to clipboard

compress bson file with CodecZstd.jl ?

Open norci opened this issue 4 years ago • 1 comments

Can I save a compressed BSON file? such as using CodecZstd.jl ?

e.g. Pandas has a parameter compression='infer' https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html

norci avatar Feb 13 '21 07:02 norci

I have figured out:

julia> open("/tmp/a.bson.zstd", "w") do fd
           stream = ZstdCompressorStream(fd)
           BSON.@save(stream, x = "abcd"^4096)
           close(stream)
       end

julia> open("/tmp/a.bson.zstd", "r") do fd
           stream = ZstdDecompressorStream(fd)
           display(BSON.load(stream))
           close(stream)
       end
Dict{Symbol, Any} with 1 entry:
  :x => "abcdabcd…

shell> ls -lah /tmp/a.bson.zstd
-rw-r--r-- 1 coder coder 34 Feb 13 15:47 /tmp/a.bson.zstd

Shall we add this to the document?

norci avatar Feb 13 '21 07:02 norci