File Position limited to UInt32 ?

[ThummeTo]

When adding many larger files via addfile, I run into an exception:

ERROR: LoadError: InexactError: trunc(UInt32, 4320522198)
Stacktrace:
  [1] throw_inexacterror(f::Symbol, #unused#::Type{UInt32}, val::Int64)
    @ Core .\boot.jl:614
  [2] checked_trunc_uint
    @ .\boot.jl:644 [inlined]
  [3] toUInt32
    @ .\boot.jl:728 [inlined]
  [4] UInt32(x::Int64)
    @ Core .\boot.jl:768
  [5] addfile(w::ZipFile.Writer, name::String; method::UInt16, mtime::Float64)
    @ ZipFile C:\Users\thummeto\.julia\packages\ZipFile\6aqG6\src\ZipFile.jl:610

The corresponding codeline looks indeed like this:

f = WritableFile(w._io, name, UInt16(method), dostime, dosdate,
        UInt32(0), UInt32(0), UInt32(0), UInt32(position(w._io)),
        Int64(-1), w._io, false)

So position(w._io) is limited to UInt32. Is this a design limit or should it be possible to zip larger files?

Thanks and best regards!

[ssingh13-rms]

@ThummeTo I ran into the same issue trying to ZIP a large CSV (5+ GB). I've opened #99 that should hopefully fix it but I haven't tested it thoroughly. Would you be able to run it for your file? We can work together to increase support for larger files! CC: @fhs

[ThummeTo]

I can check that soon (best think would be to define a MWE that writes large nonsense-files). I don't know if it's at easy as replacing Uint32 with UInt64, propably there are limitations in the underlaying library to Int32 ... but why not try it like in your PR ...

[nhz2]

To save larger files, ZIP64 needs to be used, see https://en.wikipedia.org/wiki/ZIP_(file_format)#ZIP64

Just replacing the UInt32 with UInt64 won't work. #88 Added support for reading ZIP64, but not writing yet. So ZIP64 support for writing still needs to be worked on.