mcap icon indicating copy to clipboard operation
mcap copied to clipboard

go: WriterOptions makes chunking configuration error-prone

Open wkalt opened this issue 1 year ago • 0 comments

Currently most options for the golang writer are of the form "SkipXXX", such that the field defaults to false as is the behavior of go, and XXX is the defaulted configuration. This is not the case for the "Chunked" setting. If you don't specify Chunked=true, you are going to get an unchunked file out.

There are two issues with this:

  • It's better to default to a chunked, and indexed file, than an unchunked file. The current behavior is an oversight, more than intentional. Copies and transfers over network will be faster, space on disk less, and remote access is much improved. There are use-cases for unchunked files but a typical user on a decent system should prefer chunked. So IMO we're doing the wrong thing with the current default.
  • It is easy to make the mistake of setting compression to a value, without setting Chunked=true. Currently this has the effect of doing nothing - there's no chunks so there's nothing to compress. This should probably return an error at construction time.

wkalt avatar Jul 21 '22 03:07 wkalt