archive
archive copied to clipboard
Support creating InputStreamBase instance from Sink
Recently the ArchiveFile.stream() method changed from taking a List<int> to an InputStreamBase instance making the stream() method more flexible. So, the data can be feed from List<int> using an InputStream from from a file using InputFileStream.
In addition to in-memory buffer and a file it would be also great to be able to create an InputStreamBase from a Sink instance. E.g. to allow chaining encoders, something like SinkInputStream(utf8.encoder.startChunkedConversion(..)) and avoid loading an entire encoder conversion result in memory.
I don't know much about the Sink API, I haven't done much Dart programming other than supporting this library for a long time. For some decoders, they need random access. For decoding, zip stores its tables at the end of the file and needs random access to access the file entries within the zip; inflate decoding also needs random access to look back at previous codes.
For encoding, you could use an OutputFileStream to not store the entire encoding in memory. But that comes at a potentially significant cost that File IO is very expensive.
Memory vs performance, with an API that has the needed features, has been a really challenging problem. I won't have any time to research it further for a while, but am always open for pull requests.