rust-linereader icon indicating copy to clipboard operation
rust-linereader copied to clipboard

Way to differentate file that ends without a newline vs a line that was truncated to the buffer size

Open dimo414 opened this issue 3 years ago • 0 comments

When next_batch() returns a slice that doesn't end in the delimiter there's no easy way to tell whether this is because there's nothing more to read or because the line is larger than the buffer.

  • You can call next_*() once more to see if it returns None, but you need to copy the previously returned line(s) before you can do so. This is tedious to get right (e.g. bstr does something like this).
  • You can configure the reader's capacity (since there isn't a capacity() method on LineReader) and then check if the returned string is the same size as that capacity. This is roundabout and can still have false-positives.

It would be great if it was apparent from the API whether the returned slice was incomplete or not, such as by returning an error or a different type that contained this bit.

Taking this a step further, would it be feasible/welcome to eliminate this limitation of LineReader (possibly as optional behavior)? For a caller that wants to support arbitrarily long lines there's really no option other than allocating enough memory to fit the whole line, so it seems like LineReader could just do this for the caller by resizing its buffer in response to overly-long lines.

I might be able to contribute some of these changes if there's interest.

dimo414 avatar Oct 03 '22 05:10 dimo414