problem-solving icon indicating copy to clipboard operation
problem-solving copied to clipboard

Add `.bufs`/`.blobs` methods to `IO::Path` / `IO::Handle`

Open lizmat opened this issue 1 year ago • 6 comments

Just as .lines provides a lazy Seq of lines, I think it would be worthwhile to have a lazy Seq of buf8/blob8 objects of a file. This could be helpful in some workflows where serially decoding a (large) file is the bottleneck. Using .blobs would give the opportunity to only need to decode those parts one is actually interested in.

lizmat avatar Feb 18 '24 11:02 lizmat

But how would it decide how big the blobs should be?

Leont avatar Feb 19 '24 18:02 Leont

Whatever IO::Handle.READ returns :-)

lizmat avatar Feb 19 '24 18:02 lizmat

Whatever IO::Handle.READ returns :-)

Does that use read(2) or fread(3) semantics? I have a memory of it doing the latter, which would be unusable; I may be mistaken.

Basically, does it retry when getting less than $wanted bytes or not. It truly shouldn't.

Leont avatar Feb 19 '24 23:02 Leont

nqp::readfh($!PIO,nqp::create(buf8.^pun),$bytes)

I'm not sure about the semantics, but probably with retry. Not sure what the backends provide.

lizmat avatar Feb 20 '24 09:02 lizmat

FWIW, I think an iterator should use READ semantics.

OTOH, a supply based on asyncreadbytes would be the truly lazy approach.

lizmat avatar Feb 20 '24 10:02 lizmat

Basically, does it retry when getting less than $wanted bytes or not. It truly shouldn't.

This is another example of such an issue.

Leont avatar Feb 21 '24 17:02 Leont