Greg Lindahl
Greg Lindahl
I agree that this is a real bug. I don't think I have this tested in the "warcio check" branch, either! If you look at the WARC standard it's a...
Sebastian, most Unix things do print the EPIPE, and it's sometimes very helpful when the indexing process has a separate error logfile from the other end of the pipe. What...
I think I'm remembering EPIPE being useful for things like scp and other network tools. But you're right, normal on-node tools like `head` have always been silent about EPIPE. This...
It's not obvious, but this is the same as #64 -- and we should fix it while keeping warcio's ability to do streaming.
This is by design. I agree that this isn't obvious and that we can improve the documentation and runtime error messages for this case. What you should do instead is...
Yes, please leave it open, this is not the only place where we have a lack of clarity about streaming vs files.
I see an endless loop possibility involving a truncated file, are you reading the last record when the problem happens?
I'm happiest to debug this if you send us the warc with the problem. I could explore the possible bug I found, but really, solving your actual bug is probably...
Can you prepare and attach a cut-down warc that starts at 32624067591 and is long enough to have 2 complete records plus a fair bit of the 3rd? Compressed the...
This is a not unusual situation in the face of a hardware or software bug. Either the file ends in a hole, which means the filesystem metadata is corrupt, or...