Decompression causes java.io.IOException: Pipe closed

[dmak]

trafficstars

I have created LZMA archive using LZMA command-line utility form Debian.

When some such LZMA files (one of them I've posted to RapidShare) the following exception occurs:

java.io.IOException: Pipe closed
    at java.io.PipedInputStream.checkStateForReceive(PipedInputStream.java:244)
    at java.io.PipedInputStream.awaitSpace(PipedInputStream.java:252)
    at java.io.PipedInputStream.receive(PipedInputStream.java:215)
    at java.io.PipedOutputStream.write(PipedOutputStream.java:132)
    at lzma.sdk.lz.OutWindow.flush(OutWindow.java:81)
    at lzma.sdk.lz.OutWindow.copyBlock(OutWindow.java:105)
    at lzma.sdk.lzma.Decoder.code(Decoder.java:369)
    at lzma.streams.LzmaDecoderWrapper.code(LzmaDecoderWrapper.java:60)
    at org.cservenak.streams.CoderThread$1.run(CoderThread.java:46)
    at org.cservenak.streams.CoderThread.run(CoderThread.java:91)

Reading code is roughly the following:

BufferedReader r = new BufferedReader(new InputStreamReader(new LzmaInputStream(new BufferedInputStream(new FileInputStream(testFile), 500 * 1024), new Decoder())));
r.readLine(); // in cycle

[cburroughs]

https://github.com/ning/jvm-compressor-benchmark/issues/1 suggested that this may be a swallowed OOM

[dmak]

I confirm: this is swallowed OOM. If this issue is a duplicate of issue#1, then it can be closed.

[anthony-o]

I had that same error with compression (using an LzmaOutputStream). I think the OOM should be thrown & displayed in the system error outputstream because "Pipe closed" is not really self explicit. I then used a medium dictionnary size but got the same problem. Perhaps the dictionnary size could be configured automatically by detecting the available memory ? Or perhaps there's a memory leak somewhere ? I then switch & use http://tukaani.org/xz/ which worked like a charm "out of the box".