Pipe to pxz? Can the parallel algorithm support stdin without eating RAM?

[marcominetti]

I'd like to do something like:

dd if=/dev/sda bs=1M | pxz -kvc -9 -T8 > out.lzma

I've tried and work but eats lots of ram. Is there any optimization that could be implemented?

[jurov]

XZ -9 just needs 674 MiB per thread, that's hardly avoidable. Use lower compression level.

[marcominetti]

Thanks

[JoSys]

You may want to buffer both input and output like so: dd if=/dev/sda conv=sync,noerror bs=16M | pxz -kvc -9 -T8 | dd of=out.xz bs=16M With a SSD you might want to use a bigger buffer. Anyway, just use pv command in-between pipes to check out if performance improves.

[mvrk69]

With pv would be like this?

dd if=/dev/sda conv=sync,noerror bs=16M | pv -s <size_of_sda> | pxz -kvc -9 -T8 | dd of=out.xz bs=16M