tribler icon indicating copy to clipboard operation
tribler copied to clipboard

Lz4 does not compress well enough.

Open kotenok2000 opened this issue 4 years ago • 1 comments

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Populate channels folder with data (subscribe to channels)
  2. navigate to channels folder via cmd
  3. Run compact /c /s /a /i /exe:lzx
  4. See that it was able to compress files 1,3 to 1 that have already were compressed with lz4

Expected behavior Files are not able to be compressed further Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: Windows 11
  • Tribler's version: 7.11.0

Additional context Add any other context about the problem here.

kotenok2000 avatar Jan 07 '22 14:01 kotenok2000

We're using LZ4 streaming compression feature to fit data into UDP packets and file-based chunks. Streaming mode is less efficient, but it allows us to avoid backtracking (e.g. when adding the last entry will overflow the chunk size). That said, fitting the chunks tightly is not a priority when dumping them to disk. However, using the same code for both network and torrent-based transfers is.

ichorid avatar Jan 07 '22 22:01 ichorid