housisong comments

Results 94 comments of


                                            housisong

A "CreateSignature" for diff tool would be nice

> a diff can be created without the original source to be present, only it's signature. hdiffpatch lib have a demo [hsynz](https://github.com/sisong/hsynz), only used new data's hash datas(signature), to create...

For help: I want to replace zlib with zstd in my project (like zsync) , it works now, but compress speed is quite slow.

Yes, I can use it; I tested still call ZSTD_compressStream2(s,...,ZSTD_e_flush), is run ok and same slow speed. over 99% CPU time in rebuild match tree, but the input prefix data(8M)...

For help: I want to replace zlib with zstd in my project (like zsync) , it works now, but compress speed is quite slow.

"it is an O(dictSize) operation" Yes, this is that's the reason for slow compressing. Thanks @embg "Random access comes to mind" Yes, I need ”Random access“ compressed blocks when decompress...

For help: I want to replace zlib with zstd in my project (like zsync) , it works now, but compress speed is quite slow.

Not really random access. My project is sync for large file，like rsync or **zsync**. **server**: the large file have **new version B**; clip B to blocks by block size(2K); `blocksInfoB`...

For help: I want to replace zlib with zstd in my project (like zsync) , it works now, but compress speed is quite slow.

Thank you for your reply and advice. @Cyan4973 Them have a cost, such as lowering the compression rate or using large of memory when decompressing. And zlib used O(2k) time...

For help: I want to replace zlib with zstd in my project (like zsync) , it works now, but compress speed is quite slow.

I tested zlib and zstd refPrefix compression speed ``` test file linux_5.19.9.src.tar, filesize 1269637120 byte compress by 4k block: zlib level 6 dict 16k, compressed size 233999388 byte time 25.5s...

For help: I want to replace zlib with zstd in my project (like zsync) , it works now, but compress speed is quite slow.

The point is not the level, but the complexity of the algorithm, O(dictSize) is not acceptable. Had to drop support for zstd in my project.

For help: I want to replace zlib with zstd in my project (like zsync) , it works now, but compress speed is quite slow.

Source code compiled at the same time by VC++ 2019 with O2 & NDEBUG ``` test file linux_5.19.9.src.tar, filesize 1269637120 byte, compress by 2k block: zlib level 6 dict 16k,...

For help: I want to replace zlib with zstd in my project (like zsync) , it works now, but compress speed is quite slow.

Because a similar function in zstd now is ZSTD_DCtx_refPrefix(dctx,prefix,dictSize); which need data be compressed with ZSTD_CCtx_refPrefix(cctx,prefix,dictSize) to match, and the time complexity of ZSTD_CCtx_refPrefix is O(dictSize), so total compress time...

For help: I want to replace zlib with zstd in my project (like zsync) , it works now, but compress speed is quite slow.

@Cyan4973 @felixhandte Thank you for your prompt and detailed response. I may have solved the issue, if no bugs were found. I tried to construct the compression code with local...