zstd icon indicating copy to clipboard operation
zstd copied to clipboard

real-time dictionary function

Open ppsnumi opened this issue 2 years ago • 4 comments

Is your feature request related to a problem? Please describe. I hope zstd's dictionary creation/analysis function works in real time. The dictionary function operates only through analysis of existing data. This is not suitable for services that use zstd in real time.

Describe the solution you'd like It would be nice if the dictionary function managed in real time was added through other options of zstd. However, since many zstd processes can read/write a dictionary file at the same time, concurrency control seems very important.

Describe alternatives you've considered If the real-time dictionary is not supported, I will have to analyze the data the day before and apply the dictionary file the day before to the real-time data.

Additional context The real-time dictionary function doesn't work as well as the current dictionary function, nor is it as desirable, but it would be a great feature for fast compression/decompression that zstd seeks. In the beginning, even if it only operates in the form of continuously adding new data of the dictionary file, it is inefficient than the existing dictionary file structure, and the difficulty in producing this function seems to be lowered.

thank you

  • It was written with Google Translate.

ppsnumi avatar Apr 12 '23 04:04 ppsnumi