bit7z icon indicating copy to clipboard operation
bit7z copied to clipboard

Is there a method to set the number of cpu thread for 7z compressing?

Open LARVM opened this issue 4 years ago • 7 comments

LARVM avatar Nov 07 '19 03:11 LARVM

Unfortunately bit7z does not currently support this feature. However, I expect to be able to include it in the next stable version (v4.0) which is in the works.

rikyoz avatar Nov 09 '19 12:11 rikyoz

  1. I adjust the BufferUpdateCallback class to support many file stream compressing,which refines the class of FileUpdateCallback. Curently it work, but I want to know, is there any problem? And would you add this kind of method?
  2. A fast compress method is our product urgent need, so I will also try to implement muti-threaded compressing. Would you remind me some ideas to carry out this work? I know that there is a property of "mtf", which can set 7-zip opening muti-thread, but it is not useful when i add it in method of BitArchiveCreator::setArchiveProperties.

LARVM avatar Nov 11 '19 02:11 LARVM

  1. I adjust the BufferUpdateCallback class to support many file stream compressing,which refines the class of FileUpdateCallback.

The BufferUpdateCallback class is meant to be used when the input is a single buffer and you want it to be compressed to a filesystem archive file, another buffer, or a standard stream. I'm sorry, but I probably don't understand what you mean by "support many file stream compressing". Multiple buffers compressing?

Curently it work, but I want to know, is there any problem?

There is no problem, feel free to open a pull request and I'll review it asap.

And would you add this kind of method?

I still don't quite catch what functionality you're trying to add, so I'll be able tell you only when I will.

  1. A fast compress method is our product urgent need, so I will also try to implement muti-threaded compressing. Would you remind me some ideas to carry out this work? I know that there is a property of "mtf", which can set 7-zip opening muti-thread, but it is not useful when i add it in method of BitArchiveCreator::setArchiveProperties.

As far as I know, the property "mtf" is useful for setting the multithreading mode only for the filters of the 7z format. For enabling the complete multithreading mode the correct property should be "mt" (or at least this is what I understand from the command line documentation: https://sevenzip.osdn.jp/chm/cmdline/switches/method.htm). I didn't test this, but it is the way I would try in implementing this. Let me know if you are successful and feel free to open a pull request, if you want!

rikyoz avatar Nov 11 '19 21:11 rikyoz

I found the reason why 7z couldn't multithread when I compressed multiple files. Just need to remove the setting of SOLID_ARCHIVE. image

LARVM avatar Nov 24 '19 14:11 LARVM

I found the reason why 7z couldn't multithread when I compressed multiple files. Just need to remove the setting of SOLID_ARCHIVE. image

Interesting: by default, the "solid archive" mode is set to false; however, if I understand correctly, 7-zip cannot exploit the multi-threading all the same, right? Maybe explicitly specifying the "s" property disables it, or the property must be set to true to make multiple threads to work, I don't know. Anyway, I cannot remove those lines of code for obvious reasons (they are needed to support compression to solid archives). Possibly, in the future, that "if" will check if multi-threading is requested or not, and enable/disable solid compression accordingly.

rikyoz avatar Nov 26 '19 20:11 rikyoz

Possibly, in the future, that "if" will check if multi-threading is requested or not, and enable/disable solid compression accordingly.

Please don't do that.

It should be the bit7z library user's responsibility to know what 7-Zip options work together.

If I wanted to make a solid archive, but I also specified multi-threading, I'd like to get the same behavior from the library that I would get from 7-Zip File Manager.

In my opinion, it is bad to create API whose behavior differs from the UI.

levicki avatar Mar 23 '20 09:03 levicki

Please don't do that.

It should be the bit7z library user's responsibility to know what 7-Zip options work together.

If I wanted to make a solid archive, but I also specified multi-threading, I'd like to get the same behavior from the library that I would get from 7-Zip File Manager.

In my opinion, it is bad to create API whose behavior differs from the UI.

Undoubtedly you've raised some very good points. However, the problem here, as far as I understand it, is that whether the solid archive option is set to false or true, in all cases multi-threading is unavailable! This issue may be caused by the fact that the option for solid compression is always set explicitly, rather than only when the user set it to true (a change that I did some time ago). At least, this is the only reason I can think of for the need of removing those lines, instead of simply call BitArchiveCreator::setSolidMode(true). I haven't had a chance yet to test if this is the problem, anyway I will study the default behavior of 7-zip and try to provide a reasonable API for both multi-threading and solid compression.

rikyoz avatar Mar 25 '20 21:03 rikyoz

Implemented in v4.0.0.

rikyoz avatar Oct 01 '23 19:10 rikyoz