GAM icon indicating copy to clipboard operation
GAM copied to clipboard

Use PyInstaller OneDir for better GAM performance

Open jay0lee opened this issue 3 years ago • 0 comments

Historically GAM has used PyInstaller's OneFile feature because it looks a bit cleaner and results in fewer files for GAM to bundle. However what OneFile does EVERY time it's run is:

  • Extract it's contents to a temp folder on the system
  • execute from within the temp folder
  • cleanup the temp folder after execution completes with GAM batch operations this can amount to a HUGE amount of wasted I/O and time spent doing the same extract / run / cleanup thousands of times and hurts performance.

Some rough benchmarking shows PyInstaller's default One Folder mode can reduce GAM startup and execution time by .2 - .5 seconds which again is VERY significant when you are running thousands of GAM commands in batch modes.

Things to be considered:

  • The gam.spec file has already been modified to support onefile or onefolder builds, If an environment variable PYINSTALLER_BUILD_ONEFILE=yes is set GAM will build with OneFile, if not it will use OneDir.
  • Unfortunately it looks like StaticX, the tool GAM currently uses to make itself compatible with older Linux systems does not support OneDir. For this reason the Linux "legacy" builds will remain on OneFile. I expect between using OneFile and StaticX that these will perform much worse.
  • Packaging GAM installers needs updating. This shouldn't be very difficult for the .zip and .tar.bz2 packages as we're just adding more files to those but the Windows MSI needs to know about all of those OneDir files :-/ I have a few ideas of how that can be done but it's going to take some scripting work.

@taers232c fyi

jay0lee avatar Dec 01 '22 18:12 jay0lee