Use script name instead of project name for xref and dot files

[heuripedes]

Is your feature request related to a problem? Please describe.

When building multiple package bundle (MERGE) pyinstaller overwrites the xref and dot files, making it harder on the user/developer to analyze build problems.

Describe the solution you'd like

PyInstaller should use the pattern build/{project}/xref-{script}.html instead of build/{project}/xref-{project}.html and do the same for the dot file so they don't get overwritten by Analysis calls.

Describe alternatives you've considered

Manually rename xref and dot files after each Analysis step.

Additional context

The following spec file will create sample files in the current directory and use MERGE to merge dependencies. It will also make a copy and print out the crc32 checksum of the xref files generated during the build.

Run with pyinstaller --log-level=WARN --noconfirm --noupx complex.spec for clarity/speed.

# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.config import CONF
import binascii
import os

with open("complex.py", "w") as f:
    f.write("""
from jsonschema import Draft7Validator
print(Draft7Validator)
""")

with open("simple.py", "w") as f:
    f.write("""
import os
print("current working directory is", os.getcwd())
""")

block_cipher = None

a1 = Analysis(['complex.py'],
             pathex=['.'],
             binaries=[],
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)

with open(CONF['xref-file'], "rb") as f:
    print(f"{CONF['xref-file']} crc32:", binascii.crc32(f.read()) & 0xffffffff)
import shutil
shutil.copy(CONF['xref-file'], CONF['xref-file'] + "_complex.html")

a2 = Analysis(['simple.py'],
             pathex=['C:\\Code\\xrefbug'],
             binaries=[],
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)

with open(CONF['xref-file'], "rb") as f:
    print(f"{CONF['xref-file']} crc32:", binascii.crc32(f.read()) & 0xffffffff)
shutil.copy(CONF['xref-file'], CONF['xref-file'] + "_simple.html")

MERGE(
    (a1, 'complex', 'complex'),
    (a2, 'simple', 'simple')
)

pyz1 = PYZ(a1.pure, a1.zipped_data,
             cipher=block_cipher)
exe1 = EXE(pyz1,
          a1.scripts,
          [],
          exclude_binaries=True,
          name='complex',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=False,
          console=True )
coll1 = COLLECT(exe1,
               a1.binaries,
               a1.zipfiles,
               a1.datas,
               strip=False,
               upx=False,
               upx_exclude=[],
               name='complex')
pyz2 = PYZ(a2.pure, a2.zipped_data,
             cipher=block_cipher)
exe2 = EXE(pyz2,
          a2.scripts,
          [],
          exclude_binaries=True,
          name='simple',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=False,
          console=True )
coll2 = COLLECT(exe2,
               a2.binaries,
               a2.zipfiles,
               a2.datas,
               strip=False,
               upx=False,
               upx_exclude=[],
               name='simple')

try:
    os.remove("complex.py")
except:
    pass
try:
    os.remove("simple.py")
except:
    pass

[bwoodsend]

Sounds sensible.

[rizsyed1]

@heuripedes could you please submit a PR implementing the spec-file you posted above? That would be really helpful and appreciated.

[heuripedes]

It is not only the xref file that gets overwritten, it's pretty much every file generated and affects non-debug builds too.

The solution I found was to set workpath for every program i.e.:

from PyInstaller.config import CONF
BASE_WORKPATH = os.path.dirname(CONF["workpath"])
CONF["workpath"] = os.path.join(BASE_WORKPATH, "prog1")
os.makedirs(CONF["workpath"], exist_ok=True)
prog1_a = Analyse(...)
CONF["workpath"] = os.path.join(BASE_WORKPATH, "prog2")
os.makedirs(CONF["workpath"], exist_ok=True)
prog2_a = Analyse(...)

MERGE([
    (prog1_a ...),
    (prog2_a ...)
])

CONF["workpath"] = os.path.join(BASE_WORKPATH, "prog1")
prog1_pyz = PYZ(...)
prog1_exe = EXE(...)
prog1_coll = COLLLECT(...)
CONF["workpath"] = os.path.join(BASE_WORKPATH, "prog2")
prog2_pyz = PYZ(...)
prog2_exe = EXE(...)
prog2_coll = COLLLECT(...)

This has the additional benefit of greatly increasing incremental (i.e. non-clean) build times.