[BUG] Windows fatal exception: access violation

[Thoufak]

First of all, I'm not sure if this a bug in Pyarmor, but I hope you can help me find out. I'm also using PyInstaller and have opened an issue there too. However, it doesn't look that the problem lies inside PyInstaller. Please also note that it would be very hard for me to test if the problem is present when I use just Pyarmor (without PyInstaller), because it can't be easily reproduced (see details below).

Sometimes a compiled version of my program will crash with this exception. It would look something like this:

I’ve observed some notable features about when and how this happens, and I will list them below. Note that I’ve been using pyarmor + PyInstaller for over a year for this project, but I started seeing this problem only recently.

Rebuilding helps

If I see that the resulting exe has this issue, I can build it again using exactly the same source code and command. Nothing seems to be changing from my side. And this will produce a build which doesn’t have the crash, or it will now appear in a different place (see the next point).

It crashes on different lines.

If a specific build has this crash, and I don’t change any input parameters to my program, it will crash on the same line every time I start it (I mainly looked at the main thread stacktrace). In other words, this crash is very well reproducible within the same build. However, as I said, if I make a new build, this error will either be gone (most likely) or appear in a completely different place. I couldn’t find any meaningful similarities between the different lines of code I’ve seen my program crashing at.

Notice how I said "and I don’t change any input parameters to my program". I’ve once observed a crash inside a ‘for’ loop which is used to load lines from a text file. I managed to find the exact number of lines needed to crash my program. If the number of lines was below, it wouldn’t crash on that loop (but it probably will crash a bit later. I don’t remember this detail but I can check). It looks as though there's a defined number of "operations" that it does before crashing.

Also, it seems that this crash can happen in any place in the code. Sometimes it’s right at the start of the program, but sometimes you have to use it for a while to see the crash.

Visual Studio debugger

I’ve tried attaching a debugger from Visual Studio. I don’t think it produced any useful information, but I will attach the log anyway. VSCode debugger.txt

More tracebacks examples

These are from different builds.

Versions

PyInstaller: 4.9 Pyarmor: 7.4.3 Python: 3.9.5 EXE built on: Windows 11 Home ARM64 (this is a machine from Parallels Desktop running on Mac M1 Pro) Command for pyarmor: pyarmor pack src\safe_main.py -x="--advanced 2 --restrict=2" -e="-F -p="." --key "some_key" --icon Icon.ico --version-file VersionInfo.txt --hidden-import requests (about 20 hidden imports omitted).

[jondy]

First check if there is any system packages in the path src/, move them to other path.

Secondly copy all the contents in the path src/ to dist/, then obfuscate them

pyarmor obfuscate -r --advanced 2 --restrict=2 src/safe_main.py

And test the obfuscated scripts

python dist/safe_main.py

[Thoufak]

Thank you for a quick response.

First check if there is any system packages

Please clarify what a "system package" is.

[Thoufak]

A assumed there are no "system packages" in my folder and followed your instructions. When I execute the command python dist/safe_main.py my app starts, and it works as normal. What's next?

[jondy]

If dist/safe_main.py works fine, it's most possible that PyInstaller results in this issue.

[Thoufak]

If dist/safe_main.py works fine, it's most possible that PyInstaller results in this issue.

Why do you think so? As I said, the crash is not reproduced on every build even if I enable PyInstaller.

[jondy]

Try to use repack, it may be helpful to find the problem https://pyarmor.readthedocs.io/en/latest/advanced.html#repack-pyinstaller-bundle-with-obfuscated-scripts

[Thoufak]

Ok, I will try and let you know how it went

[Thoufak]

I took one of my app builds that has this crash. Then, I extracted all the files from the built exe using pyinstxtractor. After that I launched safe_main.pyc, and the crash is still reproduced.

So, the problem lies neither in PyInstaller, nor in my own code (the crash isn't reproduced in a plain unpacked script).

[jondy]

How about using pyinstaller to pack the plain script with same option (include --key also), then extract all the files using pyinstaxtractor, and run safe_main.pyc?

If you have tested, try to remove option --key, and try pyarmor pack

Only by these information, I have no more idea to find what's the problem.

[Thoufak]

While this would take a very big amount of time, I could try to reproduce this crash the way you suggest. However, I don't see how this will help us troubleshoot the problem. There are python files now that are not packed by PyInstaller and the crash is still reproduced.

[Thoufak]

What do I need to do in order to confirm that this is a Pyarmor bug? And if I confirm it, will you be willing to look into it and try to fix it?

[jondy]

If I could not reproduce this bug, it's not easy for me to fix it.

[jondy]

And try to upgrade Pyarmor to latest version.

[Thoufak]

If I could not reproduce this bug, it's not easy for me to fix it.

Will it help if I send you the safe_main.pyc (protected by pyarmor) which crashes?

[jondy]

No, it's not helpful.

[jondy]

By the way, I have fixed a crash bug for super mode in aarch64 platform in Pyarmor v7.7.0. If you're run obfuscated scripts in aarch64 windows, try to upgrade Pyarmor to latest version (now it's v7.7.3). In your report, Pyarmor version is v7.4.3

[Thoufak]

Yes, my machine is exactly aarch64. I will update and let you know.

[Thoufak]

I've updated PyArmor and tested several builds, and it seems the crash is now gone. Thank you very much for fixing it!

[Thoufak]

Actually no, I've just reproduced the same issue on PyArmor 7.7.3.

[jondy]

Suddenly I have a question. There is no super mode library for windows.aarch64.11.py39, only windows.x86_64.11.py39 exists, so which binary file is used in this platform?

To check this, obfuscate a simple script

pyarmor obfuscate --advanced 2 --exact foo.py

Check the log, find one line to copy extension pytransform.pyd. For example, in my Darwin and python 3,7, it looks like this

INFO     Copying /Users/jondy/.pyarmor/platforms/darwin/x86_64/11/py37/pytransform.cpython-37m-darwin.so

[Thoufak]

INFO Copying C:\Users\ss2\.pyarmor\platforms\windows\x86_64\11\py39\pytransform.pyd

[Thoufak]

Any news on this? Should downgrading to python 3.8 help?

[Thoufak]

Sorry for the ping, but did the provided information give you any hints on how to solve it?

[jondy]

No idea. And now I'm busy on upgrading Pyarmor to support python 3.11, no more time to research this.

[Thoufak]

Hi. I saw you released pyarmor 8. Congratulations on that. I was wondering if you could look into this issue now? This problem indeed seems to have appeared since I moved from an x86 to arm machine

[jondy]

I'll appreciated if you could provide a simple script to reproduce this issue, it will be handled after pyarmor 8.0 is stable.

[grinchify]

When would pyarmor 8.0 be stable?

[jondy]

@Thoufak about this issue, my suggestion is

1. do not use pack, only obfuscate the script.
2. first obfuscate the scripts without --restrict 2
3. if it still doesn't works, please replace the obfuscated script in the traceback with plain script, and check it work or not For example, there is <frozen a.b> in the traceback, then copy src/a/b.py to dist/a/b.py One by one to replace those obfuscated modules to locate the problem script and function.