hachoir
hachoir copied to clipboard
[bug] MemoryErrors when using subfile
Error output:
$ python3 -m hachoir.subfile DSN-CTL-V23R01.exe
[+] Start search on 6475848 bytes (6.2 MB)
[+] File at 0 size=57344 (56.0 KB): Microsoft Windows Portable Executable: Intel 80386, Windows GUI
[!] Memory error!
[+] End of search -- offset=524288 (512.0 KB)
Total time: 676 ms -- global rate: 756.7 KB/sec
Can be reproduced with https://dsn-ctl.fr/DSN-CTL-V23R01.exe
$ wget https://dsn-ctl.fr/DSN-CTL-V23R01.exe
--2023-07-04 13:19:30-- https://dsn-ctl.fr/DSN-CTL-V23R01.exe
Resolving dsn-ctl.fr (dsn-ctl.fr)... 85.236.158.186
Connecting to dsn-ctl.fr (dsn-ctl.fr)|85.236.158.186|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6475848 (6.2M) [application/x-msdownload]
Saving to: ‘DSN-CTL-V23R01.exe’
DSN-CTL-V23R01.exe 100%[==========================================>] 6.18M 3.71MB/s in 1.7s
2023-07-04 13:19:32 (3.71 MB/s) - ‘DSN-CTL-V23R01.exe’ saved [6475848/6475848]
$ shasum -a256 DSN-CTL-V23R01.exe
b4e855f92c4ae8cec77b9ccaf8b6e0cf53134eb47f5e668980e20afdc149d99f DSN-CTL-V23R01.exe
Host info:
$ uname -rvm
6.2.6-76060206-generic #202303130630~1685473338~22.04~995127e SMP PREEMPT_DYNAMIC Tue M x86_64
Python version:
$ python3 -V
Python 3.10.6
Hachoir version:
$ pip show hachoir
Name: hachoir
Version: 3.2.0
Summary: Package of Hachoir parsers used to open binary files
Home-page: http://hachoir.readthedocs.io/
Author: Hachoir team (see AUTHORS file)
Author-email:
License: GNU GPL v2
Location: /home/agrajag9/.local/lib/python3.10/site-packages
Requires:
Required-by:
And on RHEL 7.9:
$ python3 -m hachoir.subfile ba6981176279a3547cb62566559361d628d3f7e665f31065b5a5e420a307e5b7
[+] Start search on 6471808 bytes (6.2 MB)
[+] File at 0 size=57344 (56.0 KB): Microsoft Windows Portable Executable: Intel 80386, Windows GUI
Fatal Python error: Cannot recover from MemoryErrors while normalizing exceptions.
Python runtime state: initialized
Current thread 0x00007fdd8327f740 (most recent call first):
File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/search.py", line 224 in guess
File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/search.py", line 192 in findMagic
File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/search.py", line 136 in searchSubfiles
File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/core/memory.py", line 97 in limitedMemory
File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/search.py", line 89 in main
File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/main.py", line 73 in runSearch
File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/main.py", line 96 in main
File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/__main__.py", line 3 in <module>
File "/opt/rh/rh-python38/root/usr/lib64/python3.8/runpy.py", line 87 in _run_code
File "/opt/rh/rh-python38/root/usr/lib64/python3.8/runpy.py", line 194 in _run_module_as_main
Aborted
$ uname -rvm
3.10.0-1160.92.1.el7.x86_64 #1 SMP Thu May 18 11:23:40 UTC 2023 x86_64
$ python3 -V
Python 3.8.14
Verbose log with python3 -v -m hachoir.subfile DSN-CTL-V23R01.exe 2>&1 | tee hachoir.log
https://gist.github.com/agrajag9/a60a8580fbcae56999635331d1b38029