Archive.extractall hangs on password protected .rar file without throwing exception

[lennier1]

I'm writing a python script to process some old archives that include many .rar files. I can successfully extract data from them using Archive.extractall from pyunpack. However, occasionally I will hit a password protected file, and the program just hangs. I have the code in a try block, but it's not throwing any exception. Would it be possible to throw an exception in this circumstance so my script can skip the file and continue?

Here's the relevant portion of my code:

import os
import rarfile
from pyunpack import Archive

for subdir,dirs,files in os.walk(directoryToExtract):
        for file in files:
            if rarfile.is_rarfile(os.path.join(subdir, file)):
                print("Found rarfile " + os.path.join(subdir, file))
                try:                    
                    Archive(os.path.join(subdir, file)).extractall(os.path.join(subdir, file[0:len(file)-4]),auto_create_dir=True)
                except Exception as e:
                    print("ERROR: Couldn't unrar " + os.path.join(subdir, file))
                    print(str(e))

[ponty]

I created a test for a rar file with password. The test passes on all platforms. https://github.com/ponty/pyunpack/actions/runs/1734014609

The test: https://github.com/ponty/pyunpack/blob/master/tests/test_rarpw.py The rar file: https://github.com/ponty/pyunpack/blob/master/tests/testpw.rar

Check this file if it hangs on your system.

[lennier1]

I downloaded the script and .rar file, added a call to test_rarpw() to main, and ran it. It hung for me as I saw in my own program. I replaced the password protected .rar file with a regular .rar file and got "Failed: DID NOT RAISE <class 'pyunpack.PatoolError'>", which I assume is expected.

Windows 10 Pro Python 3.7.9 (tags/v3.7.9:13c94747c7, Aug 17 2020, 16:30:00) [MSC v.1900 64 bit (AMD64)] on win32 pyunpack 0.2.2 patool 1.12 pytest 6.2.5

As I hadn't updated python recently, I updated to Python 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)] on win32

This actually broke things further, as I can't get it to find patool even though pip says it's installed. Perhaps there is something wrong with the way I have everything set up, but I don't see any issues.

pip install patool Requirement already satisfied: patool in c:\users\paul\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (1.12)

pip install pyunpack Requirement already satisfied: pyunpack in c:\users\paul\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (0.2.2) Requirement already satisfied: entrypoint2 in c:\users\paul\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (from pyunpack) (1.0) Requirement already satisfied: easyprocess in c:\users\paul\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (from pyunpack) (1.1)

python test_rarpw.py Traceback (most recent call last): File "E:\recursive extractor\test_rarpw.py", line 23, in test_rarpw() File "E:\recursive extractor\test_rarpw.py", line 21, in test_rarpw cli.extractall(f, d) File "C:\Users\Paul\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pyunpack\cli.py", line 19, in extractall Archive(filename, backend).extractall(directory, auto_create_dir=auto_create_dir) File "C:\Users\Paul\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pyunpack_init_.py", line 94, in extractall self.extractall_patool(directory, patool_path) File "C:\Users\Paul\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pyunpack_init_.py", line 50, in extractall_patool raise ValueError("patool not found! Please install patool!") ValueError: patool not found! Please install patool!

[ponty]

Check if patool is in your PATH. (Run patool in command line) Reinstall patool, check if patool works with rar file:

$ patool --non-interactive extract testpw.rar 
patool: Extracting testpw.rar ...
patool: running /usr/bin/rar x -p- -y -- /tmp/testpw.rar
patool:     with cwd='./Unpack_qxvp0mda'

The specified password is incorrect.patool error: error extracting testpw.rar: Command `['/usr/bin/rar', 'x', '-p-', '-y', '--', '/tmp/testpw.rar']' returned non-zero exit status 11

Pyunpack is just a wrapper for patool.

Lates test results: https://github.com/ponty/pyunpack/actions/runs/1735545359 Windows 2019 and 2022 are passing tests.

[lennier1]

Thanks. I found my patool script:

python c:\users\paul\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\Scripts\patool

It's a little confusing that it has no extension, but turns out it's just python code, so I can run it through python. I tried extracting the .rar file with the password and it does still hang for me. Mine tried to use 7z.EXE to do it. I'd have expected that to work, but it is different than your setup.

python c:\users\paul\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\Scripts\patool --non-interactive extract testpw.rar patool: Extracting testpw.rar ... patool: running "C:\Program Files\7-Zip\7z.EXE" x -y -o.\Unpack_me_2wr88 -- testpw.rar

I tried the same thing on a .rar file without a password and it worked fine.

python c:\users\paul\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\Scripts\patool --non-interactive extract Landofheroes.rar patool: Extracting Landofheroes.rar ... patool: running "C:\Program Files\7-Zip\7z.EXE" x -y -o.\Unpack__u87784e -- Landofheroes.rar patool: ... Landofheroes.rar extracted to `tibia_land_of_heroes_uk.sis'.

[ponty]

This is a patool feature. If you run directly C:\Program Files\7-Zip\7z.EXE x -y -- testpw.rar then 7z will stop for password input. This is my run on Linux:

$ 7z x -y   -- testpw.rar

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,6 CPUs Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz (906EA),ASM,AES-NI)

Scanning the drive for archives:
1 file, 154 bytes (1 KiB)

Extracting archive: testpw.rar
--
Path = testpw.rar
Type = Rar5
Physical Size = 154
Solid = -
Blocks = 1
Encrypted = -
Multivolume = -
Volumes = 1

   
Enter password (will not be echoed):

The solution has been found, but it was not merged into patool: https://github.com/wummel/patool/pull/83 "Adds '-p-' to 7z cmdlist to skip password input prompt."

Related issue: https://github.com/wummel/patool/issues/55

Rar is working with patool you can try to install it.

[eisene]

I made a pull request #14 to fix this issue

[ponty]

#14 is a nice addition but it doesn't solve this issue: starting extractall without password hangs sometimes. The problem is in the patool implementation. However your use case may be fixed if you always set the password when needed.

[eisene]

True what you say. In my current use case I just provide a dummy password. I looked through patool actually and the root problem is 7z, it doesn't seem to have a true non-interactive mode. I may have missed something.