rar2john should reject non-RAR files

[Siltal]

Checklist

• [x] :1st_place_medal: I've read and understood these instructions;
  • This is not a support forum, it's a bug tracker. For questions and support, review postings on the john-users mailing list.
• [ ] :thumbsup: I've tested using latest bleeding version from this repository.
  • Be clear about your environment and what you are doing. Share a sample hash or file that can be used to reproduce.
• [x] :confused: I'm confused and I need guidance.
  • Please, read the instructions at https://www.openwall.com/john/#lists, then join the list before posting.

IMPORTANT

We expect only reports of issues with the latest revision found in this GitHub repository. We do not expect in here, and have little use for, reports of issues only seen in a release or in a distro package.

Attach details about your OS and about john, including:

• OS Windows 10 Pro WorkStation 19044.1826
• The output of ./john --list=build-info.

Version: 1.9.0-jumbo-1
Build: cygwin 64-bit x86_64 AVX2 AC OMP
SIMD: AVX2, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
CPU tests: AVX2
CPU fallback binary: john-xop
OMP fallback binary: john-avx2-non-omp
$JOHN is /run/
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
SINGLE_IDX_MAX: 2147483648
SINGLE_BUF_MAX: 4294967295
Effective limit: Number of salts vs. SingleMaxBufferSize
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 7.4.0
OpenCL headers version: 2.2
Crypto library: OpenSSL
OpenSSL library version: 01010102f
OpenSSL 1.1.1b  26 Feb 2019
GMP library version: 6.1.2
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's

Issue

• The command line I am using.

rar2john file.rar

• its output

bad rar crypt version byte

when I using bandizip to open rar file, it can list inner file in rar but encryption. file size 616MB, too large to uploading.

[solardiz]

• I've tested using latest bleeding version from this repository.

You actually did not - your report is against the 1.9.0-jumbo-1 release. However, this should not matter at this time for rar2john in particular, because the only changes made to it since the release look irrelevant to your problem:

commit 7d963e367313de4ec2953045a61a446a61b660eb
Author: magnum <[email protected]>
Date:   Wed Feb 3 12:32:03 2021 +0100

    rar2john: Mention the -v option in usage
    
    Closes #4066

commit e26e88c8e119e0b4723827970b963db138d19cd1
Author: magnum <[email protected]>
Date:   Wed Sep 30 13:54:31 2020 +0200

    rar2john: Warn when no encrypted data found in RAR5 archive
    
    Closes #4361

commit 4ef1ef28f91034cebce396f81456c540f00355ab
Author: magnum <[email protected]>
Date:   Mon Feb 3 11:56:37 2020 +0100

    Strip ':' from login fields produced by *2john tools.  Closes #4200

Given where the bad rar crypt version byte message is in code, it looks like the archive was in fact detected as RAR, but we don't support its particular variation.

Can you generate a sample archive just like this (producing the same message from rar2john), but much smaller and with a known password, and attach that? If you can't, then if you have Cygwin installed, can you run these commands and include their output:

file file.rar
head -c 1024 file.rar > file-firstk.bin

The first command (yes, there's a command literally called file on Cygwin) should tell us about the file type, and the second command should save the first kilobyte of the file to file-firstk.bin, which you can then attach to here. Thanks.

[Siltal]

I crete a sample rar with password can generate hash, It should be a problem with the rar file

[Siltal]

I check file and found its 7-zip archive data, version 0.4 I am apologize for my ignorance

[solardiz]

@Siltal Does 7z2john.pl work on this file?

@magnumripper Can we enhance rar2john to produce a better error message on files that are not RAR archives, or does the file format not allow for that? I just ran our current rar2john on all files in our run directory just to see how it'd fail on them, and it produced a variety of error messages for some and output $rar5$ "hashes" for many others (none of which are actually RAR).

[Siltal]

@solardiz Yes, 7z2john.pl works fine, btw the 616MB 7z generate 1.2GB Hash, that's going too far.

[solardiz]

the 616MB 7z generate 1.2GB Hash, that's going too far.

Yet that's how it currently is. Those are not really hashes, we just reuse core JtR functionality that had been intended only for hashes until we introduced support for non-hashes such as archives.

[solardiz]

@Siltal While this may be completed for your needs, it is not yet completed for the needs of our project - please see my question to @magnumripper above. So I'll reopen and refocus this issue.

[solardiz]

I've just changed the issue title to "rar2john should reject non-RAR files", although I think there might be reasons for the current implementation (e.g., maybe we're unable to reliably detect self-extracting archives, or something of that sort). So this is up for discussion for now. Maybe we'll end up just closing the issue after discussion, or maybe we'll plan on making code changes.

[magnumripper]

@magnumripper Can we enhance rar2john to produce a better error message on files that are not RAR archives, or does the file format not allow for that? I just ran our current rar2john on all files in our run directory just to see how it'd fail on them, and it produced a variety of error messages for some and output $rar5$ "hashes" for many others (none of which are actually RAR).

Oh, that's bad. We should look into it. I believe there are file magics we should look for.

[magnumripper]

I believe there are file magics we should look for.

There are and we did look for them already, but there was a logic flaw for RAR5 making some files pass without the magic.

After #5192:

magnum@mbpr:src [rar2john-logic-fix]$ make -sj8 && ../run/rar2john ../run/*
! ../run/1password2john.py: Not a RAR file
! ../run/7z2john.pl: Not a RAR file
! ../run/DPAPImk2john.py: Not a RAR file
  (... many similar lines snipped ...)
! ../run/zed2john.py: Not a RAR file
! ../run/zip2john: Not a RAR file
! ../run/ztex: Not a RAR file