mvt
mvt copied to clipboard
mvt-project
Error in running extraction from module Files: not enough values to unpack (expected 6, got 1)
Android, POCO MIUI 13.03 python 3.9 (anaconda) mvt installed from pip (1.5.4)
Could you share privately (at tek [AT] randhome.io) what the following commands give in adb ?
find '/data/local/tmp/' -printf '%T@ %m %s %u %g %p\n' 2> /dev/null
find '/sdcard/' -printf '%T@ %m %s %u %g %p\n' 2> /dev/null
find '/tmp/' -printf '%T@ %m %s %u %g %p\n' 2> /dev/null
Can confirm as well:
Looking up 458 files... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% -:--:--
INFO [mvt.android.modules.adb.packages] Extracted at total of 486
installed package names
12:08:41 INFO [mvt.android.modules.adb.packages] The Packages module
produced no detections!
12:08:42 INFO [mvt.android.modules.adb.logcat] Running module Logcat...
12:08:47 INFO [mvt.android.modules.adb.logcat] Current logcat logs stored at
/home/username/Desktop/res3/logcat.txt
INFO [mvt.android.modules.adb.logcat] Logcat logs prior to last
reboot stored at /home/username/Desktop/res3/logcat_last.txt
INFO [mvt.android.modules.adb.logcat] The Logcat module does not
support checking for indicators
INFO [mvt.android.modules.adb.root_binaries] Running module
RootBinaries...
12:08:48 INFO [mvt.android.modules.adb.root_binaries] The RootBinaries
module does not support checking for indicators
INFO [mvt.android.modules.adb.files] Running module Files...
12:08:57 ERROR [mvt.android.modules.adb.files] Error in running extraction
from module Files: not enough values to unpack (expected 6,
got 1)
Traceback (most recent call last):
File
"/home/username/.local/lib/python3.9/site-packages/mvt/common/mod
ule.py", line 166, in run_module
module.run()
File
"/home/username/.local/lib/python3.9/site-packages/mvt/android/mo
dules/adb/files.py", line 124, in run
self.find_files(media_folder)
File
"/home/username/.local/lib/python3.9/site-packages/mvt/android/mo
dules/adb/files.py", line 87, in find_files
[unix_timestamp, mode, size,
ValueError: not enough values to unpack (expected 6, got 1)
Furthermore, only /tmp/ returns any output - the commands with /data/local/tmp and /sdcard/ do not return any output.
LMDE 5 Latest ADB downloaded from google's website Android 12 SKQ1.210908.001, MIUI 13.0.10 Global Xiaomi Redmi Note 10 Pro Python 3.9.2 MVT 2.1.5, via pip
P.S.: I do not use English as the phone's language
Quick comments:
-
Yours may be a non-standard Android edition. Try running
adb shell cat /proc/versionand post the results here, after some basic anonymizing of the output. -
FYI, out of these three commands only this one:
find '/tmp/' -printf '%T@ %m %s %u %g %p\n' 2> /dev/nulldoes not show any files on my Droid, which is no wonder as:
ls /tmp/
ls: /tmp/: No such file or directory
- FYI, yours is:
MVT 2.1.5, via pip
while the current one is:
pip3 install --upgrade mvt ...
mvt-android version
-> ... Version: 2.2
Try upgrading - it may help.
Result with MVT 2.2:
14:12:34 ERROR [mvt.android.modules.adb.files] Error in running extraction
from module Files: not enough values to unpack (expected 6,
got 1)
Traceback (most recent call last):
File
"/home/user/.local/lib/python3.9/site-packages/mvt/common/mod
ule.py", line 166, in run_module
module.run()
File
"/home/user/.local/lib/python3.9/site-packages/mvt/android/mo
dules/adb/files.py", line 124, in run
self.find_files(media_folder)
File
"/home/user/.local/lib/python3.9/site-packages/mvt/android/mo
dules/adb/files.py", line 87, in find_files
[unix_timestamp, mode, size,
ValueError: not enough values to unpack (expected 6, got 1)
Result of adb shell cat /proc/version:
Linux version 4.14.190-perf-gd1f5180ae73c (clang version 10.0.7 for Android NDK, GNU ld (binutils-2.27-bd24d23f) 2.27.0.20170315) #1 SMP PREEMPT Wed Aug 10 19:37:40 CST 2022
Small correction on my part when it comes to the previous post, I probably accidentally ran the commands without adb. On the adb shell (today), the directory /data/local/tmp/ is empty; the directory /tmp/ does not exist, And of course the /sdcard/ directory shows all my files. Apologies for the confusion above.
Android Debug Bridge version 1.0.41 Version 28.0.2-debian Python 3.9.2
Thank you. I am not a Python developer or maintainer of this tool, so quick informal advice:
- Your Android (phone) is newer than mine (flashed Lineage):
Linux version 3.18.124-perf-g9100df495471-dirtybut then your clang is older, cf mine:clang version 12.0.6I doubt that it affects anything because: - This sample page explains this error https://www.delftstack.com/howto/python/not-enough-values-to-unpack/ in more detail.
- My tip would be to run it in any debugger (Pycharm etc.) and check what exactly is this discrepancy between the expected number of arguments (values) and the ones fed in, as per the URL above:
A ValueError is a common exception in Python that occurs when the number of values doesn’t match the number of variables either taking input, direct assignment or through an array or accessing restricted values. To understand the ValueError, let’s take an example:
# this input statement expects three values as input
x,y,z = input("Enter values for x, y and z: ").split(",")
Output:
Enter values for x, y and z: 1,2
ValueError: not enough values to unpack (expected 3, got 2)
Alas, I cannot give you more help - the maintainers should know the root causes much better. Good luck!
Hi,
Could you share the first 10 lines of find '/data/media/0' -type f -printf '%T@ %m %s %u %g %p\n' 2> /dev/null and find '/sdcard/' -type f -printf '%T@ %m %s %u %g %p\n' 2> /dev/null? (Either here or by email to tek AT randhome.io)
Thanks
Hello there!
The first command doesn't produce any output. I just sent the first 10 lines of the output of the second command to your email.
I have the same problem. The output is as follows:
INFO [mvt.android.modules.adb.files] Running module Files...
23:53:16 ERROR [mvt.android.modules.adb.files] Error in running extraction
from module Files: not enough values to unpack (expected 6,
got 1)
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.11/lib/python
3.11/site-packages/mvt/common/module.py", line 166, in
run_module
module.run()
File
"/Library/Frameworks/Python.framework/Versions/3.11/lib/python
3.11/site-packages/mvt/android/modules/adb/files.py", line
124, in run
self.find_files(media_folder)
File
"/Library/Frameworks/Python.framework/Versions/3.11/lib/python
3.11/site-packages/mvt/android/modules/adb/files.py", line 87,
in find_files
[unix_timestamp, mode, size,
ValueError: not enough values to unpack (expected 6, got 1)
Context: This is the result when an OnePlus 9Pro is inspected. A OnePlus 7t Pro does not cause any errors. The same result occurs when tested on two PCs and a Macbook. Python is 3.11 MVT version is 2.2.1 Linux version 5.4.147-qgki-g5d92ddfddb85 Android 6877366 LLD 11.0.2 clang version 11.0.2
None of the above three commands mentioned output anything.
Hi everyone, I am not fully sure where the problem is as it is quite complex to reproduce a bug with adb. My guess is that there is a parsing issue when the folder doesn't exist. I have made a check for that with d05e6fa, could you check if it solves the problem for you all? (you need to get the last mvt version from github, not from pip)
Edit:
Sadly, this did not fix the issue:
Looking up 456 files... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% -:--:--
INFO [mvt.android.modules.adb.packages] Extracted at total of 491
installed package names
INFO [mvt.android.modules.adb.logcat] Running module Logcat...
18:52:52 INFO [mvt.android.modules.adb.logcat] The Logcat module does not
support checking for indicators
INFO [mvt.android.modules.adb.root_binaries] Running module
RootBinaries...
INFO [mvt.android.modules.adb.root_binaries] The RootBinaries
module does not support checking for indicators
INFO [mvt.android.modules.adb.files] Running module Files...
18:53:15 ERROR [mvt.android.modules.adb.files] Error in running extraction
from module Files: not enough values to unpack (expected 6,
got 1)
Traceback (most recent call last):
File
"/home/myuser/.local/lib/python3.10/site-packages/mvt/common/mo
dule.py", line 166, in run_module
module.run()
File
"/home/myuser/.local/lib/python3.10/site-packages/mvt/android/m
odules/adb/files.py", line 127, in run
self.find_files(media_folder)
File
"/home/myuser/.local/lib/python3.10/site-packages/mvt/android/m
odules/adb/files.py", line 90, in find_files
[unix_timestamp, mode, size,
ValueError: not enough values to unpack (expected 6, got 1)
To make sure the correct version was installed, I manually edited the version before installing with pip3:
mvt-android version
MVT - Mobile Verification Toolkit
https://mvt.re
Version: 2.2.2
So @itsintavis might be correct - it very well could be a character issue, considering my files list (find '/sdcard/' -printf '%T@ %m %s %u %g %p\n' 2> /dev/null) contains a few files with names that include non-standard Unicode characters. In my case, this includes Greek characters, Bulgarian Cyrillic characters, and non-ASCII Polish letters, such as ą/ę/ż/ł. Notably, my phone's system language is Polish.
Will test also and revert. I can somewhat foresee that this is a Greek language / characters issue. At least two persons reporting it have phones that are likely to have Greek characters on them.
Instalamos : pip3 install –user mvt
mvt-android check-adb
Crea una carpeta en tu sistema
I believe I have found the/a culprit in my case. @Te-k
So, I added a print() statement in mvt/android/modules/adb/files.py such that:
def find_files(self, folder: str) -> None:
if self.full_find:
cmd = f"find '{folder}' -type f -printf '%T@ %m %s %u %g %p\n' 2> /dev/null"
output = self._adb_command(cmd)
if output.strip() == "":
return
for file_line in output.splitlines():
print(file_line) #<-- here
[unix_timestamp, mode, size,
owner, group, full_path] = file_line.rstrip().split(" ", 5)
mod_time = convert_unix_to_iso(unix_timestamp)
Which would print every file up to the point of the crash. I have found that the culprit in this case was at:
1647250715.651322768 660 184632 u0_a294 sdcard_rw /sdcard/Android/data/gr.winbank.mobilenext/files/Documents/winbank mobile - TRANSACTION DETAILS
.pdf
which is a transaction receipt from my bank.
Then, I ran the find command manually via ADB and:
1647250715.651322768 660 184632 u0_a294 sdcard_rw /sdcard/Android/data/gr.winbank.mobilenext/files/Documents/winbank mobile - TRANSACTION DETAILS
.pdf
It seems that the bank was inserting a special character (\n ?) after the filename and before the file suffix (.pdf). So, MVT was parsing .pdf as a separate file. Obviously, there are no arguments on date, UNIX permissions, owners, etc about that file on that line. As such, the error was ValueError: not enough values to unpack (expected 6, got 1)
I then browsed to that path using ADB and ran ls:
-rw-rw---- 1 u0_a294 sdcard_rw 184632 2022-03-14 11:38 winbank\ mobile\ -\ TRANSACTION\ DETAILS\n.pdf
Then I deleted both the file and the folder using ADB (as the folder's contents weren't accessible via my file browser) and ran mvt-android check-adb again.
1670679156.660784672 660 0 root everybody /sdcard/.fe_tmp
11:31:30 INFO [mvt.android.modules.adb.files] Found 56312 files in primary
Android tmp and media folders
INFO [mvt.android.modules.adb.files] Processing full file listing.
This may take a while...
And it worked!
So, it seems that the culprit is stray \n characters in filenames.
I can think of several ways to counteract this issue, but I'll leave it up to you to decide which is the optimal solution.
This is great, thank you so much! I don't see any very clean way to manage that, maybe the easiest would be to skip lines that aren't in the right format. I don't see any easy way to identify is a misformated line is the end of a file name or something else, so I'm not sure I want to try reconstructing file names.
To avoid abuse (e.g. hiding malware in files with \n and other special characters), whenever a line is skipped, I'd recommend printing the content of the line being skipped + the content of the previous line. E.g.:
[WARNING]: Skipping line ".pdf", previous line:
"1647250715.651322768 660 184632 u0_a294 sdcard_rw /sdcard/Android/data/gr.winbank.mobilenext/files/Documents/winbank mobile - TRANSACTION DETAILS"
due to incorrect formatting. Renaming the file might fix this issue.
It might not be a perfect solution, but it's probably a good-enough solution
Hi, I just fixed that in the repository. The code will be part of MVT 2.2.2 that should be on pypi soon. Let me know if you still encounter that issue.