"lzvn error" has been occurred in AUTOSTART plugin when analyzing a M1 Mac disk image

[mnrkbys]

Hi, @ydkhatri, I am trying to analyze an M1 Mac (macOS 12.5) disk image created by Digital Collector (former MacQuisition).

While AUTOSTART plugin is running, I get the following error "lzvn error - could not decompress stream". As a result, the plugin can not analyze plist files. However, not all plist files will have errors.

2022-07-28 13:44:28|MAIN.HELPERS.MACINFO|DEBUG|Trying to open plist file : /Library/Extensions/HighPointIOP.kext/Contents/Info.plist
2022-07-28 13:44:28|MAIN.HELPERS.APFS_READER|DEBUG|Trying to open file : /Library/Extensions/HighPointIOP.kext/Contents/Info.plist
2022-07-28 13:44:28|MAIN.HELPERS.MACINFO|DEBUG|Trying to read plist file : /Library/Extensions/HighPointIOP.kext/Contents/Info.plist
2022-07-28 13:44:28|MAIN.HELPERS.APFS_READER|DEBUG|lzvn compressed size seems incorrect, trying to correct..
2022-07-28 13:44:28|MAIN.HELPERS.APFS_READER|DEBUG|found end of stream, correcting now.. old size=1193 new size=1172, diff=21
2022-07-28 13:44:28|MAIN.HELPERS.APFS_READER|ERROR|lzvn error - could not decompress stream, returning nulls
2022-07-28 13:44:28|MAIN.AUTOSTART|ERROR|Problem reading plist for /Library/Extensions/HighPointIOP.kext/Contents/Info.plist - Not a plist!  Error was : Invalid file
2022-07-28 13:44:28|MAIN.HELPERS.APFS_READER|DEBUG|Trying to copy out /Library/Extensions/HighPointIOP.kext/Contents/Info.plist
2022-07-28 13:44:28|MAIN.HELPERS.APFS_READER|DEBUG|lzvn compressed size seems incorrect, trying to correct..
2022-07-28 13:44:28|MAIN.HELPERS.APFS_READER|DEBUG|found end of stream, correcting now.. old size=1193 new size=1172, diff=21
2022-07-28 13:44:28|MAIN.HELPERS.APFS_READER|ERROR|lzvn error - could not decompress stream, returning nulls

Curiously, this error does not occur with an Intel Mac (macOS 12.5) disk image. Furthermore, this error does not occur with any other plugins.

Do you know any similar problems like this?

[ydkhatri]

Can't say without looking at the file. Can you share a couple of files which exhibit this problem?

[mnrkbys]

I have uploaded here. https://1drv.ms/u/s!Aj92UJs3UQTPg-B_9W_Jl8waI_hA2w?e=2E8D9D

It can be downloaded until Aug 6, 2022.

[ydkhatri]

Ok, well this is not possible to debug as the files you sent are already uncompressed. Would it be possible to send the compressed raw version of one of the files from disk image? You'd have to modify the mac_apt code to do so, or put a breakpoint on the lzvn error when running it in a debug mode and dump out the raw buffer to a file.

[mnrkbys]

Thanks for your reply. I have modified the code like below. https://github.com/mnrkbys/mac_apt/commit/15f8b363522144136604642c3888063efa8f6f3a

Then I have attached a zipped output file here. lzvn_error_raw_data.zip

[mnrkbys]

I found the difference between the dumped data and the data compressed with liblzfse (see attached image below). 0000000000000000_1172_3442.bin is the dumped data from the disk image. It corresponds to "/Library/Extensions/HighPointIOP.kext/Contents/Info.plist". And, compressed_HighPointIOP_Info.bin is the data compressed with liblzfse.compress() method.

0000000000000000_1172_3442.bin is missing part of the header indicated by the red box. It also contains unnecessary data at the end, as indicated by the blue box. Correcting these information will ensure that the data is decompressed correctly.

>>> liblzfse.decompress(b'bvxn' + b'\x72\x0D\x00\x00' + b'\x78\x04\x00\x00' + file_1172_data[:1144] + b'bvx$')
b'<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">\n<plist version="1.0">\n<dict>\n\t<key>BuildMachineOSBuild</key>\n\t<string>19F101</string>\n\t<key>CFBundleDevelopmentRegion</key>\n\t<string>English</string>\n\t<key>CFBundleExecutable</key>\n\t<string>HighPointIOP</string>\n\t<key>CFBundleGetInfoString</key>\n\t<string>Version: 4.4.5, Copyright (c) 2020 HighPoint Technologies, Inc.</string>\n\t<key>CFBundleIdentifier</key>\n\t<string>com.highpoint-tech.kext.HighPointIOP</string>\n\t<key>CFBundleInfoDictionaryVersion</key>\n\t<string>6.0</string>\n\t<key>CFBundleName</key>\n\t<string>HighPointIOP</string>\n\t<key>CFBundlePackageType</key>\n\t<string>KEXT</string>\n\t<key>CFBundleShortVersionString</key>\n\t<string>4.4.5</string>\n\t<key>CFBundleSignature</key>\n\t<string>????</string>\n\t<key>CFBundleSupportedPlatforms</key>\n\t<array>\n\t\t<string>MacOSX</string>\n\t</array>\n\t<key>CFBundleVersion</key>\n\t<string>4.4.5</string>\n\t<key>DTCompiler</key>\n\t<string>com.apple.compilers.llvm.clang.1_0</string>\n\t<key>DTPlatformBuild</key>\n\t<string>11E608c</string>\n\t<key>DTPlatformVersion</key>\n\t<string>GM</string>\n\t<key>DTSDKBuild</key>\n\t<string>19E258</string>\n\t<key>DTSDKName</key>\n\t<string>macosx10.15</string>\n\t<key>DTXcode</key>\n\t<string>1150</string>\n\t<key>DTXcodeBuild</key>\n\t<string>11E608c</string>\n\t<key>IOKitPersonalities</key>\n\t<dict>\n\t\t<key>MarvellSASIOP</key>\n\t\t<dict>\n\t\t\t<key>CFBundleIdentifier</key>\n\t\t\t<string>com.highpoint-tech.kext.HighPointIOP</string>\n\t\t\t<key>IOClass</key>\n\t\t\t<string>HPTMV9580IOPController</string>\n\t\t\t<key>IOKitDebug</key>\n\t\t\t<integer>0</integer>\n\t\t\t<key>IOMatchCategory</key>\n\t\t\t<string>IOService</string>\n\t\t\t<key>IOPCIPauseCompatible</key>\n\t\t\t<true/>\n\t\t\t<key>IOPCIPrimaryMatch</key>\n\t\t\t<string>0x45201103 0x45211103 0x45221103 0x63281103 0x63341103</string>\n\t\t\t<key>IOPCITunnelCompatible</key>\n\t\t\t<true/>\n\t\t\t<key>IOProviderClass</key>\n\t\t\t<string>IOPCIDevice</string>\n\t\t\t<key>IOResourceMatch</key>\n\t\t\t<string>IOKit</string>\n\t\t\t<key>IOUserClientClass</key>\n\t\t\t<string>HPTIOPControllerClient</string>\n\t\t\t<key>Physical Interconnect</key>\n\t\t\t<string>SAS</string>\n\t\t\t<key>Physical Interconnect Location</key>\n\t\t\t<string>External</string>\n\t\t</dict>\n\t\t<key>MarvellSATAIOP</key>\n\t\t<dict>\n\t\t\t<key>CFBundleIdentifier</key>\n\t\t\t<string>com.highpoint-tech.kext.HighPointIOP</string>\n\t\t\t<key>IOClass</key>\n\t\t\t<string>HPTMV9580IOPController</string>\n\t\t\t<key>IOKitDebug</key>\n\t\t\t<integer>0</integer>\n\t\t\t<key>IOMatchCategory</key>\n\t\t\t<string>IOService</string>\n\t\t\t<key>IOPCIPauseCompatible</key>\n\t\t\t<true/>\n\t\t\t<key>IOPCIPrimaryMatch</key>\n\t\t\t<string>0x36221103 0x36401103 0x36201103</string>\n\t\t\t<key>IOPCITunnelCompatible</key>\n\t\t\t<true/>\n\t\t\t<key>IOProviderClass</key>\n\t\t\t<string>IOPCIDevice</string>\n\t\t\t<key>IOResourceMatch</key>\n\t\t\t<string>IOKit</string>\n\t\t\t<key>IOUserClientClass</key>\n\t\t\t<string>HPTIOPControllerClient</string>\n\t\t\t<key>Physical Interconnect Location</key>\n\t\t\t<string>External</string>\n\t\t</dict>\n\t</dict>\n\t<key>LSMinimumSystemVersion</key>\n\t<string>10.9</string>\n\t<key>OSBundleLibraries</key>\n\t<dict>\n\t\t<key>com.apple.iokit.IOPCIFamily</key>\n\t\t<string>2.5</string>\n\t\t<key>com.apple.iokit.IOSCSIParallelFamily</key>\n\t\t<string>2.0</string>\n\t\t<key>com.apple.kpi.iokit</key>\n\t\t<string>10.0.0d1</string>\n\t\t<key>com.apple.kpi.libkern</key>\n\t\t<string>10.0.0d1</string>\n\t\t<key>com.apple.kpi.mach</key>\n\t\t<string>10.0.0d1</string>\n\t</dict>\n</dict>\n</plist>\n'

However, I do not know how to retrieve the missing header data.

[ydkhatri]

Hmm, the problem is those extra bytes you've highlighted in blue. The reference lzfse/lzvn implementation used for decompression requires the correct compressed size, any extra bytes will cause it to throw an error. There is already some correction code built into the _lzvn_decompress function. Hence you see the output in the log found end of stream, correcting now.. old size=1193 new size=1172, diff=21. So the size was corrected once, however that was still not the true end of file.

I'm speculating that the file was larger once, then edits to it made it smaller, and subsequent edits even smaller, and those additional end of stream markers are basically slack space. We could just modify the code to find the first instance of the lzvn_end_marker instead of the last. It might be a bit inefficient but it will fix the problem.

[mnrkbys]

Ok, I have modified the code and PRed #88 . Please check it.

[ydkhatri]

I've merged the changes now. Thanks