acquire Acquire collection of a Windows disk image results in a damaged tar file

The target is Windows system stored in the Encase Image format.

target-info <TARGET>.E01
XXX [error    ] Failed to open an encrypted volume <Volume name='Basic data partition' size=XXXX fs=None> with volume manager bitlocker: Failed to unlock BDE volume [dissect.target.volume]
XXX [warning  ] <Target <TARGET>.E01>: Can't identify filesystem: <Volume name='Microsoft Reserved' size=XXX fs=None> [dissect.target.target]
<Target <TARGET>.E01>

Disks
- <Disk type="EwfContainer" size="XXX">

Volumes
- <Volume name="EFI system partition" size="XXX" fs="FatFilesystem">
- <Volume name="Microsoft Reserved" size="XXX" fs="NoneType">
- <Volume name="part_0d900000" size="XXX" fs="NtfsFilesystem">
- <Volume name="part_1b8eb00000" size="XXX" fs="NtfsFilesystem">
- <Volume name="Basic data partition" size="XXX" fs="NoneType">

Mounts
- <Mount fs="NtfsFilesystem" path="sysvol">
- <Mount fs="FatFilesystem" path="efi">
- <Mount fs="NtfsFilesystem" path="c:">
- <Mount fs="NtfsFilesystem" path="/$fs$/fs0">

[....]

Using Acquire on the disk image results in a damaged tar file. This issue does not occur with a ZIP collection --output-type zip

acquire --profile minimal <TARGET>.E01
[...]

[XXX] [INFO ] - Collecting file sysvol/windows/system32/config/SAM to: fs/C:/windows/system32/config/SAM
[XXX] [INFO ] - Collecting file sysvol/windows/system32/config/SAM succeeded
[XXX] [INFO ] - Collecting file sysvol/windows/system32/config/SAM to: fs/C:/windows/system32/config/SAM
[XXX] [INFO ] - Collecting file sysvol/windows/system32/config/SAM succeeded
[XXX] [INFO ] - Collecting file sysvol/windows/system32/config/SECURITY to: fs/C:/windows/system32/config/SECURITY
[XXX] [INFO ] - Collecting file sysvol/windows/system32/config/SECURITY succeeded
[XXX] [INFO ] - Collecting file sysvol/windows/system32/config/SOFTWARE to: fs/C:/windows/system32/config/SOFTWARE
[XXX] [INFO ] - Collecting file sysvol/windows/system32/config/SOFTWARE succeeded

tar -tf <TARGET>.tar
fs/C:/$MFT
fs/C:/$Boot
fs/C:/$Secure:$SDS
fs/C:/$Extend/$Usnjrnl:$J
fs/$fs$/fs0/$MFT
fs/$fs$/fs0/$Boot
fs/$fs$/fs0/$Secure:$SDS

1. The output shows an incomplete collection that does not represent the outcome of the Acquire log.
   [...]

1. The log below is repeated multiple times, but in some cases it is not shown.
   tar: Damaged tar archive
   tar: Retrying...

diff --git i/acquire/outputs/tar.py w/acquire/outputs/tar.py
index 69b7290..d9f0755 100644
— i/acquire/outputs/tar.py
+++ w/acquire/outputs/tar.py
@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import io
+import logging
 import tarfile
 from typing import TYPE_CHECKING, BinaryIO
 
@@ -14,6 +15,8 @@ if TYPE_CHECKING:
 
 TAR_COMPRESSION_METHODS = {"gzip": "gz", "bzip2": "bz2", "xz": "xz"}
 
+log = logging.getLogger("acquire")
+
 
 class TarOutput(Output):
     """Tar archive acquire output format. Output can be compressed and/or encrypted.
@@ -100,7 +103,11 @@ class TarOutput(Output):
             if stat:
                 info.mtime = stat.st_mtime
 
-        self.tar.addfile(info, fh)
  +        try:
  +            self.tar.addfile(info, fh)
  +        except Exception as e:
  +            log.warning("Error occurred for %s with the size %i", info, info.size)
  +            log.debug("", exc_info=e)
 
     def close(self) -> None:
         """Closes the tar file.""

While debugging the issue the logs shows the errors shown below. The errors are not consistent.

acquire --profile minimal <TARGET>.E01 -vvv
[XXX] [INFO ] Successfully copied fs/C:/windows/system32/config/DEFAULT
[...]

[XXX] [WARNING] Error occurred for <TarInfo 'fs/C:/WINDOWS/system32/winevt/logs/Microsoft-Windows-Store%4Operational.evtx' at 0x328025900> with the size 20058112
[2025-08-12 14:19:27,574] [DEBUG] 
Traceback (most recent call last):
  File "/acquire/.venv/lib/python3.13/site-packages/dissect/ntfs/stream.py", line 103, in _read
    read_list.append(lznt1.decompress(io.BytesIO(b"".join(buf)))[: self.compression_unit_size])
                     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/acquire/.venv/lib/python3.13/site-packages/dissect/util/compression/lznt1.py", line 61, in decompress
    header = ord(src.read(1))
TypeError: ord() expected a character, but string of length 0 found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/acquire/acquire/outputs/tar.py", line 107, in write
    self.tar.addfile(info, fh)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/3.13/lib/python3.13/tarfile.py", line 2292, in addfile
    copyfileobj(fileobj, self.fileobj, tarinfo.size, bufsize=bufsize)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/3.13/lib/python3.13/tarfile.py", line 250, in copyfileobj
    buf = src.read(bufsize)
  File "/acquire/.venv/lib/python3.13/site-packages/dissect/util/stream.py", line 141, in read
    self._fill_buf()
    ~~~~~~~~~~~~~~^^
  File "/dissect/util/stream.py", line 59, in _fill_buf
    self._buf = self._read(self._pos_align, self.align)
                ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dissect/ntfs/stream.py", line 105, in _read
    raise IOError("Decompression failed")
OSError: Decompression failed

[2025-08-12 14:19:29,362] [WARNING] Error occurred for <TarInfo 'fs/C:/WINDOWS/system32/winevt/logs/System.evtx' at 0x328145a80> with the size 20975616
[2025-08-12 14:19:29,362] [DEBUG] 
Traceback (most recent call last):
  File "/acquire/acquire/outputs/tar.py", line 107, in write
    self.tar.addfile(info, fh)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/3.13/lib/python3.13/tarfile.py", line 2292, in addfile
    copyfileobj(fileobj, self.fileobj, tarinfo.size, bufsize=bufsize)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/3.13/lib/python3.13/tarfile.py", line 252, in copyfileobj
    raise exception("unexpected end of data")
OSError: unexpected end of data

Aug 13 '25 07:08 respondersGY

Related: #261.

Looks like some kind of edge case in our NTFS implementation. Can you provide some debug information from the MFT record of fs/C:/WINDOWS/system32/winevt/logs/Microsoft-Windows-Store%4Operational.evtx? For example what kind of attributes it has?

Aug 13 '25 08:08 Schamper

Is there other information that you need?

In [17]: r = fs.ntfs.mft.get("/WINDOWS/system32/winevt/logs/Microsoft-Windows-Store%4Operational.evtx")

In [18]: r.attributes
Out[18]: {16: [<$STANDARD_INFORMATION name=>], 32: [<$ATTRIBUTE_LIST name=>], 48: [<$FILE_NAME name=>, <$FILE_NAME name=>], 128: [<$DATA name=>, <$DATA name=>, <$DATA name=>, <$DATA name=>]}

In [20]: r.resident
Out[20]: False

In [21]: r.filename
Out[21]: 'Microsoft-Windows-Store%4Operational.evtx'

Aug 13 '25 08:08 respondersGY

It's a bit hard to remote debug-over-GitHub-issue, but if you could poke around these pieces of code, maybe you can find some inconsistencies:

https://github.com/fox-it/dissect.ntfs/blob/ea6a0ae3118c0bd4daa98f392e5467db2385e12a/dissect/ntfs/util.py#L127-L134

The multiple $DATA attributes should get "consolidated" and opened for reading here, you can inspect the headers for maybe some weird flags?

https://github.com/fox-it/dissect.ntfs/blob/ea6a0ae3118c0bd4daa98f392e5467db2385e12a/dissect/ntfs/stream.py#L64

lznt1.decompress fails because of EOF (fh.read(1) returned b""), maybe you can inspect the flow of CompressedRunlistStream for abnormalities (weird buffer sizes or other unexpected values).

Aug 13 '25 08:08 Schamper

The CompressedRunlistStream.size is not the same as the actual size of the record therefore decompression fails.

In [16]: fh = r.open()

In [17]: len(fh.runlist)
Out[17]: 619

In [18]: fh.size
Out[18]: 20058112

In [19]: fh.read(10)
Out[19]: b'ElfFile\x00\x16\x01'

In [20]: fh.read(fh.size)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File /dissect.ntfs/dissect/ntfs/stream.py:103, in CompressedRunlistStream._read(self, offset, length)
    102     buf.append(b"\x00" * 64)
--> 103     read_list.append(lznt1.decompress(io.BytesIO(b"".join(buf)))[: self.compression_unit_size])
    104 except Exception:

File /dissect/util/compression/lznt1.py:61, in decompress(src)
     60 while src.tell() < block_end:
---> 61     header = ord(src.read(1))
     62     for mask in TAG_MASKS:

TypeError: ord() expected a character, but string of length 0 found

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
Cell In[20], line 1
----> 1 fh.read(fh.size)

File /dissect/util/stream.py:135, in AlignedStream.read(self, n)
    132     count, n = divmod(n, align)
    134     read_len = count * align
--> 135     r.append(self._read(self._pos, read_len))
    137     self._set_pos(self._pos + read_len)
    139 # Misaligned end

File /dissect.ntfs/dissect/ntfs/stream.py:105, in CompressedRunlistStream._read(self, offset, length)
    103         read_list.append(lznt1.decompress(io.BytesIO(b"".join(buf)))[: self.compression_unit_size])
    104     except Exception:
--> 105         raise IOError("Decompression failed")
    106 else:
    107     # Uncompressed CU
    108     read_list.append(b"".join(buf))

OSError: Decompression failed

Aug 19 '25 14:08 respondersGY

The decompression failing is a red herring. The buffer it passes to the decompression function is empty, we need to figure out why it's empty.

Aug 19 '25 14:08 Schamper