pdbconv broke on NDIS PDB from Win10 x64 version 17763

[atcuno]

While working on a new plugin, I triggered this backtrace when the code was attempting to lookup a symbol in NDIS.sys (a kernel driver):

[snip]
File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbutil.py", line 411, in symbol_table_from_pdb
    _, symbol_table_name = cls._modtable_from_pdb(
                           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbutil.py", line 454, in _modtable_from_pdb
    symbol_table_name = cls.load_windows_symbol_table(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbutil.py", line 114, in load_windows_symbol_table
    cls.download_pdb_isf(
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbutil.py", line 277, in download_pdb_isf
    ).get_json()
      ^^^^^^^^^^
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 505, in get_json
    self.read_necessary_streams()
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 214, in read_necessary_streams
    self.read_tpi_stream()
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 222, in read_tpi_stream
    type_references = self._read_info_stream(2, "TPI", self.types)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 285, in _read_info_stream
    output, consumed = self.consume_type(module, offset, length)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 775, in consume_type
    subfield, sub_consumed = self.consume_type(
                             ^^^^^^^^^^^^^^^^^^
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 821, in consume_type
    raise TypeError(f"Unhandled leaf_type: {leaf_type}")
TypeError: Unhandled leaf_type: 5392

I then pulled the pdb file down directly:

http://msdl.microsoft.com/download/symbols/ndis.pdb/67635B4B9BD13A95BC64AD0F906827AC1/ndis.pdb

and ran pdbconv on it, and got another error but this time at a different leaf type:

$ PYTHONPATH=. python3 volatility3/framework/symbols/windows/pdbconv.py -f ../ndis.pdb -o ../t.txt
Traceback (most recent call last):ading TPI layer
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 1073, in <module>
    converted_json = convertor.get_json()
                     ^^^^^^^^^^^^^^^^^^^^
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 505, in get_json
    self.read_necessary_streams()
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 212, in read_necessary_streams
    self.read_pdb_info_stream()
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 447, in read_pdb_info_stream
    self.read_ipi_stream()
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 236, in read_ipi_stream
    type_references = self._read_info_stream(4, "IPI", ipi_list)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 285, in _read_info_stream
    output, consumed = self.consume_type(module, offset, length)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rk/volatility3/volatility3/framework/symbols/windows/pdbconv.py", line 821, in consume_type
    raise TypeError(f"Unhandled leaf_type: {leaf_type}")
TypeError: Unhandled leaf_type: 5634

Looking at LVM: https://docs.hdoc.io/hdoc/llvm-project/e6DFC71D5E32DD199.html

5634 = MemberFuncId 5392 = NestedType

Are there plans to support these? Also, is there a way to specify not backtracing if types can't be parsed, since in this case I only care about symbols?

[ikelos]

I suspect this is a C++ library, it's been known for a while that pdbconv doesn't support this (see #650 and #540 ) but the work somewhat stalled because the extra types require a bit of thought to integrate and there wasn't a great deal of interest in getting them sorted. I could probably use some support regarding exactly what needs doing, and we'll need to make a decision whether/how to unmunge C++ names for use in Volatility (theoretically there's support for having an alias, so we can record both, but which one takes precedence and how to we let people access them via the other, etc, etc).

[IridiumXOR]

Please fix it, at least with a workaround to continue the PDB parsing... there are others issues about this problem from 2020...

[ikelos]

If we produce a workaround that's wrong, then we have potentially a whole bunch of seemingly complete symbol tables, that won't have the right types and structures in, which will later require us to invalidate whole swathes of ISF files. We've seen this issue in dwarf2json output that wasn't complete for a feature we later needed, and it took a lot of time to resolve.

Given volatility automatically generates windows ISF files, and only once, this would be a big undertaking to fix when we eventually come to need the correct types in place. C++ types aren't required for analysis of the kernel and no one's had the time to step up and implement a suitable fix for this suggesting that the issue doesn't affect a significant number of people trying to use volatility. If you'd like to contribute a fix, we'll be happy to review it and offer suggestions to help get it merged into the tree.

[github-actions[bot]]

This issue is stale because it has been open for 200 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 60 days since being marked as stale.