Handles unable to process sample Volatility 2 reports correctly. UnionType error?

[atcuno]

Describe the bug Running windows.handles on a memory sample that Volatility 2 supports fully causes a strange backtrace in Vol3:

# python3.8 vol.py -f data.lime windows.handles --pid 3704
Volatility 3 Framework 2.7.0
Progress:  100.00		PDB scanning finished                        
PID	Process	Offset	HandleValue	Type	GrantedAccess	Name
Traceback (most recent call last):
  File "/home/analyst/vol3/volatility3/framework/plugins/windows/handles.py", line 91, in _get_item
    if not self.context.layers[virtual].is_valid(handle_table_entry.Object):
  File "/home/analyst/vol3/volatility3/framework/objects/__init__.py", line 971, in __getattr__
    raise AttributeError(
AttributeError: UnionType has no attribute: symbol_table_name1!_HANDLE_TABLE_ENTRY.Object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "vol.py", line 10, in <module>
    volatility3.cli.main()
  File "/home/analyst/vol3/volatility3/cli/__init__.py", line 871, in main
    CommandLine().run()
  File "/home/analyst/vol3/volatility3/cli/__init__.py", line 469, in run
    renderer.render(grid)
  File "/home/analyst/vol3/volatility3/cli/text_renderer.py", line 198, in render
    grid.populate(visitor, outfd)
  File "/home/analyst/vol3/volatility3/framework/renderers/__init__.py", line 245, in populate
    for level, item in self._generator:
  File "/home/analyst/vol3/volatility3/framework/plugins/windows/handles.py", line 378, in _generator
    for entry in self.handles(object_table):
  File "/home/analyst/vol3/volatility3/framework/plugins/windows/handles.py", line 348, in handles
    for handle_table_entry in self._make_handle_array(TableCode, table_levels):
  File "/home/analyst/vol3/volatility3/framework/plugins/windows/handles.py", line 311, in _make_handle_array
    for x in self._make_handle_array(entry, level - 1, depth):
  File "/home/analyst/vol3/volatility3/framework/plugins/windows/handles.py", line 323, in _make_handle_array
    item = self._get_item(entry, handle_value)
  File "/home/analyst/vol3/volatility3/framework/plugins/windows/handles.py", line 111, in _get_item
    raise AttributeError(
AttributeError: Unable to find the SAR value for decoding handle table pointers

Context Volatility Version: latest develop Python Version: 3.8

[ikelos]

Closed by #1145

[atcuno]

@ikelos this one is not fixed by #1145 . This is a newly uncovered bug that triggered once I had 1145 code in place.

[ikelos]

Oh sorry, I saw them filed at the end same time and thought they were related. The lower exception says the SAR table couldn't be decoded so I think it might be one for you or @iMHLv2 ?

[eve-mem]

@atcuno - I think the UnionType error is a red herring, it's just symptomatic of how the logic works in this part of the handles plugin works.

This part is handling the pre windows 7 part, which will raise that Union error because handle_table_entry doesn't have an Object. The logic could be rewritten to check for handle_table_entry.Object rather than trying to access it and raising an error if it's easier? Happy to do that if it's useful but it's not really the cause of this issue. https://github.com/volatilityfoundation/volatility3/blob/1b3ba6ab2ef6f086993ad382f68d9f01a4ae7bdf/volatility3/framework/plugins/windows/handles.py#L89-L97

The real issue as @ikelos saw is the SAW table error which is being thrown on purpose as find_sar_value didn't return a magic.

Having a quick look at find_sar_value it will always return if capstone isn't found. At first I thought that might be as simple as that. https://github.com/volatilityfoundation/volatility3/blob/1b3ba6ab2ef6f086993ad382f68d9f01a4ae7bdf/volatility3/framework/plugins/windows/handles.py#L146-L148

The error you have though shows that capstone is there, so I think it'll be something more complex? (and therefore above my pay grade...! 😅) There are only a few routes for find_sar_value to return None - so hopefully it's not too difficult to work out. https://github.com/volatilityfoundation/volatility3/blob/1b3ba6ab2ef6f086993ad382f68d9f01a4ae7bdf/volatility3/framework/plugins/windows/handles.py#L108-L118

[eve-mem]

@atcuno Are you able to uncomment this line handy debug line in the handles plugin, and share the results on your sample that doesn't work?

https://github.com/volatilityfoundation/volatility3/blob/1b3ba6ab2ef6f086993ad382f68d9f01a4ae7bdf/volatility3/framework/plugins/windows/handles.py#L176

Here is the output on my sample that works:

272709641434864 5 mov qword ptr [rsp + 8], rbx
272709641434869 5 mov qword ptr [rsp + 0x10], rdi
272709641434874 5 mov rax, qword ptr [rsp + 0x30]
272709641434879 3 mov rdi, r9
272709641434882 3 mov rbx, r8
272709641434885 3 mov r11, rcx
272709641434888 3 add dword ptr [rax], 0x28
272709641434891 3 mov r10d, dword ptr [rax]
272709641434894 4 cmp r10d, 0x28
272709641434898 2 jae 0xf807294df71f
272709641434900 6 mov r9d, 0xc0000095
272709641434906 5 jmp 0xf807294df7c6
272709641434911 5 cmp dword ptr [rsp + 0x28], r10d
272709641434916 2 jae 0xf807294df731
272709641434918 6 mov r9d, 0xc0000004
272709641434924 5 jmp 0xf807294df7c6
272709641434929 3 mov rax, qword ptr [rcx]
272709641434932 3 xor r9d, r9d
272709641434935 3 mov r8, qword ptr [r8]
272709641434938 4 sar r8, 0x10

For your sample not to work there either needs to be no sar instruction in the capstone output, or the read on line 164 fails. If you get nothing printed out it'll mean the read failed, and that capstone output will let us work out why it's not ending up with a sar.

I've tested my sample using capstone 4.0.2, and then I updated to the latest 5.0.1 and they were both fine. I was wondering if perhaps a capstone update made a difference in the output of the disassembly.

Version info on my sample if it ends up being important:

Symbols <snip>/ntkrnlmp.pdb/D7ABE9B23BAD553213DE9BB10F1677B8-1.json.xz
Major/Minor     15.19041
MachineType     34404

If you wanted to dig into in volshell, I think you could call _decode_pointer with shifts from 0->48 and eyeball the results that look okay. It might even be an option to try and decode the pointers with various shifts and see which ones actually result in valid pointers - feels a little bit to heuristic though. It should mean you can at least get it running on your particular sample and get back to whatever the actual work is you wanted to do.

[eve-mem]

Hello @ikelos - I think this one still needs to stay open. Maybe auto closed by me referencing this issue in the PR? (sorry!) Don't think we've yet worked out why @atcuno sample doesn't work. The Extra debugging from https://github.com/volatilityfoundation/volatility3/pull/1147 might help diagnose it - but I think we still need to look into it.

At the moment I'm suspecting that the read fails due to the memory being paged out - but we need to see what is actually happening. If it is the memory being paged out, there is an option to make the handles plugin more resilient by trying to guess this shift value - but really just need to what is actually going on first.

[ikelos]

Ok, sorry it looks like it was linked to the PR (so when the PR gets merged it automatically gets closed).

[atcuno]

@eve-mem I got back to this today, it is the needed symbol being paged out.

[atcuno]

@eve-mem I got back to this today, it is the needed symbol being paged out.

I hardcoded it to 0x10 to match your disasm as a test, and I get the handles back

[eve-mem]

That makes sense at least. I'll try and make the plugin more resilient when i get the time.

Thanks for double checking it!

[atcuno]

@ikelos @eve-mem for this fix, can we go with catching that exception when the SAR decoding fails and then:

1. Emit a warning print that it could not be decoded and we are falling back to a common value of 0x10
2. Force the function to return 0x10

this way we will produce the correct result in most modern samples even if it can't be auto determined. If this path works then I can MR it pretty easily.

[eve-mem]

I can do that.