volatilityfoundation
windows.psscan: Error only when using --physical
vol -vvvvvvvvvvvvv -f banking-malware.vmem windows.psscan
Level 6 volatility3.framework.objects: Void size requested
Level 6 volatility3.framework.objects: Void size requested
Level 6 volatility3.framework.objects: Void size requested
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_EPROCESS_QUOTA_BLOCK
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_PAGEFAULT_HISTORY
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_PSP_CPU_QUOTA_APC
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_JOB_ACCESS_STATE
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ACTIVATION_CONTEXT_DATA
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_FLS_CALLBACK_INFO
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ASSEMBLY_STORAGE_MAP
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_SCSI_REQUEST_BLOCK
3488 636 WmiPrvSE.exe 0x7cf2fb00 16 329 0 False 2020-11-15 03:24:41.000000 N/A Disabled
1832 500 dllhost.exe 0x7d203930 21 195 0 False 2020-11-15 03:24:29.000000 N/A Disabled
3200 500 SDXHelper.exe 0x7d24d2f0 16 302 1 False 2021-02-09 00:51:12.000000 N/A Disabled
3196 500 SDXHelper.exe 0x7d24da30 16 302 1 False 2021-02-09 00:51:12.000000 N/A Disabled
3028 360 conhost.exe 0x7d2508c0 2 33 0 False 2021-02-09 00:51:13.000000 N/A Disabled
2044 500 dllhost.exe 0x7d2735e0 19 207 0 False 2020-11-15 03:24:30.000000 N/A Disabled
2104 500 VSSVC.exe 0x7d2e5060 6 119 0 False 2020-11-15 03:24:31.000000 N/A Disabled
3188 636 dllhost.exe 0x7d2f3580 7 138 1 False 2021-02-09 00:51:25.000000 N/A Disabled
2192 500 taskhost.exe 0x7d33eb00 12 218 1 False 2020-11-15 03:24:33.000000 N/A Disabled
vol -vvvvvvvvvvvvv -f banking-malware.vmem windows.psscan --physical
Level 6 volatility3.framework.objects: Void size requested
Level 6 volatility3.framework.objects: Void size requested
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_EPROCESS_QUOTA_BLOCK
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_PAGEFAULT_HISTORY
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_PSP_CPU_QUOTA_APC
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_JOB_ACCESS_STATE
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ACTIVATION_CONTEXT_DATA
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_FLS_CALLBACK_INFO
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ASSEMBLY_STORAGE_MAP
DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_SCSI_REQUEST_BLOCK
DEBUG volatility3.cli: Traceback (most recent call last):
File "/src/volatility3/volatility3/cli/__init__.py", line 333, in run
renderers[args.renderer]().render(constructed.run())
File "/src/volatility3/volatility3/cli/text_renderer.py", line 178, in render
grid.populate(visitor, outfd)
File "/src/volatility3/volatility3/framework/renderers/__init__.py", line 211, in populate
for (level, item) in self._generator:
File "/src/volatility3/volatility3/framework/plugins/windows/psscan.py", line 183, in _generator
(_, _, offset, _, _) = list(memory.mapping(offset = proc.vol.offset, length = 0))[0]
File "/src/volatility3/volatility3/framework/layers/intel.py", line 203, in mapping
for offset, size, mapped_offset, mapped_size, map_layer in self._mapping(offset, length, ignore_errors):
File "/src/volatility3/volatility3/framework/layers/intel.py", line 236, in _mapping
mapped_offset, _, layer_name = self._translate(offset)
File "/src/volatility3/volatility3/framework/layers/intel.py", line 373, in _translate
return self._translate_swap(self, offset, self._bits_per_register // 2)
File "/src/volatility3/volatility3/framework/layers/intel.py", line 326, in _translate_swap
return super()._translate(offset)
File "/src/volatility3/volatility3/framework/layers/intel.py", line 105, in _translate
entry, position = self._translate_entry(offset)
File "/src/volatility3/volatility3/framework/layers/intel.py", line 134, in _translate_entry
raise exceptions.PagedInvalidAddressException(self.name, offset, position + 1, entry,
volatility3.framework.exceptions.PagedInvalidAddressException: Page Fault at entry 0x0 in table page table
Volatility was unable to read a requested page:
Page error 0x7cf2fb00 in layer layer_name (Page Fault at entry 0x0 in table page table)
* Memory smear during acquisition (try re-acquiring if possible)
* An intentionally invalid page lookup (operating system protection)
* A bug in the plugin/volatility3 (re-run with -vvv and file a bug)
No further results will be produced
This issue is stale because it has been open for 200 days with no activity.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue was closed because it has been inactive for 60 days since being marked as stale.
Hello @garanews - I missed this before and only looked into it when the bot closed it.
I think this is a issue with the psscan plugin and how it works out virtual offsets once it has found a process via scanning.
In fact I think the --physical should work, as it's the calculation of virtual offsets that's the issue.
e.g. in your output here I believe these addresses (0x7cf2fb00, 0x7d203930, 0x7d24d2f0, etc) are actually physical even though they're under the Offset(V) heading.
3488 636 WmiPrvSE.exe 0x7cf2fb00 16 329 0 False 2020-11-15 03:24:41.000000 N/A Disabled
1832 500 dllhost.exe 0x7d203930 21 195 0 False 2020-11-15 03:24:29.000000 N/A Disabled
3200 500 SDXHelper.exe 0x7d24d2f0 16 302 1 False 2021-02-09 00:51:12.000000 N/A Disabled
3196 500 SDXHelper.exe 0x7d24da30 16 302 1 False 2021-02-09 00:51:12.000000 N/A Disabled
Here is a worked example on a sample I have to show this. This output shows that without the --physical flag the plugin runs and shows output. Note the "virtual" offset shown for iexplore.exe is 0x14b13b0
$ python vol.py -f win-xp-laptop-2005-06-25.img windows.psscan
Volatility 3 Framework 2.5.2
Progress: 100.00 PDB scanning finished
PID PPID ImageFileName Offset(V) Threads Handles SessionId Wow64 CreateTime ExitTime File output
2536 580 mqtgsvc.exe 0x1343790 9 119 0 False 2005-06-25 16:48:05.000000 N/A Disabled
2392 1812 iexplore.exe 0x14b13b0 9 365 0 False 2005-06-25 16:51:02.000000 N/A Disabled
2740 944 PluckTray.exe 0x1ed76b0 3 105 0 False 2005-06-25 16:51:10.000000 N/A Disabled
4012 2624 dd.exe 0x1ed84e8 1 22 0 False 2005-06-25 16:58:46.000000 N/A Disabled
3076 1812 PluckUpdater.ex 0x1f269e0 0 - 0 False 2005-06-25 16:51:15.000000 2005-06-25 16:51:30.000000 Disabled
1400 580 tcpsvcs.exe 0x1f48da0 2 94 0 False 2005-06-25 16:47:58.000000 N/A Disabled
Here now trying to use the --physical flag we get a page fault:
$ python vol.py -f win-xp-laptop-2005-06-25.img windows.psscan --physical
Volatility 3 Framework 2.5.2
Progress: 100.00 PDB scanning finished
PID PPID ImageFileName Offset(P) Threads Handles SessionId Wow64 CreateTime ExitTime File output
Volatility was unable to read a requested page:
Page error 0x1343790 in layer layer_name (Page Fault at entry 0x0 in table page table)
* Memory smear during acquisition (try re-acquiring if possible)
* An intentionally invalid page lookup (operating system protection)
* A bug in the plugin/volatility3 (re-run with -vvv and file a bug)
No further results will be produced
The fault is on 0x1343790 which just so happens to match the "Virtual" offset of the first process that is returned.
PID PPID ImageFileName Offset(V) Threads Handles SessionId Wow64 CreateTime ExitTime File output
2536 580 mqtgsvc.exe 0x1343790 9 119 0 False 2005-06-25 16:48:05.000000 N/A Disabled
To really confirm this here is a session in volshell.
# get the iexplore.exe proc
(layer_name) >>> for proc in ps():
... if proc.UniqueProcessId == 2392:
... break
...
# prove to myself it is the correct proc
(layer_name) >>> proc.UniqueProcessId
2392
(layer_name) >>> proc.ImageFileName.cast(
... "string",
... max_length=proc.ImageFileName.vol.count,
... errors="replace",
... )
'iexplore.exe'
# get it's virtual offset
(layer_name) >>> hex(proc.vol.offset)
'0x814b13b0'
# get the intel layer and translate that virtual offset to a physical one
(layer_name) >>> intel_layer = context.layers['layer_name']
(layer_name) >>> intel_layer.translate(proc.vol.offset)
(21697456, 'memory_layer')
# convert that physical offset to hex
(layer_name) >>> hex(21697456)
'0x14b13b0'
See how 0x14b13b0 matches the psscan output when it is supposed to be showing virtual addresses.
So the issue is that the plugin can show physical addresses correctly, but the logic for the command line is flipped (at least in these samples). Then when it comes to actually translating to a virtual address that is where we hit a problem. I have some ideas on how to fix it, but just documenting this here in case someone wants to jump in a fix it first.
Thanks!
Hello I've been looking into this a little - it's interesting.
It looks like that the windows poolscanner will in some cases (I think modern windows version) scan a 'virtual' layer rather than a 'physical' one. That means when that the objects that get made are created on this virtual layer. However when the scanning happens on a 'physical' the object is built on that physical layer, with it's native layer being correctly set to the virtual one.
It means if we read out pointers etc from the object the native layer being set correct means we get the right answer.
However it means that object.vol.offset is a reference to the physical layer - and that's why @garanews was having this issue.
Here is a rough attempt at working around this, just changing windows.psscan. https://github.com/eve-mem/volatility3/tree/windows_psscan_issue_591
With those changes it displays the correct virtual / physical offsets as requested by the user without crashing.
Physical:
$ python vol.py -f win-xp-laptop-2005-06-25.img windows.psscan --physical
Volatility 3 Framework 2.5.2
Progress: 100.00 PDB scanning finished
PID PPID ImageFileName Offset(P) Threads Handles SessionId Wow64 CreateTime ExitTime File output
2536 580 mqtgsvc.exe 0x1343790 9 119 0 False 2005-06-25 16:48:05.000000 N/A Disabled
2392 1812 iexplore.exe 0x14b13b0 9 365 0 False 2005-06-25 16:51:02.000000 N/A Disabled
2740 944 PluckTray.exe 0x1ed76b0 3 105 0 False 2005-06-25 16:51:10.000000 N/A Disabled
4012 2624 dd.exe 0x1ed84e8 1 22 0 False 2005-06-25 16:58:46.000000 N/A Disabled
3076 1812 PluckUpdater.ex 0x1f269e0 0 - 0 False 2005-06-25 16:51:15.000000 2005-06-25 16:51:30.000000 Disabled
<snip>
Virtual
$ python vol.py -f win-xp-laptop-2005-06-25.img windows.psscan
Volatility 3 Framework 2.5.2
Progress: 100.00 PDB scanning finished
PID PPID ImageFileName Offset(V) Threads Handles SessionId Wow64 CreateTime ExitTime File output
2536 580 mqtgsvc.exe 0x81343790 9 119 0 False 2005-06-25 16:48:05.000000 N/A Disabled
2392 1812 iexplore.exe 0x814b13b0 9 365 0 False 2005-06-25 16:51:02.000000 N/A Disabled
2740 944 PluckTray.exe 0x81ed76b0 3 105 0 False 2005-06-25 16:51:10.000000 N/A Disabled
4012 2624 dd.exe 0x81ed84e8 1 22 0 False 2005-06-25 16:58:46.000000 N/A Disabled
3076 1812 PluckUpdater.ex 0x81f269e0 0 - 0 False 2005-06-25 16:51:15.000000 2005-06-25 16:51:30.000000 Disabled
<snip>
1960 1812 Fast.exe - 1 22 0 False 2005-06-25 16:47:48.000000 N/A Disabled
448 4 smss.exe - 3 21 N/A False 2005-06-25 16:47:28.000000 N/A Disabled
1224 580 spoolsv.exe - 12 136 0 False 2005-06-25 16:47:39.000000 N/A Disabled
Note that when virtual addresses are requested, sometimes the virtual_process_from_physical isn't able to build the virtual process so no offset is displayed. I haven't looked at virtual_process_from_physical to see if there are ways to improve that.
I don't have enough modern windows samples to be sure that these changes work correctly there too. I'd welcome any testing someone else can do, I'll try and hunt down some samples myself to test on as well. I won't raise a PR until I'm happy that works.
Also during the process of looking into this I had two thoughts. What do you think of these @ikelos? Maybe better in a different issue perhaps?
-
Would it be useful to add a
object.vol.native_offsetto objects in a similar was thatobject.vol.offsetworks? Except it returns the offset of the object in the native layer (if one can be calculated)? Might be harder than it first seems to do that. That could make the logic here a lot easier, and might be useful / reusable elsewhere in the framework? -
When using an
UnreadableValue(or other vol type like that) for something that can be read it can't be formatted byformat_hints.Hex()as it's not an int, it raises an exception. Would it be useful to catch that in format_hints itself and just return theUnreadableValue? It would work when the formatting happens just before a plugin yields a result - but maybe it causes problems elsewhere in the framework where it would be expecting an int etc?
Errr, so off the top of my head, I'm not sure it's possible to calculate the native_offset in a virtual layer with only the offset in the physical layer. The original reason for this was so that objects could be constructed in the physical layer, but that any pointers they contained could be followed and automatically end up in the right layer. There isn't really a way to offload the reverse mapping (which might not be unique) to go from virtual to physical, so I think this one is out unfortunately...
For number 2, I think that already happens? The Renderers are part of the CLI, and the type map applied the optional wrapper to each of the format_hints, so as the format_hint is renderer, it checks whether it's a BaseAbsentValue and then prints it appropriately. The format hints should only be applied just before rendering and the BaseAbsentValues are just normal objects, and the format hints are type casts, so the actual value isn't ever processed (or shouldn't be)? Do you have a concrete example where this is going wrong? I'd expect BaseAbsentValues to be completely broken if hints barfed whenever they appeared in the results?
I'm just trying to trace the origin of this, and it looks like it was added as #584. Most scanners only ever scan the physical space, although this one did apparently always scan the virtual layer? I'm still not quite following how the problem arose, and I wonder if we're not overthinking it? Perhaps we should remove the flag and only ever return the physical offset?
Ok, so the culprit is the generate_pool_scan method which scans different layers depending upon the version of windows in use. That's very awkward because then it's hard to tell what the column title should be before you generate the results... 5:S
To make sure things are in sync, rather than doing our own jiggery pokery in the middle of it, we should probably call the same function to figure out what the output's going to be, so set the appropriate P/V for the offset in the column title. It's a bit ugly (because you then have to know the internal workings of generate_pool_scan, but the other option is wait until the result and then determine if the native_layer matches, and in some cases you can't then form the right mapping. This got real ugly real quick. I don't recall why generate_pool_scan runs on virtual layers in windows 10.
Alright, so more digging and it seems @iMHLv2 originally committed this which scans the virtual layer for windows 10, but... sadly... without documentation as to why... 5:S I assume it was to speed up scanning in some way, but I don't recall.
So essentially generate_pool_scan operates just fine, but returns objects that may have been made on the physical layer or may have been made on the virtual layer. I think that's all I've got time to dig at the moment, but I'll have a ponder over the best way to fix it. @eve-mem 's branch looks reasonable but also seems more complex than I'd like, so I'll keep thinking for now...
Yes, as i started to get my head around it I did question why it would return a virtual address in a scanner, but didn't fancy removing functionality.
I agree with my branch looking complex - i couldn't think of a better way to handle it without messing around outside of the plugin. Hopefully you come up with a smart idea. 😀
@ikelos Re the native layer offset you're right. I was only thinking about a simple case with addresses that would translate neatly into kernel memory. Just need to look at the strings plugin to realise that's not as simple generically as i was thinking this morning.
Re the format hint, i think i was having problems with it, but let me go away and test properly and double check. I'll let you know if i think there is an issue.