volatilityfoundation
Fixes transition-PDEs being treated as large pages
For windows, the overloaded _page_is_valid is used (from WindowsMixin), which also returns true if a PTE/PDE is in transition state. While this is fine for most cases (normal/4kb pages resp. PTEs), this does not work when checking for large pages. Large pages are not paged out (at least not so far; see Windows Internals 7th Edition Part 1, page 304), so they don't reach the transition state. Hence, _MMPTE_TRANSITION does not contain the LargePage bit, but instead contains the page protection where also the LargePage bit would be. _translate_entry does, however, check bit 7 (which is only for _MMPTE_HARDWARE the LargePage bit) in both cases, and hence treats a PDE in transition state as a large page, that actually references a Page Table.
Here an example for the translation of an affected virtual address. In the first case without the fix, in the second with the fix. The final PTE in the second case has a value of 0x0, so the exception is expected:
(primary_Process5032_1) >>> proc_layer.translate(0x7fff30400000) (5970591744, 'memory_layer')
With the fix:
(primary_Process5032_1) >>> proc_layer.translate(0x7fff30400000)
Traceback (most recent call last):
File "
Hiya, thanks for this. I'm a little concerned that to fix a windows problem it's changing the logic for all operating systems, and may look redundant given that the normal _page_is_valid already does this test?
So my question is, would:
def _page_is_valid(entry):
"""Check for bit 11 (windows specific) and bit 10 (also windows specific) but ensure bit 7 is not set"""
return bool((entry & 1) or ((entry & 1 << 11) and not (entry & 1 << 10) and not (entry & 1 << 7)))
have the same effect, or is that only true when it's a large page (in which case, we could pass that into the _page_is_valid method as a parameter)?
I disagree with the decision Microsoft made to make use of pages that are marked as not present, and I'm keen to avoid their decision impacting the speed/flow for other operating systems, even if it's just one extra quick check. It's carried out every time a page translation happens, so the more streamlined we can make it, the better...
Hi,
yes you are right, with this fix you would do the test for the valid bit twice, and no, your suggested change won't work, because you are checking the bit 7 for the transition state, and this bit is in this case part of the protection field, so depending on the protection of the corresponding page, bit 7 can be set or not. It does, however, not have anything to do with large pages.
To illustrate, this is the PTE in hardware state, with the LargePage field (bit 7):
[_MMPTE_HARDWARE] @ 0x75acafa0
0x0 Accessed 0x0 bitfield (bit 5)
0x0 CacheDisable 0x0 bitfield (bit 4)
0x0 CopyOnWrite 0x0 bitfield (bit 9)
0x0 Dirty 0x0 bitfield (bit 6)
0x0 Dirty1 0x0 bitfield (bit 1)
0x0 Global 0x0 bitfield (bit 8)
0x0 LargePage 0x1 bitfield (bit 7)
0x0 NoExecute 0x1 bitfield (bit 63)
0x0 Owner 0x1 bitfield (bit 2)
0x0 PageFrameNumber 0x3165 bitfield (bits 12-48)
0x0 SoftwareWsIndex 0x232 bitfield (bits 52-63)
0x0 Unused 0x0 bitfield (bit 10)
0x0 Valid 0x1 bitfield (bit 0)
0x0 Write 0x0 bitfield (bit 11)
0x0 WriteThrough 0x0 bitfield (bit 3)
0x0 reserved1 0x0 bitfield (bits 48-52)
And here in transition state:
[_MMPTE_TRANSITION] @ 0x66fa3968
0x0 CacheDisable 0x0 bitfield (bit 4)
0x0 PageFrameNumber 0x1e749 bitfield (bits 12-48)
0x0 Protection 0x4 bitfield (bits 5-10)
0x0 Prototype 0x0 bitfield (bit 10)
0x0 Spare 0x1 bitfield (bit 2)
0x0 Transition 0x1 bitfield (bit 11)
0x0 Unused 0x8830 bitfield (bits 48-64)
0x0 Valid 0x0 bitfield (bit 0)
0x0 Write 0x1 bitfield (bit 1)
0x0 WriteThrough 0x0 bitfield (bit 3)
As you can see, bit 7 is here part of the Protection field.
One way to reduce the double checks would be to change my fix from:
if large_page and (entry & 1) and (entry & (1 << 7)):
to this:
if large_page and (entry & (1 << 7)) and (entry & 1):
This would mean that we only check the valid bit twice for PTEs in transition state and actual large pages. I did the effort of doing a quick evaluation with a VM running several processes (browsers, office, ...) and deactivated the pagefile, so pages are not paged out but end up at max in the transition state (to create a potential worst case scenario for the test). I invoked the translation for each virtual address for each VAD for each process, which resulted in a total of 4.244.510 virtual addresses.
The amount of (entry & 1) checks in the first version of the if statement were 12.260.351, in the second version only 74.004. Obviously this is not representative, but it at least shows that this can significantly reduce the double checks.
The only other alternative that I can currently think of is to not "overload" the original page_is_valid and to add this Windows function:
page_is_in_transition: Checks for the transition state.
Then it would be possible to fix this issue without any double checks.
But then, page_is_valid will not return true for pages in transition state. This could be fixed by adding another function:
page_is_mapped: Which would do the same as the current page_is_valid, so checking for valid and transition. In consequence, the usage of page_is_valid should then be replaced by page_is_mapped where appropriate.
But I'm not sure if that's the option you want to go for.
Cheers, Frank
Hiya, just as an update I'm sorry I've taken so long, but I'm still considering more/better page fault handling code than we have at the moment.
Sorry it took me a while, but I've worked on the page fault handler mechanism, and I'm now happier that it can be kept more separate from the Intel layers (which should behave like the hardware) The pull request allows for faults to be handled in a single method (which in windows is currently used to handle transition pages as well as swap pages). It doesn't directly follow the hardware page handler (which sets CR2 and then provides an error code indicating why the fault occurred), but I think we pass essentially the same information and make it easier for the worklfow of those trying to duplicate architectures.
Let me know what you think, very happy for revisions, modifications, and I'd be interested to verify that it solves the same issue this PR is trying solve...