volatility icon indicating copy to clipboard operation
volatility copied to clipboard

Windows 10 x64 psscan error: ^: 'int' and 'NoneType'

Open gleeda opened this issue 7 years ago • 33 comments

Note: Win10x64_14393 is the correct profile for this memory sample.

$ python vol.py -f Windows\ 10\ x64-c4aa8f1b.vmem --profile=Win10x64_14393 psscan
Volatility Foundation Volatility Framework 2.6
Offset(P)          Name                PID   PPID PDB
Time created                   Time exited
------------------ ---------------- ------ ------ ------------------
------------------------------ ------------------------------
WARNING : volatility.debug    : Cannot find nt!ObGetObjectType
WARNING : volatility.debug    : Cannot find nt!ObGetObjectType
Traceback (most recent call last):
  File "vol.py", line 192, in <module>
    main()
  File "vol.py", line 183, in main
    command.execute()
  File "volatility/volatility/commands.py", line 147, in execute
    func(outfd, data)
  File "volatility/volatility/plugins/filescan.py", line 423, in render_text
    for eprocess in data:
  File "volatility/volatility/poolscan.py", line 252, in scan
    skip_type_check = skip_type_check)
  File "volatility/volatility/plugins/overlays/windows/windows.py", line
1144, in get_object
    return self.get_object_top_down(struct_name, object_type,
skip_type_check)
  File "volatility/volatility/plugins/overlays/windows/windows.py", line
1117, in get_object_top_down
    header.get_object_type() == object_type):
  File "volatility/volatility/plugins/overlays/windows/win7.py", line
155, in get_object_type
    return self.type_map.get(int(self.TypeIndex), '')
  File "volatility/volatility/plugins/overlays/windows/win10.py", line
279, in TypeIndex
    return ((addr >> 8) ^ cook ^ indx) & 0xFF
TypeError: unsupported operand type(s) for ^: 'int' and 'NoneType'

gleeda avatar Jun 21 '17 21:06 gleeda

What's the GUID for that ntoskrnl.exe?

iMHLv2 avatar Jun 21 '17 22:06 iMHLv2

3E1470698B694782ADAE95555171FC1F1

gleeda avatar Jun 22 '17 00:06 gleeda

So the warning Cannot find nt!ObGetObjectType indicates that we're able to find the NT module in memory, but can't parse its export table (thus we can't find the cookie value, thus we can't decode the object header)....all of which is required for the scanning plugins for that OS version. If you're able to moddump the NT module (with and without --memory, because I don't know which one would be more complete) we could try to find the function using symbols rather than the export table...and then you could get psscan working by specifying --cookie=<value that we find> at that point.

iMHLv2 avatar Jun 22 '17 01:06 iMHLv2

Actually, I can tell that the RVA for that GUID is 0x3a6460 so if you just add that to the base address of the NT module, then you should be able to supply that absolute address to --cookie. That's probably worth a shot before extracting things.

iMHLv2 avatar Jun 22 '17 01:06 iMHLv2

Sadly, the --cookie option doesn't work with the value you gave:

$ python vol.py -f Win10x64_14393.vmem --profile=Win10x64_14393 psscan --cookie=0xfffff801761c5460
Volatility Foundation Volatility Framework 2.6
Offset(P)          Name                PID   PPID PDB                Time created                   Time exited                   
------------------ ---------------- ------ ------ ------------------ ------------------------------ ------------------------------
0x000099065aae031e 56004F2}         51...8 42...3 0x92e0fffffff00000    

I get the same thing with just the RVA as well:

$ python vol.py -f Win10x64_14393.vmem --profile=Win10x64_14393 psscan --cookie=0x3a6460
Volatility Foundation Volatility Framework 2.6
Offset(P)          Name                PID   PPID PDB                Time created                   Time exited                   
------------------ ---------------- ------ ------ ------------------ ------------------------------ ------------------------------
0x000099065aae031e 56004F2}         51...8 42...3 0x92e0fffffff00000 

This is also after I updated the vtypes, since they were just slightly different.

gleeda avatar Jun 22 '17 16:06 gleeda

Just in case... It's a clean PE.

ntoskrnl.exe.zip

gleeda avatar Jun 22 '17 16:06 gleeda

Thanks...yeah, the RVA 0x3a6460 is correct for that file. Can you dd the address 0xfffff801761c5460 in volshell and paste the output?

iMHLv2 avatar Jun 22 '17 16:06 iMHLv2

In [1]: dd(0xfffff801761c5460)
fffff801761c5460  00000800 80000000 00000200 80000000
fffff801761c5470  00000800 00000000 00000200 00000000
fffff801761c5480  00000000 00000000 00000018 80000000
fffff801761c5490  00000018 00000000 00000018 00000000
fffff801761c54a0  00000818 80000000 00000218 80000000
fffff801761c54b0  00000818 00000000 00000218 00000000
fffff801761c54c0  00000000 00000000 00000000 80000000
fffff801761c54d0  00000000 00000000 00000000 00000000

gleeda avatar Jun 22 '17 17:06 gleeda

Definitely doesn't look like a cookie value at that address. Can you moddump --base=0xfffff80175e1f000 (with and without --memory) and post those two?

iMHLv2 avatar Jun 22 '17 17:06 iMHLv2

(this is both of them):

ntos_dumped.zip

gleeda avatar Jun 22 '17 17:06 gleeda

OK, i got it working with --cookie=0x4ddbae87... I'm not sure what was broken yet, but I'll document it here in a sec. I had to manually find the function and trace through the cookie code in the win10.py and got the this address, which works with psscan

gleeda avatar Jun 22 '17 18:06 gleeda

Ah nice. Well one thing that was broken is I told you the --cookie takes the address of nt!ObHeaderCookie when it actually takes the cookie value (dword stored at nt!ObHeaderCookie) - sorry. However, I'm not sure why we don't see the cookie value at the alleged base + RVA. The inability to find the exported function in the first place is likely due to paging of the PE header values (I can't parse the EAT in either of the modules you dumped).

iMHLv2 avatar Jun 22 '17 18:06 iMHLv2

OK, I'm not sure if I have this all correct, but, here's how I got the cookie:

First I'm looking at ntoskrnl.exe and I see what instructions we have for ObGetObjectType:

screen shot 2017-06-22 at 2 32 56 pm

Figured out where that is in memory, by using yarascan:

$ python vol.py -f Win10x64_14393.vmem --profile=Win10x64_14393 yarascan -K -Y "{48 8D 41 D0 0F B6 49 E8}"
Volatility Foundation Volatility Framework 2.6
Rule: r1
Owner: ntoskrnl.exe
0xf80176309d24  48 8d 41 d0 0f b6 49 e8 48 c1 e8 08 0f b6 c0 48   H.A...I.H......H
0xf80176309d34  33 c1 0f b6 0d 23 c7 eb ff 48 33 c1 48 8d 0d b9   3....#...H3.H...
0xf80176309d44  cb eb ff 48 8b 04 c1 c3 48 8b c4 48 89 58 08 48   ...H....H..H.X.H

Then investigate with volshell:

In [28]: addr = 0xf80176309d24

In [29]: dis(addr)
0xf80176309d24 488d41d0                         LEA RAX, [RCX-0x30]
0xf80176309d28 0fb649e8                         MOVZX ECX, BYTE [RCX-0x18]
0xf80176309d2c 48c1e808                         SHR RAX, 0x8
0xf80176309d30 0fb6c0                           MOVZX EAX, AL
0xf80176309d33 4833c1                           XOR RAX, RCX
0xf80176309d36 0fb60d23c7ebff                   MOVZX ECX, BYTE [RIP-0x1438dd]
0xf80176309d3d 4833c1                           XOR RAX, RCX
0xf80176309d40 488d0db9cbebff                   LEA RCX, [RIP-0x143447]
0xf80176309d47 488b04c1                         MOV RAX, [RCX+RAX*8]
0xf80176309d4b c3                               RET

[snip]

In [30]: for m in getmods():
    nt_mod = m
    break
   ....: 

In [31]: mode = distorm3.Decode64Bits

In [32]: data = nt_mod.obj_vm.read(addr, 100)

In [33]: ops = distorm3.Decompose(addr, data, mode, distorm3.DF_STOP_ON_RET)

In [34]: for op in reversed(ops):
    if (op.size == 7 and 'FLAG_RIP_RELATIVE' in op.flags and len(op.operands) == 2 and op.operands[0].type == 'Register' and
                                op.operands[1].type == 'AbsoluteMemory' and
                                op.operands[1].size == 8):
        addr2 = op.address + op.size + op.operands[1].disp
   ....:         

In [35]: nt_mod.obj_vm.is_valid_address(addr2)
Out[35]: True

In [36]: db(addr2)
0xf801761c6460  87 ae db 4d 00 00 00 00 de 01 00 00 dd 01 00 00   ...M............
0xf801761c6470  00 00 00 00 5a 62 02 00 39 24 9a 02 00 00 00 00   ....Zb..9$......
0xf801761c6480  00 00 00 00 00 00 00 00 38 43 53 00 00 00 00 00   ........8CS.....
[snip]

In [37]: cookie = obj.Object("unsigned int", offset = addr2, vm = nt_mod.obj_vm)

In [38]: hex(cookie)
Out[38]: '0x4ddbae87L'

gleeda avatar Jun 22 '17 18:06 gleeda

Ahhh ok... so that explains it :-)

gleeda avatar Jun 22 '17 18:06 gleeda

So, probably in the least, we should make sure to check the returned cookie before we operate on it, just to avoid the TypeError that's thrown here. I'm guessing we could put a warning message to say that the PE header values are paged or something along those lines.

gleeda avatar Jun 22 '17 18:06 gleeda

Sounds good...I'll de-prioritize a bit since you got it working, but will take care of the TypeError shortly.

iMHLv2 avatar Jun 22 '17 19:06 iMHLv2

This is still an issue. I keep coming across it with different Windows 10 machines... I think I might have another way to get around this

gleeda avatar Sep 28 '18 03:09 gleeda

Hey. Is there any guide how to get value cookie?

shpulka avatar Oct 29 '18 20:10 shpulka

Hello, are there any updates on this issue? I have some Windows 2016 Server images (profile is Win2016x64_14393) and I have the same problem. Is there a workaround to follow to allow pslist, psxview and the other to run?

slackeater avatar Jun 16 '20 14:06 slackeater

@gleeda from where you brought this data: "{48 8D 41 D0 0F B6 49 E8}"?

mydockergit avatar Aug 06 '20 04:08 mydockergit

@mydockergit Those are the first few bytes of the ObGetObjectType function. You can see the function in the screenshot (without the bytes the ASM instructions translate to). The hex values are the first two instructions of that function.

oold avatar Aug 06 '20 06:08 oold

@oold The first line with the address in the screenshot is 0x1404ead24.
I don't understand what you mean by "without the bytes the ASM instructions translate to". I don't see anything related to 48 8D 41 D0 0F B6 49 E8 in the screenshot.

mydockergit avatar Aug 07 '20 06:08 mydockergit

I have similar problem:

root@kali:~/Desktop# volatility -f desktop.raw --profile=Win10x64_15063 psscan
Volatility Foundation Volatility Framework 2.6
Offset(P)          Name                PID   PPID PDB                Time created                   Time exited                   
------------------ ---------------- ------ ------ ------------------ ------------------------------ ------------------------------
WARNING : volatility.debug    : Cannot find nt!ObGetObjectType

I saw #240 had similar issue so I tried to dump the ntoskrnl module using:

$ python vol.py -f [memdump] --profile [profile] moddump -r ntos -D [dumpdir]

But in my case it didn't do anything:

root@kali:~/Desktop# volatility -f desktop.raw --profile=Win10x64_15063 moddump -r ntos -D .
Volatility Foundation Volatility Framework 2.6
Module Base        Module Name          Result
------------------ -------------------- ------

mydockergit avatar Aug 07 '20 06:08 mydockergit

@mydockergit That's what I meant. The bytes corresponding to the instructions can't be seen in the screenshot, but what's fed into yarascan is exactly those bytes. If you look into the code of my pull request, you'll find this:

# 488d41d0 lea rax, qword ptr [rcx - 0x30]
# 0fb649e8 movzx ecx, byte ptr [rcx - 0x18]

If you want to fix your issue, you have three options:

  • Follow gleeda's instructions. Just take the byte signature as-is. It should work fine. No need to extract the signature yourself. And yes, doing it manually is painful and takes a lot of time.
  • Download the patch generated from my pull request from https://patch-diff.githubusercontent.com/raw/volatilityfoundation/volatility/pull/729.patch and use git apply to apply the patch to your local Volatility repo.
  • Wait until/if my PR gets merged.

oold avatar Aug 07 '20 09:08 oold

I download your branch (issue-436) and tried it but now I received a different error:

root@kali:~/Desktop# git clone --single-branch --branch=issue-436 https://github.com/oold/volatility.git
Cloning into 'volatility'...

remote: Enumerating objects: 48, done.
remote: Counting objects: 100% (48/48), done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 18616 (delta 26), reused 14 (delta 7), pack-reused 18568
Receiving objects: 100% (18616/18616), 14.87 MiB | 1.38 MiB/s, done.
Resolving deltas: 100% (14553/14553), done.
root@kali:~/Desktop# 
root@kali:~/Desktop# cd volatility/
root@kali:~/Desktop/volatility# python vol.py -f desktop.raw --profile=Win10x64_15063 psscan
Volatility Foundation Volatility Framework 2.6.1
ERROR   : volatility.debug    : The requested file doesn't exist

Not sure what is the issue

mydockergit avatar Aug 07 '20 11:08 mydockergit

Well, you changed your working directory to the checked out repo. The desktop.raw file you’re trying to scan probably isn’t in there.


From: mydockergit [email protected] Sent: Friday, August 7, 2020 1:15:32 PM To: volatilityfoundation/volatility [email protected] Cc: Oliver Old [email protected]; Mention [email protected] Subject: Re: [volatilityfoundation/volatility] Windows 10 x64 psscan error: ^: 'int' and 'NoneType' (#436)

I download your branch (issue-436) and tried it but now I received a different error:

root@kali:~/Desktop# git clone --single-branch --branch=issue-436 https://github.com/oold/volatility.git Cloning into 'volatility'...

remote: Enumerating objects: 48, done. remote: Counting objects: 100% (48/48), done. remote: Compressing objects: 100% (41/41), done. remote: Total 18616 (delta 26), reused 14 (delta 7), pack-reused 18568 Receiving objects: 100% (18616/18616), 14.87 MiB | 1.38 MiB/s, done. Resolving deltas: 100% (14553/14553), done. root@kali:~/Desktop# root@kali:~/Desktop# cd volatility/ root@kali:~/Desktop/volatility# python vol.py -f desktop.raw --profile=Win10x64_15063 psscan Volatility Foundation Volatility Framework 2.6.1 ERROR : volatility.debug : The requested file doesn't exist

Not sure what is the issue

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://eur05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fvolatilityfoundation%2Fvolatility%2Fissues%2F436%23issuecomment-670466594&data=02%7C01%7C%7C480bbecfdcf8489a6e6808d83ac33399%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637323957334809752&sdata=MbHmvICRgktfZ6g49gIth4OZAA5kiBkSLd17vhqihgo%3D&reserved=0, or unsubscribehttps://eur05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACZ2P3BCDZRGWDL2K6UYB6DR7PO5JANCNFSM4DQGHCCA&data=02%7C01%7C%7C480bbecfdcf8489a6e6808d83ac33399%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637323957334809752&sdata=2M5kHHO0s7dlfiLSDoO3PCKPbo4hupugv4JJSd3TELc%3D&reserved=0.

oold avatar Aug 07 '20 11:08 oold

Okay, now I am not getting the error but I don't get any information about processes on the system, something weird. I think it is related to #741.

root@kali:~/Desktop/volatility# python vol.py -f /root/Desktop/desktop.raw --profile=Win10x64_15063 psscan
Volatility Foundation Volatility Framework 2.6.1
Offset(P)          Name                PID   PPID PDB                Time created                   Time exited                   
------------------ ---------------- ------ ------ ------------------ ------------------------------ ------------------------------
root@kali:~/Desktop/volatility# python vol.py -f /root/Desktop/desktop.raw --profile=Win10x64_15063 pslist
Volatility Foundation Volatility Framework 2.6.1
Offset(V)          Name                    PID   PPID   Thds     Hnds   Sess  Wow64 Start                          Exit                          
------------------ -------------------- ------ ------ ------ -------- ------ ------ ------------------------------ ------------------------------
0xffffc88a0ba57540                      786732 39...9      0 -------- ------      0                                                              
root@kali:~/Desktop/volatility# python vol.py -f /root/Desktop/desktop.raw --profile=Win10x64_15063 hivelist
Volatility Foundation Volatility Framework 2.6.1
Virtual            Physical           Name
------------------ ------------------ ----
root@kali:~/Desktop/volatility# 

mydockergit avatar Aug 07 '20 14:08 mydockergit

Are you sure you're using the correct profile?

oold avatar Aug 08 '20 10:08 oold

@oold you can ignore it, I just understood it was corrupted memory file. Thanks for your help!

mydockergit avatar Aug 08 '20 13:08 mydockergit

@mydockergit That's what I meant. The bytes corresponding to the instructions can't be seen in the screenshot, but what's fed into yarascan is exactly those bytes. If you look into the code of my pull request, you'll find this:

# 488d41d0 lea rax, qword ptr [rcx - 0x30]
# 0fb649e8 movzx ecx, byte ptr [rcx - 0x18]

If you want to fix your issue, you have three options:

  • Follow gleeda's instructions. Just take the byte signature as-is. It should work fine. No need to extract the signature yourself. And yes, doing it manually is painful and takes a lot of time.
  • Download the patch generated from my pull request from https://patch-diff.githubusercontent.com/raw/volatilityfoundation/volatility/pull/729.patch and use git apply to apply the patch to your local Volatility repo.
  • Wait until/if my PR gets merged.

I know this is a pretty dated discussion - but can you elaborate on option #2 (downloading patch - command line syntax for example). I'm not having any luck applying this patch.

kbleich avatar Nov 22 '22 16:11 kbleich