volatility3 icon indicating copy to clipboard operation
volatility3 copied to clipboard
verified publisher icon volatilityfoundation

The windows.dumpfiles.DumpFiles plugin cannot dump all the files I want to dump.Some files can be dumped, some files cannot be dumped

Open or4gevi opened this issue 2 years ago • 4 comments

Describe the bug A clear and concise description of what the bug is. vCenter suspended the VM. Downloaded the VMEM file (16gb) and attempted to use Volatility3. The windows.dumpfiles plugin cannot dump all the files I want to dump.Some files can be dumped, some files cannot be dumped

Context Volatility Version: 3 Suspected Operating System: Windows Server 2016 10.0.14393

To Reproduce Steps to reproduce the behavior:

Command: $ python3 vol.py -vvv -f xx.vmem windows.info

Is64Bit	True
IsPAE	False
layer_name	0 WindowsIntel32e
memory_layer	1 VmwareLayer
base_layer	2 FileLayer
meta_layer	2 FileLayer
KdVersionBlock	0xf801526edcf8
Major/Minor	15.14393
MachineType	34404
KeNumberProcessors	1
SystemTime	2023-03-29 01:58:49
NtSystemRoot	C:\Windows
NtProductType	NtProductLanManNt
NtMajorVersion	10
NtMinorVersion	0INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input

PE MajorOperatingSystemVersion	10
PE MinorOperatingSystemVersion	0
PE Machine	34404
PE TimeDateStamp	Mon Oct  9 01:45:44 2017

$ python3 vol.py -vvv -f xx.vmem windows.filescan.FileScan |grep ntds.dit

INFO     volatility3.cli: Volatility plugins path: ['/home/volatility3-2.4.1/volatility3/plugins', '/home/volatility3-2.4.1/volatility3/framework/plugins']
INFO     volatility3.cli: Volatility symbols path: ['/home/volatility3-2.4.1/volatility3/symbols', '/home/volatility3-2.4.1/volatility3/framework/symbols']
INFO     volatility3.framework.automagic: Detected a windows category plugin
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.FileScan.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.FileScan.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.FileScan.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.FileScan
INFO     volatility3.framework.automagic: Running automagic: SymbolCacheMagic
INFO     volatility3.framework.automagic: Running automagic: LayerStacker
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
DEBUG    volatility3.framework.automagic.windows: Detecting Self-referential pointer for recent windows
DEBUG    volatility3.framework.automagic.windows: DtbSelfRef64bit test succeeded at 0x1aa000
DEBUG    volatility3.framework.automagic.windows: DTB was found at: 0x1aa000
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.FileScan.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name.memory_layer
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name.memory_layer.base_layer
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.FileScan.kernel.layer_name.memory_layer.meta_layer
Level 9  volatility3.framework.interfaces.configuration: TypeError - kernel_virtual_offset requirements only accept int type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - kernel_virtual_offset requirements only accept int type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - kernel_banner requirements only accept str type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - kernel_banner requirements only accept str type: None
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.FileScan.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.FileScan
DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['IntelLayer', 'VmwareLayer', 'FileLayer']
INFO     volatility3.framework.automagic: Running automagic: WinSwapLayers
INFO     volatility3.framework.automagic: Running automagic: KernelPDBScanner
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.FileScan.kernel.symbol_table_name
DEBUG    volatility3.framework.automagic.pdbscan: Kernel base determination - searching layer module list structure
DEBUG    volatility3.framework.automagic.pdbscan: Kernel base determination - searching layer module list structure
DEBUG    volatility3.framework.automagic.pdbscan: Setting kernel_virtual_offset to 0xf80152402000
DEBUG    volatility3.framework.symbols.windows.pdbutil: Using symbol library: ntkrnlmp.pdb/35B4FD549B8D4779BEEF22E3E2BF3984-1
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
INFO     volatility3.framework.automagic: Running automagic: SymbolFinder    
INFO     volatility3.framework.automagic: Running automagic: KernelModule
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_EPROCESS_QUOTA_BLOCK
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_PAGEFAULT_HISTORY
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_JOB_ACCESS_STATE
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_JOB_CPU_RATE_CONTROL
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_JOB_NET_RATE_CONTROL
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_JOB_NOTIFICATION_INFORMATION
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_PSP_STORAGE
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ACTIVATION_CONTEXT_DATA
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_FLS_CALLBACK_INFO
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ASSEMBLY_STORAGE_MAP
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_WNF_SCOPE_MAP
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ETW_SOFT_RESTART_CONTEXT
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ETW_STACK_CACHE
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ETW_PERFECT_HASH_FUNCTION
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_HAL_PMC_COUNTERS
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_SCSI_REQUEST_BLOCK
0xd10ac0648430	\Windows\NTDS\ntds.dit	216
...

python3 vol.py -vvv -f xx.vmem windows.dumpfiles.DumpFiles --virtaddr 0xd10ac0648430

Volatility 3 Framework 2.4.1
INFO     volatility3.cli: Volatility plugins path: ['/home/volatility3-2.4.1/volatility3/plugins', '/home/volatility3-2.4.1/volatility3/framework/plugins']
INFO     volatility3.cli: Volatility symbols path: ['/home/volatility3-2.4.1/volatility3/symbols', '/home/volatility3-2.4.1/volatility3/framework/symbols']
INFO     volatility3.framework.automagic: Detected a windows category plugin
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.DumpFiles.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.DumpFiles.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.DumpFiles.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.DumpFiles
Level 9  volatility3.framework.interfaces.configuration: TypeError - pid requirements only accept int type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - pid requirements only accept int type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - physaddr requirements only accept int type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - physaddr requirements only accept int type: None
INFO     volatility3.framework.automagic: Running automagic: SymbolCacheMagic
INFO     volatility3.framework.automagic: Running automagic: LayerStacker
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
DEBUG    volatility3.framework.automagic.windows: Detecting Self-referential pointer for recent windows
DEBUG    volatility3.framework.automagic.windows: DtbSelfRef64bit test succeeded at 0x1aa000
DEBUG    volatility3.framework.automagic.windows: DTB was found at: 0x1aa000
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.DumpFiles.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name.memory_layer
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name.memory_layer.base_layer
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.DumpFiles.kernel.layer_name.memory_layer.meta_layer
Level 9  volatility3.framework.interfaces.configuration: TypeError - kernel_virtual_offset requirements only accept int type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - kernel_virtual_offset requirements only accept int type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - kernel_banner requirements only accept str type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - kernel_banner requirements only accept str type: None
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.DumpFiles.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.DumpFiles
Level 9  volatility3.framework.interfaces.configuration: TypeError - pid requirements only accept int type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - pid requirements only accept int type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - physaddr requirements only accept int type: None
Level 9  volatility3.framework.interfaces.configuration: TypeError - physaddr requirements only accept int type: None
DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['IntelLayer', 'VmwareLayer', 'FileLayer']
INFO     volatility3.framework.automagic: Running automagic: WinSwapLayers
INFO     volatility3.framework.automagic: Running automagic: KernelPDBScanner
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.DumpFiles.kernel.symbol_table_name
DEBUG    volatility3.framework.automagic.pdbscan: Kernel base determination - searching layer module list structure
DEBUG    volatility3.framework.automagic.pdbscan: Kernel base determination - searching layer module list structure
DEBUG    volatility3.framework.automagic.pdbscan: Setting kernel_virtual_offset to 0xf80152402000
DEBUG    volatility3.framework.symbols.windows.pdbutil: Using symbol library: ntkrnlmp.pdb/35B4FD549B8D4779BEEF22E3E2BF3984-1
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
INFO     volatility3.framework.automagic: Running automagic: SymbolFinder    
INFO     volatility3.framework.automagic: Running automagic: KernelModule

Cache	FileObject	FileName	Result
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_EPROCESS_QUOTA_BLOCK
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_PAGEFAULT_HISTORY
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_JOB_ACCESS_STATE
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_JOB_CPU_RATE_CONTROL
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_JOB_NET_RATE_CONTROL
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_JOB_NOTIFICATION_INFORMATION
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_PSP_STORAGE
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ACTIVATION_CONTEXT_DATA
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_FLS_CALLBACK_INFO
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ASSEMBLY_STORAGE_MAP
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_SCSI_REQUEST_BLOCK
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_WNF_SCOPE_MAP
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ETW_SOFT_RESTART_CONTEXT
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ETW_STACK_CACHE
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_ETW_PERFECT_HASH_FUNCTION
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!_HAL_PMC_COUNTERS

or4gevi avatar Apr 19 '23 01:04 or4gevi

Hi there, Volatility can only dump the files it can identify from the memory image provided. There's no guarantee that all files you requested will be present in memory, or that the structures needed to find them will be present. If there's a specific file that you expected dumpfiles to be able to extract and can show that it's accessible through the technique that dumpfiles uses to locate the files it extracts, then we can investigate that bug, but the output you provided makes no mention of files that couldn't be recovered (nor any that could), so it's unclear that volatility isn't operating as expected. If you could provide more information as to why you believe volatility should have been able to determine the contents of a specific file from memory then we can investigate further... 5:)

ikelos avatar Apr 25 '23 20:04 ikelos

This issue is stale because it has been open for 200 days with no activity.

github-actions[bot] avatar Nov 12 '23 01:11 github-actions[bot]

Hi there, Volatility can only dump the files it can identify from the memory image provided. There's no guarantee that all files you requested will be present in memory, or that the structures needed to find them will be present. If there's a specific file that you expected dumpfiles to be able to extract and can show that it's accessible through the technique that dumpfiles uses to locate the files it extracts, then we can investigate that bug, but the output you provided makes no mention of files that couldn't be recovered (nor any that could), so it's unclear that volatility isn't operating as expected. If you could provide more information as to why you believe volatility should have been able to determine the contents of a specific file from memory then we can investigate further... 5:)

I've encountered the same issue. So, how can I proceed to export this file? After such a long time, is the domain database file still unrecognized? Do I need to send you an example of this file? Thank you.

stream1990 avatar Dec 28 '23 12:12 stream1990

It's not about the type of file actually recorded there, it's about identifying the structure recording the information about the file. It turns out the technique volatility uses to find these files is different between the two plugins. The filescan plugin uses the poolscanner to hunt for entries, the dumpfiles looks for file handles to dump the contents of. As such, entries found in filescan may not be files the can be retrieved by dumpfiles. @iMHLv2 may be able to explain the difference a little better, and let us know if it would be possible to add a --dump option to the filescan plugin, but there's no guarantee the contents of the file are present in memory just because the entry is found in a pool.

ikelos avatar Jan 01 '24 20:01 ikelos