volatility3 icon indicating copy to clipboard operation
volatility3 copied to clipboard

Unsatisfied requirement for analysis of Linux memory

Open infinitebugs32 opened this issue 2 years ago • 9 comments

Hi there,

I have a dump of a Virtualbox I try to analyze with volatility3 but There is an error :

aaaa@aaaa-HP-Z440-Workstation:~/Desktop/volatility3$ python3 ./vol.py -f /home/aaaa/testlinux.raw linux.pslist
Volatility 3 Framework 2.4.2
Progress:  100.00		Stacking attempts finished                 
Unsatisfied requirement plugins.PsList.kernel.layer_name: 
Unsatisfied requirement plugins.PsList.kernel.symbol_table_name: 

A translation layer requirement was not fulfilled.  Please verify that:
	A file was provided to create this layer (by -f, --single-location or by config)
	The file exists and is readable
	The file is a valid memory image and was acquired cleanly

A symbol table requirement was not fulfilled.  Please verify that:
	The associated translation layer requirement was fulfilled
	You have the correct symbol file for the requirement
	The symbol file is under the correct directory or zip file
	The symbol file is named appropriately or contains the correct banner

Unable to validate the plugin requirements: ['plugins.PsList.kernel.layer_name', 'plugins.PsList.kernel.symbol_table_name']

Also, is there any way to download at once every symbols available on this website ?

Thanks

infinitebugs32 avatar Jun 14 '23 07:06 infinitebugs32

-vvvv:

Volatility 3 Framework 2.4.2
INFO     volatility3.cli: Volatility plugins path: ['/home/aaaa/Desktop/volatility3/volatility3/plugins', '/home/aaaa/Desktop/volatility3/volatility3/framework/plugins']
INFO     volatility3.cli: Volatility symbols path: ['/home/aaaa/Desktop/volatility3/volatility3/symbols', '/home/aaaa/Desktop/volatility3/volatility3/framework/symbols']
DEBUG    volatility3.framework: Traceback (most recent call last):
  File "/home/aaaa/Desktop/volatility3/volatility3/framework/__init__.py", line 185, in import_file
    importlib.import_module(module)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/aaaa/Desktop/volatility3/volatility3/plugins/prefetch.py", line 6, in <module>
    import logging, pathlib, datetime, io, numpy
ModuleNotFoundError: No module named 'numpy'

DEBUG    volatility3.framework: Failed to import module volatility3.plugins.prefetch based on file: /home/aaaa/Desktop/volatility3/volatility3/plugins/prefetch.py
DEBUG    volatility3.framework: Traceback (most recent call last):
  File "/home/aaaa/Desktop/volatility3/volatility3/framework/__init__.py", line 185, in import_file
    importlib.import_module(module)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/aaaa/Desktop/volatility3/volatility3/plugins/windows/chromehistory.py", line 35, in <module>
    import volatility.plugins.common as common
ModuleNotFoundError: No module named 'volatility'

DEBUG    volatility3.framework: Failed to import module volatility3.plugins.windows.chromehistory based on file: /home/aaaa/Desktop/volatility3/volatility3/plugins/windows/chromehistory.py
DEBUG    volatility3.framework: Traceback (most recent call last):
  File "/home/aaaa/Desktop/volatility3/volatility3/framework/__init__.py", line 185, in import_file
    importlib.import_module(module)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/aaaa/Desktop/volatility3/volatility3/plugins/windows/chromehistory.py", line 35, in <module>
    import volatility.plugins.common as common
ModuleNotFoundError: No module named 'volatility'

DEBUG    volatility3.framework: Failed to import module volatility3.plugins.windows.chromehistory.chromehistory based on file: /home/aaaa/Desktop/volatility3/volatility3/plugins/windows/chromehistory/chromehistory.py
DEBUG    volatility3.framework: Traceback (most recent call last):
  File "/home/aaaa/Desktop/volatility3/volatility3/framework/__init__.py", line 185, in import_file
    importlib.import_module(module)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/aaaa/Desktop/volatility3/volatility3/framework/plugins/linux/envars.py", line 1, in <module>
    from volatility3.plugins import envvars
ImportError: cannot import name 'envvars' from 'volatility3.plugins' (/home/aaaa/Desktop/volatility3/volatility3/plugins/__init__.py)

DEBUG    volatility3.framework: Failed to import module volatility3.plugins.linux.envars based on file: /home/aaaa/Desktop/volatility3/volatility3/framework/plugins/linux/envars.py
INFO     volatility3.cli: The following plugins could not be loaded (use -vv to see why): volatility3.plugins.linux.envars, volatility3.plugins.prefetch, volatility3.plugins.windows.chromehistory, volatility3.plugins.windows.chromehistory.chromehistory
INFO     volatility3.framework.automagic: Detected a linux category plugin
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList
INFO     volatility3.framework.automagic: Running automagic: SymbolCacheMagic
INFO     volatility3.framework.automagic: Running automagic: LayerStacker
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using AVMLStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LimeStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using Elf64Stacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using QemuStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using WindowsCrashDumpStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using VmwareStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LinuxIntelStacker
DEBUG    volatility3.framework.automagic.linux: No suitable linux banner could be matched
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: TypeError - Layer is not the required Architecture: FileLayer
DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['FileLayer']
INFO     volatility3.framework.automagic: Running automagic: SymbolFinder  
INFO     volatility3.framework.automagic: Running automagic: LinuxSymbolFinder
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name
INFO     volatility3.framework.automagic: Running automagic: KernelModule
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name

Unsatisfied requirement plugins.PsList.kernel.layer_name: 
Unsatisfied requirement plugins.PsList.kernel.symbol_table_name: 

A translation layer requirement was not fulfilled.  Please verify that:
	A file was provided to create this layer (by -f, --single-location or by config)
	The file exists and is readable
	The file is a valid memory image and was acquired cleanly

A symbol table requirement was not fulfilled.  Please verify that:
	The associated translation layer requirement was fulfilled
	You have the correct symbol file for the requirement
	The symbol file is under the correct directory or zip file
	The symbol file is named appropriately or contains the correct banner

Unable to validate the plugin requirements: ['plugins.PsList.kernel.layer_name', 'plugins.PsList.kernel.symbol_table_name']

infinitebugs32 avatar Jun 14 '23 07:06 infinitebugs32

Hello,

I don't think there is a way to download all the ISF files from https://isf-server.techanarchy.net/ at once. However really most of them will be useless to you. You require the very specific version that matches the linux kernel found in your mem sample exactly. It has to be be a perfect match.

If you read the linux tutorial it will cover creating symbols for your sample, and then you should be able to get to work.

The short version is:

Run the banners plugin to find the linux kernel banner you're looking for:

python3 ./vol.py -f /home/aaaa/testlinux.raw banners

Then you need to use dwarf2json to create the ISF file from a matching debug package (or download it from https://isf-server.techanarchy.net). That's the symbols you need nicely together in a json file.

Then run the isfinfo plugin to ensure it matches the output from banners.

python3 ./vol.py -f /home/aaaa/testlinux.raw isfinfo

It that then matches exactly (and I mean exactly, 100% the same) then pslist should work.

Good luck, and if you need help do ask here for join the slack channel: https://www.volatilityfoundation.org/slack

If you do have success could you please update us here and close the issue.

I am interested in why envars/envvars did not import, but it looks like you have a few community plugins so there may be something happening there, we can revisit that if the envvars plugin isn't working for you correctly when you have the ISF you need.

eve-mem avatar Jun 14 '23 07:06 eve-mem

Hi @eve-mem thank you for your answer. The reason why I actually need to download at once all the ISF files from that website is that I try to write a little tool that automates the analysis of a memory file. The automation works well on Windows files because I downloaded all the symbols provided by the volatility foundation, however for Linux there are very few symbols, that's why I wanted all the ISF files. Maybe I'm dreaming ? I want to create a program that only takes the path of the memory file as user input, and automatically runs all the available plugins and write the result to csv.

I don't understand what the banners command is, what it does, what is its purpose ? Anyway here is the output:

aaaa@aaaa-HP-Z440-Workstation:~/Desktop/volatility3$ python3 ./vol.py -vvvv -f /home/aaaa/testlinux.raw banners
Volatility 3 Framework 2.4.2
INFO     volatility3.cli: Volatility plugins path: ['/home/aaaa/Desktop/volatility3/volatility3/plugins', '/home/aaaa/Desktop/volatility3/volatility3/framework/plugins']
INFO     volatility3.cli: Volatility symbols path: ['/home/aaaa/Desktop/volatility3/volatility3/symbols', '/home/aaaa/Desktop/volatility3/volatility3/framework/symbols']
DEBUG    volatility3.framework: Traceback (most recent call last):
  File "/home/aaaa/Desktop/volatility3/volatility3/framework/__init__.py", line 185, in import_file
    importlib.import_module(module)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/aaaa/Desktop/volatility3/volatility3/plugins/prefetch.py", line 6, in <module>
    import logging, pathlib, datetime, io, numpy
ModuleNotFoundError: No module named 'numpy'

DEBUG    volatility3.framework: Failed to import module volatility3.plugins.prefetch based on file: /home/aaaa/Desktop/volatility3/volatility3/plugins/prefetch.py
DEBUG    volatility3.framework: Traceback (most recent call last):
  File "/home/aaaa/Desktop/volatility3/volatility3/framework/__init__.py", line 185, in import_file
    importlib.import_module(module)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/aaaa/Desktop/volatility3/volatility3/plugins/windows/chromehistory.py", line 35, in <module>
    import volatility.plugins.common as common
ModuleNotFoundError: No module named 'volatility'

DEBUG    volatility3.framework: Failed to import module volatility3.plugins.windows.chromehistory based on file: /home/aaaa/Desktop/volatility3/volatility3/plugins/windows/chromehistory.py
DEBUG    volatility3.framework: Traceback (most recent call last):
  File "/home/aaaa/Desktop/volatility3/volatility3/framework/__init__.py", line 185, in import_file
    importlib.import_module(module)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/aaaa/Desktop/volatility3/volatility3/plugins/windows/chromehistory.py", line 35, in <module>
    import volatility.plugins.common as common
ModuleNotFoundError: No module named 'volatility'

DEBUG    volatility3.framework: Failed to import module volatility3.plugins.windows.chromehistory.chromehistory based on file: /home/aaaa/Desktop/volatility3/volatility3/plugins/windows/chromehistory/chromehistory.py
DEBUG    volatility3.framework: Traceback (most recent call last):
  File "/home/aaaa/Desktop/volatility3/volatility3/framework/__init__.py", line 185, in import_file
    importlib.import_module(module)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/aaaa/Desktop/volatility3/volatility3/framework/plugins/linux/envars.py", line 1, in <module>
    from volatility3.plugins import envvars
ImportError: cannot import name 'envvars' from 'volatility3.plugins' (/home/aaaa/Desktop/volatility3/volatility3/plugins/__init__.py)

DEBUG    volatility3.framework: Failed to import module volatility3.plugins.linux.envars based on file: /home/aaaa/Desktop/volatility3/volatility3/framework/plugins/linux/envars.py
INFO     volatility3.cli: The following plugins could not be loaded (use -vv to see why): volatility3.plugins.linux.envars, volatility3.plugins.prefetch, volatility3.plugins.windows.chromehistory, volatility3.plugins.windows.chromehistory.chromehistory
INFO     volatility3.framework.automagic: No plugin category detected
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Banners.primary
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Banners.primary
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Banners.primary
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Banners.primary
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Banners
INFO     volatility3.framework.automagic: Running automagic: SymbolCacheMagic
INFO     volatility3.framework.automagic: Running automagic: LayerStacker
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Banners.primary
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using AVMLStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LimeStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using Elf64Stacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using QemuStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using WindowsCrashDumpStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using VmwareStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using MacIntelStacker
INFO     volatility3.framework.automagic.mac: No Mac banners found - if this is a mac plugin, please check your symbol files location
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LinuxIntelStacker
DEBUG    volatility3.framework.automagic.linux: No suitable linux banner could be matched
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using WindowsIntelStacker
DEBUG    volatility3.framework.automagic.windows: Detecting Self-referential pointer for recent windows
DEBUG    volatility3.framework.interfaces.layers: Scan Failure: Sections have no size, nothing to scan
DEBUG    volatility3.framework.automagic.windows: Older windows fixed location self-referential pointers
DEBUG    volatility3.framework.interfaces.layers: Invalid address in layer FileLayer found scanning FileLayer at address 30000
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Banners.primary
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Banners.primary
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Banners.primary
DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['FileLayer']
INFO     volatility3.framework.automagic: Running automagic: WinSwapLayers 
INFO     volatility3.framework.automagic: Running automagic: KernelPDBScanner
INFO     volatility3.framework.automagic: Running automagic: SymbolFinder  
INFO     volatility3.framework.automagic: Running automagic: MacSymbolFinder
INFO     volatility3.framework.automagic: Running automagic: LinuxSymbolFinder
INFO     volatility3.framework.automagic: Running automagic: KernelModule

Offset	Banner


infinitebugs32 avatar Jun 14 '23 08:06 infinitebugs32

Re this sample in particular: The banners plugin should show you the version of Linux you're looknig for. Here is an example, you can see its a 3.2 linux kernel.

python vol.py -f linux-sample-1.dmp banners
Volatility 3 Framework 2.4.2
Progress:  100.00               PDB scanning finished
Offset  Banner

0x1400070       Linux version 3.2.0-4-amd64 ([email protected]) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.57-3+deb7u2

So for your sample that's not worked, and vol3 relies on this method to detect the banner. Right now even with the correct ISF vol3 wouldn't be able to work out what one to apply. How exactly did you acquire this memory sample? The raw extension makes me believe its a raw memory sample of main memory, if you open it with a hex editor can you find a "Linux version" string at all.

Re all linux symbols for everything: I think it will be hard to cover all of the different versions of linux to make a tool that works all the time. There isn't a central place for this. In the worst case someone can quite easily compile there own version of the kernel meaning there are no debugging symbols to be found at all.

You could take on a project to get every version of the major distributions and try to make that available - but this is not a small undertaking and would need to be updated regularly. There are new minor versions of the kernel released every few months. (https://en.wikipedia.org/wiki/Linux_kernel_version_history). It would be like trying to run a bigger version of https://isf-server.techanarchy.net/ yourself.

If you can commit to doing that it would be a great resource for the community.

eve-mem avatar Jun 14 '23 08:06 eve-mem

@eve-mem I acquired the memory sample by dumping the core memory of a Virtual Box machine by following this tutorial, it is possible that I made a mistake during the process maybe. I can't find Linux memory sample files on the Internet, for free to download.

Also, instead of trying to download all the files, do you think it would be possible to generate them automatically, once the user uploads his memory file ? If yes, what would be the process to do so ? The idea is that I have a graphical tool, the user puts the path of his memory file, and the program does everything.

infinitebugs32 avatar Jun 14 '23 08:06 infinitebugs32

I think you should be able to use the file directly from the virtual box export without having to extract out the raw parts manually. Try that and see if it works.

In terms of creating the linux profiles on demand, I think it will be similarly challenging. You'd need to know how to download all different symbols from all the different distributions. If you could work out all of that logic you may as well create them all in advance. Not to day this is impossible, it's been on my ideas list for a long time to try and collect all of the symbols for the popular distros. It will never be 100%, but even if it's 50% coverage it'll help a lot of people.

eve-mem avatar Jun 17 '23 11:06 eve-mem

To add to the conversation about linux symbol tables, as Eve pointed out it's possible to compile your own kernel and in doing so you might change certain structures created by the kernel, meaning when you try to read them back you might be looking for things in the wrong place and the results wouldn't make much sense or in a worse case would try to reference memory that didn't exist and cause volatility to fail.

Volatility 2 used to use a "probably close enough" method, both with windows and other OSes, whereby the symbols we used (a very small number) were "probably close enough" and if it went wrong then we could kinda only shrug and say "might've been off". This didn't feel very good for a forensics tool, particularly when windows and mac have a much smaller (although still large) fixed number of kernels, whose symbols they distribute. This allows us to make the JSON files for those operating systems (and in the case of windows, we can do it on the fly by downloading the necessary information). However, for Linux, since every kernel could be custom compile and as a forensics tool we want to be accurate, we mandate a matching kernel string (which even that is not strictly a guarantee, but does make it far more likely to be accurate absent a malicious actor trying to trip the tool up).

As mentioned it may be possible to build up tables for the various kernels, and that's what one of our community attempted to do with the ISF server. We also have some rough development code for taking URLs of kernels from popular distributions, downloading them and converting them to ISF files. It isn't possible to determine the symbols just from a memory image, the data just isn't in there, although with only a copy of the original running kernel, it is possible to load a kernel that can then extract that information dynamically. Like Volatility 2, it only gets what it considers "the most commonly needed structures" and therefore will work for the majority of cases, but isn't particularly future-proof and is therefore why we don't advertise/suggest its use. You can find out more about it on the dwarf2json project branch for the feature.

If you'd like to get involved with the increasing the library of JSON files for common linux distributions, we'd recommend reaching out to @kevthehermit who started the ISF server project.

ikelos avatar Jun 17 '23 12:06 ikelos

Hello @infinitebugs32 - did you manage to get your virtual box memory dump working?

eve-mem avatar Jun 29 '23 07:06 eve-mem

This issue is stale because it has been open for 200 days with no activity.

github-actions[bot] avatar Jan 26 '24 01:01 github-actions[bot]

This issue was closed because it has been inactive for 60 days since being marked as stale.

github-actions[bot] avatar Mar 27 '24 01:03 github-actions[bot]