volatilityfoundation
hardcoded structures in module.c not up to date
Hi,
Some hard coded kernel structures, such as mnt_namespace have changed (7 times) since kernel 3.3 (like 4.9 here )but module.c was not updated.
I would like to make a PR to fix that. I see two ways:
- pasting the whole mnt_namespace structure every time there was a change (I find it clearer, since some fields came in and out from the structure),
- or declaring one structure with a lot of #if conditions mixed inside, as it's already done for the other structures
What do you advise?
I have the same problem with module.c. Some structures result as outdated.
Hello all!
Last semester, I, Davide, Fabio, and Andrea had developed some tools to automatically extract and generate the kernel data structure required by Volatility as listed in the original module.c. The tool itself is not yet ready to be published as there's some minor improvement required. However, the result is available for you to test. Please check this out:
https://github.com/gilanghamidy/Volatility-AutoModule-Result
You can try to build the dwarf file exactly like usual. Please try it out and let us know if there's an issue. I have randomly tried for several different kernel version and it seems to work quite fine. But definitely we need more hands to actually test it. :)
Hey @gleeda and @atcuno: if you feel like this is something you want for volatility 2/3 we can definitely try to find some spare cycles to get it done!
For Volatility 3, we are looking into supporting more flexible and varying options for ways for people to get the data structures included. We will likely have a module.c equivalent, but it will operate in new ways.
For Volatility 2, we have kept up with the data structures changes reasonably well - I have some tooling that alerts when structures change, but its definitely not ideal. For example, this reported mnt_namespace structure has changed several times, but it is been to add members that Volatility's core plugins don't use. This obviously keeps the core code stable, but could be an issue if other researchers wanted to use the new members, and we haven't gotten them added to module.c in a stable way.
Second, when attempting to cover the 10k+ possible kernels people might run against, backporting of new Linux kernel features to old kernels by distros, becomes a major, major issue. As a recent example, redhat/centos just backported the use of xa_node structures to 4.18 in their current mainline kernels. In the Linux source tree, this structure does not appear until 4.20. This breaks module.c for all profiles built against previous rhel/centos 4.18 kernels as module.c only included xa_node for >= 4.20 kernels. I expect to push a fix for that this week.
Looking at https://github.com/gilanghamidy/Volatility-AutoModule-Result , I can see that it only includes xa_node for >= 4.20 also, which would require updates to their module for 4.18 and 4.19. Since they haven't released their tool yet, it may just be that their code relies on module.c, which would be the same root cause, or if they work from stock linux.org kernels, they will always miss backports too. Having a more robust/automated way to detect backports and then regression test inclusion of them in memory forensics frameworks would be a big advancement in this area of research.
So unfortunately, I don't really have a great answer. yet for Vol2. Its currently based on our fairly extensive testbed to detect when structures change to the point that they would break analysis plugins. Given how often structures change, it would be a full time job of its own to keep module.c up to date for members that Volatility hasn't relied on, so instead we just keep it stable for the ones the core plugins currently need.
Thank you for your response. Exactly as you said, the module.c in my repository was generated from the Linux stock kernel. However, the automated tool we developed can also work with different kernel sources, including the backported one. We only need to supply the desired kernel source folder and let the tool analyze and generate a suitable module.c.
One concern we have is that currently the tool optimistically searches at a specified source file. For example struct proc_dir_entry is located at fs/proc/internal.h file. The tool is already informed of this location beforehand. If some backport or kernel fork creatively move that struct into other location, the tool will have to crawl every possible kernel sources, which can be thousands. However, the tool itself is designed to work this way. Before we implement the optimistic search, we experimented using this mechanism. it will just be slower than using the optimistic approach.
Also, the showstopper that I can think of is that if the kernel backport or fork modifies the kernel build script extensively so it breaks our tool. But I am not too sure about this as I am not experienced with building kernel from other distros.
Our current plan is to release the tool so that people can generate the module.c on their own. I hope I can find some time soon to revamp the tool and publish it, as currently I am also occupied with my thesis work. :)
That sounds great @gilanghamidy ! We are actively experimenting with different approaches to Linux profile support in Volatility 3, so your tool could definitely be useful in our experiments. You should also consider sending your work to our Volatility Plugin Contest this year. The deadline isn't until October, so there is plenty of time for improvements before needing to turn anything in:
https://volatility-labs.blogspot.com/2020/05/the-8th-annual-volatility-plugin-contest.html