linux-unofficial_grsec

linux-unofficial_grsec copied to clipboard

Spent a bit of time looking at the KPTI thing and trying to figure how the pieces impacted by the changes in this tree actually work, and what the kpti patches are doing there. It looks like there's a bunch of delta around the way tss_struct is handled, and lower down to things i dont yet understand, seemingly related to how data is laid out on the stack. It appears that some of the memory access is actually occuring from kernel context whereas the 4.9 upstream code being patched seems to expect its in the user context (in truct tss_struct doublefault_tss for instance)

@minipli: if this is actually "portable" into the fork, is there any chance you might be able to detail the changes from upstream in the commit log so we could better understand the clockwork? Thanks as always.

The KPTI Patches are coming with v4.9.75. Resolving the conflicts and making them compatible with PaX will take a lot of time, so please be patient everybody.

Thank you much. We have manually tested the meltdown poc on a bunch of Intel and a couple of opteron systems, and the latter does appear safe from the vector, while every bloody Intel chip we can test on works like a charm...

4.9.75 with KPTI is out now: https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.9.75

@minipli: if the KPTI port is too absurd to carry out, is there an option to implement a memory-localization technique for kernel/uspace like the paper suggested:

A more realistic solution would be to introduce a hard
split of user space and kernel space. This could be enabled
optionally by modern kernels using a new hardsplit
bit in a CPU control register, e.g., CR4. If the hardsplit
bit is set, the kernel has to reside in the upper half
of the address space, and the user space has to reside in
the lower half of the address space. With this hard split,
a memory fetch can immediately identify whether such a
fetch of the destination would violate a security boundary,
as the privilege level can be directly derived from
the virtual address without any further lookups. We expect
the performance impacts of such a solution to be
minimal. Furthermore, the backwards compatibility is
ensured, since the hard-split bit is not set by default and
the kernel only sets it if it supports the hard-split feature.

I figure this would drastically reduce ASLR space, or cost obnoxious amounts of wasted memory, but it might be a safer approach long term.

@sempervictus The part you're quoting is talking about a yet-to-be-implemented hardware feature. So no, that's no option. The KPTI patches are needed to prevent the meltdown attack.

Ah, thanks for the clarification. I had read that to mean that the address location was used in speculative context to determine if reads would be allowed. Didn't realize hardware would have to change for a </> comparison of the target address and some boundary address in speculative execution as a way to fail early. I'll start collecting data for benchmarks this weekend to see the resulting cost of kpti and retpolines once we upgrade and rerun.

Just found this project, great work! Do we know anything more about when patches might be available? "Resolving the conflicts and making them compatible with PaX will take a lot of time", is this days/weeks/months?

It will take the time it will take.

Thank you for your work minipli amd take your time

Expect it to be weeks/months/never. It's a pretty invasive change conflicting with a lot of PaX. :(

@minipli: What about other mechanisms to achieve the same end? IIUC, KPTI prevents speculative access to "pivot" over kernel pages mapped into the userspace memory range to the phys page tables and access memory available to the kernel in that context (prior to the privilege/exec bits being visible) by moving most of the kernel page maps out of the userspace virtual page tables. Could we use the active exploit mitigation mechanisms or UDEREF to catch the pattern of page faults generated when speculative execution gets verified with the privilege checks and discarded? If we lock out the user doing this once they start, we dont eliminate the problem (but then again, neither does KPTI really), but we prevent repetition of the condition preventing the attacker from reading ever more memory. I dont understand the internal mechanics of UDEREF very well, code was probably not written to be easily groked by the novice kernel abuser (or the comments were thinned out a bit prior to publication), so please pardon the ignorance, but at least semantically my understanding is that its supposed to address the invalid access problem (although i assume it has the same problem with needing actual page content to make the determination of whether or not the access is legal). If it does so retrospectively relative to speculative execution out of context, like the normal privilege checks work, then it should at least be useful as a post-access mechanism to record the invalid access. A refcount on these which recedes over time to address legal page faults and accumulates from these rapid hits might be a way to guess whether or not someone is trying to meltdown or rowhammer the system. If it locks the user out, they need to find another inbound access method, if it forces them to wait too long between attempts, they're likely to get inconsistent contents from memory (if they try to beat the counter/timer). I'm also a bit concerned that KPTI, being incomplete due to the memory model, still leaves some structures mapped into the userspace application's memory. If any of those can be pivoted back into the physical tables, wouldn't that render the mitigation useless? It seems like having a retroactive detection+kill mechanism provides more uniform coverage to the problem domain (without leaving esoteric holes requiring a bunch of research and reverse engineering to exploit if possible, but that most of the public is unaware of), without incurring the insane performance penalties KPTI brings, or mauling PAX code.

@minipli: you can use in first time the function GRKERNSEC_KERN_LOCKOUT for check the logs with "trap invalid opcode ip:".

Else use KPTI only on Intel processors that are impacted (disable PaX on a bugs files and waiting for a better solution). Vulnerable CPUs list : #19 Non-vulnerable CPU list : #22

I'm trying to understand "linux-4.9.74/Documentation/x86/exception-tables.txt" to find a simple hack at the expense of instruction and or specific features.

Thank you ;)

Oh god.. https://news.ycombinator.com/item?id=16087736

Oh minipli's comment is sad news :-(. Apart from that the upstream backport of KAISER seems to be flawed, see https://news.ycombinator.com/item?id=16087736 and https://twitter.com/grsecurity/status/950868975789006848

EDIT: zaolin nice coincidence that we posted the same info at the same time both beginning with the word "Oh"

minipli will you make changes to the code of your kernel for kvm ? https://patchwork.kernel.org/patch/10151859/ Thank you.

I managed to block the MELTDOWN PoC by modifying the CPU cache behavior (Using /proc/mtrr or with module to disable the CPU cache), but the performance is severely degraded :/

Spenders/pax' implementation according to twitter seems to incorporate the functionality into UDEREF (where we see a ton of conflicts)... So there's hope, especially since older kernels apparently got kaiser instead. Uderef already uses pcid, hopefully permitting use of the kpti version for the relevant functionality. Given the immense task we are all asking minipli to take on, anyone think that the effort might merit a kick starter, any reason not to start one? Hell if spender would agree to release an updated version for X dollars via crowd funding that'd be great too, but I've no idea what their thinking is regarding any public release of their Linux fork.

On a separate note, anyone consider PAGEXEC as a means to preempt the speculative bounds violations in branch predictors (spectre)?

Maybe it's time to migrate away from grsecurity? These unofficial patches will not be maintained forever even if minipli decides to patch meltdown, and 4.9 kernel will live for maybe one more year. I'm putting my hope in http://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project and SElinux :) It was a good run but maybe it's time to face the facts, grsec does not want anything to do with the community anymore

@indy-independence: KSPP is still far from offering the same protection, the proof: get-rekt-linux-hardened

Not to mention their REFCOUNT implementation : REFCOUNT on linux 4.14

@sempervictus: the crowd funding maybe a solution for help minipli.

KSPP is not targeting full porting anyway, UDEREF, KERNEXEC, and other critical things which are likely to called out as "breaking an insecure userspace for the sake of security," or "too much of a performance hit for consideration" like the PIE code with single digit hits. This situation points to the need for a longer term solution for Linux security. The kernel is owned by a single person for all intents (unless chip makers force them to adopt "fixes" for the egregious mistakes they made following the same design philosophy as that person), KSPP has to go through him, and needs to internally allocate resources to fund the work and make time for it (and given the companies involved, and how they use Linux, it should be 7 figures and a few dedicated staff). Even if spender got the Linux crown, if he and pipacs get hit by a bus, the original talent is gone - it doesn't scale well. We also can't expect or in good conscience want the talent capable of executing this work to give it away for free (we can however ask them to give it away for money, which would also aid corporate adoption since money == real in that context). To do what they did, grsec/pax have maintained a defacto fork of Linux for almost two decades and distribute it in patch form to work around the GPL redistribution requirements (unless proven otherwise). Maybe its time to try and make an official Linux kernel fork which doesn't go through the "normal process" involving the set of subjective concerns which have prioritized unsafe performance and maintenance of unsafe userspace semantics over a multidimensional defense model; but stays fully public avoiding the need for GPL workarounds and related concerns. Ideally this would create a way for grsec and others to make a good living with work which will long outlive them and be maintainable by subsequent generations. To start, we need to get a sense of whom might be willing, capable, and have the time to do such work for hire, pull in strcats todo list and add things which he doesn't need in CHOS but generic systems do require. We would then need to get estimates for work and time, set up a crowd funding system with escrow or some guarantee of payment for them, and find a cat herder (project manager) familiar enough with the subject domain to keep the process running. No small task, but I'm not sure what other avenues exist to address this critical need - at least in the US, realization of our constitutional rights to freedom of speech and privacy requires that we have security. When the ability for individual citizens to acquire an acceptable security posture goes behind an exclusive wall which they may not even be able to pay through to get access (likely because sales is overwhelmed or afraid of someone stealing their work), we stratify even further the "haves" and "have nots" in our society. Its not about giving up liberty for security as the old guard would argue, today, we can't get the security we need for liberty at all. The well intentioned mindset of the giants who's shoulders we stand on deprived them and us of security, making them the last generation to have had liberty due to their exclusive knowledge of the digital domain which now affects everyone. Doesnt sound all that altruistic in hindsight, and screams for the need of a new approach (without starting from zero like fuscia and the rust os are doing, since the world won't stop turning to wait).

I would definitely support crowd funding, though I have the bad feeling that there would be too few people actually donating, but I can't really judge how many users the patch has.

@minipli How is the current status? Could you give a rough estimation of how you would plan to go on from here? Do you plan to solve meltdown in a different approach than upstream or port PaX? And what is your opinion regarding the problem that 4.9 has an old KAISER implementation backported that is not really strong compard to KPTI? Finally, are the spectre patches easier to merge with PaX? Is there something we could do for you?

It seems that this is one of those problems money doesn't solve directly, or exclusively anyway - @minipli is (most likely) human, meaning only so much actual time and effort can be applied regardless of his level of desire. We need to find more people out there already familiar with the codebase/with time and skill available to grok it and at least try to get an effort going marking up the code with comments/docs to gain a better understanding for everyone (maybe a separate comment branch that everyone can PR on?). This should permit better functional separation, improvements/debugging by community, etc. The mystique of the codebase is likely intentional, and the first/biggest barrier to entry for the uninitiated. Without good documentation, even with all the effort @minipli is putting in this work will not be forward portable to newer kernels, and long-term, that's probably the goal to shoot for - its just too useful to let it die on the vine. For those of us who aren't useful to the implementation (especially systems/security engineers with access to lab compute resources on varied hardware), we can be useful in testing - however @minipli implements KPTI (and i hope its not KAISER as it has concerns other than efficacy as a KASLR defeat mitigation/meltdown reduction mechanism), it will need wide testing and reporting from the rest of us.

@sempervictus: Exactly. Can't agree more.

For 20 years, all the changes are included in the 20-megabyte patches, with no version control (spender probably did it intentionally), explanations, and they include everything from typo fixes, minor tweaks, hardening tricks and groundbreaking mitigation techniques, compiler plugins, etc.

PaX/grsecurity is spender's (and hackers behind the PaXTeam) personal project, and of course he can do whatever he wanted. He made great contributions to the entire community and history of computer security for totally two decades.

The problem is the entire Linux kernel security depends on this personal project (and if spender is correct, he should be, OpenBSD doesn't have a UDEREF-level of defence, so the entire Unix kernel security), and it has been here for 20 years, and nobody did anything to merge the code, improve the security, besides the patches from this personal project!

Perhaps it is the time to fork the Linux kernel and transform PaX/grsecurity to a community project of the general public now. A good start is splitting the functionality of the grsec patches to different modules (like creating a separate repo for the gcc plugins first), and the project will be supported by developers who share spender's security mindset, actively track upstream and continue development, merge useful fixes back to upstream, etc, you know what I mean.

But I doubt who is capable of doing it... Not a script kiddie like me of course. Given the fact that the KSPP is continue pushing the partial solution originated from PaX, "actively track upstream" is already a task only the real hackers capable of doing so.

@biergaizi The community fork thing is already there, it just needs more people. https://github.com/copperhead/linux-hardened/issues

Strcat works on a different OS, not general compute, they can rely on selinux isolation a lot more, and has somewhat different technical and organizational concerns that Linux users or systems ops. Pretty sure he's not chomping at the bit on this either. Guy's well deserving of all the praise in the world for CHOS, but this work needs time and a willingness to keep dealing with upstream... Upstream literally abhors security - try to count the number of times he publicly disparaged security practitioners as a group (from "those.... countries " sort of thing), and you'll see that Linux security isnt a thing by design. Also, BSD != Linux. The first hurdle it seems is finding folks capable of and willing to start documenting the code. Where's an army of grad students when you need em?

@sempervictus There has been work done already to try and split grsecurity patchset here : https://github.com/Shizmob/grsecurity-research

Linux-hardened was the original idea to start out of tree implementing grsecurity patchset for amd64 but not enough people actually were interested so strncat now only does arm64 work. I've been moving away from Linux myself, I have found *BSDs to be more open to certain things specially now most *BSDs use llvm/Clang where all the action happends.

The split patches / fork project etc. could be a long-term goal, for now the focus should be on fixing meltdown / spectre without loosing any grsec/pax functionality. Right now the state is sad as upstream kernel fixes those severe bugs but does not really offer good protection in general while the unofficial grsec/pax kernel is pretty vulnerable with those issues.

@minipli I hope you can update us soon on your state and how you would like to proceed.

Agree that we need meltdown addressed and to move on to the spectre variants in-tree, the thought process is that the page table protections are implemented in UDEREF upstream (in grsec code), which will likely require in-depth analysis of the feature proving an opportunity for commentary or even extraction.

It's not about the money, it's about the time I'm able to devote for this project. So please be patient and use an upstream kernel in the mean time if you need to.

@sempervictus I wonder how much progress has HardenedBSD made, they already have a more complete version of ASLR than OpenBSD, and it will be really cool if they managed to implement UDEREF. But, Meltdown...

@Esokrates Yes but 4.9 will be EOL in like 11 months and thats nearing fast. There are probably new variants on their way see : https://skyfallattack.com I doubt these will be the last to be uncovered this year.

Thanks @minipli for devoting time to fix this!