linker scripts merge .rodata, .sdata, .data into executable pages by default which requires permission bleed-through

[bunnie]

This isn't so much an issue with Xous as it is with ELF and linker scripts in general.

If we look at the segments of a server, this is what we find:

riscv64-unknown-elf-readelf -S target/riscv32imac-unknown-xous-elf/
release/gam
There are 23 section headers, starting at offset 0x16e254:

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            00000000 000000 000000 00      0   0  0
  [ 1] .rodata           PROGBITS        00010100 000100 00678f 00  AM  0   0 16
  [ 2] .eh_frame_hdr     PROGBITS        00016890 006890 000a3c 00   A  0   0  4
  [ 3] .eh_frame         PROGBITS        000172cc 0072cc 002420 00   A  0   0  4
  [ 4] .text             PROGBITS        0001a6ec 0096ec 02bf9e 00  AX  0   0  2
  [ 5] .sdata            PROGBITS        00047690 035690 000030 00  WA  0   0  8
  [ 6] .data             PROGBITS        000476c0 0356c0 000028 00  WA  0   0  4
  [ 7] .sbss             NOBITS          000476e8 0356e8 000044 00  WA  0   0  4
  [ 8] .bss              NOBITS          0004772c 0356e8 0001c4 00  WA  0   0  4
  [ 9] .debug_loc        PROGBITS        00000000 0356e8 02ff28 00      0   0  1
  [10] .debug_abbrev     PROGBITS        00000000 065610 0008c7 00      0   0  1
  [11] .debug_info       PROGBITS        00000000 065ed7 054f4c 00      0   0  1
  [12] .debug_aranges    PROGBITS        00000000 0bae28 0008f0 00      0   0  8
  [13] .debug_ranges     PROGBITS        00000000 0bb718 010008 00      0   0  1
  [14] .debug_str        PROGBITS        00000000 0cb720 02e85b 01  MS  0   0  1
  [15] .debug_pubnames   PROGBITS        00000000 0f9f7b 00cb8c 00      0   0  1
  [16] .debug_pubtypes   PROGBITS        00000000 106b07 011e6f 00      0   0  1
  [17] .riscv.attributes RISCV_ATTRIBUTE 00000000 118976 00002b 00      0   0  1
  [18] .debug_line       PROGBITS        00000000 1189a1 010e66 00      0   0  1
  [19] .comment          PROGBITS        00000000 129807 000013 01  MS  0   0  1
  [20] .symtab           SYMTAB          00000000 12981c 034610 10     22 13385  4
  [21] .shstrtab         STRTAB          00000000 15de2c 0000ed 00      0   0  1
  [22] .strtab           STRTAB          00000000 15df19 01033a 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  p (processor specific)

Only the .text segment is slated as X; however, ELF packs it in with no alignment to pages, so it's impossible to enforce X-only on the text segment. Notably, this has caused some troubles because for some reason the .sdata and .data pages will cause page faults if they just so happen to line up into a region of memory that is not X (if you get "lucky" and things line up so they are on an exact page boundary after .text; it's a rare corner case but we have encountered it).

Some research indicates that "this is just how it's done" [1]. It makes a bit of sense, because if you have an executable that is quite small (the entire thing fits in 4096 bytes) then you would want to merge all the sections into one page, and essentially, you're now got .rodata, .sdata, and .data that are executable, readable, and writeable. Not a great situation. It seems Linux has moved on at least and split .rodata out with gold [2], but the RISCV toolchain needs a lot of finagling to get a half-solution [3].

"At least our stack and heap is no-ex" is better than nothing, which is our status quo, but, it does feel still a bit unsanitary. That being said, we are running on a small memory machine, and burning an extra page for some few bytes of .sdata/.data per process also may not be a savory decision.

This issue is opened so that anyone doing a security audit in the future is at least aware of this, even though we don't have a good resolution to it (and maybe someone may stumble across it and inform us a better way to do this that doesn't burn a huge amount of extra RAM in sparsely populated pages).

[1] https://stackoverflow.com/questions/44938745/rodata-section-loaded-in-executable-page [2] https://stackoverflow.com/questions/57761007/why-an-elf-executable-could-have-4-load-segments/57841768#57841768 [3] https://github.com/riscv/riscv-gnu-toolchain/issues/668

[bunnie]

It's noted that the linker script that is used to generate the above output is the default of the riscv toolchain (which can be viewed by running ld --verbose

[bunnie]

More notes:

To reproduce the problem:

1. Create a new user with the name of z (or any single character name)
2. Install the Rust toolchain, including stdlib, for the user z:

• sudo su z
• install rust for that user: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs > rustup-init
• install stdlib:

RUST_VERSION=`rustc +stable --version | cut -d' ' -f2`
cd $(rustc --print sysroot)
wget https://github.com/betrusted-io/rust/releases/latest/download/riscv32imac-unknown-xous_$RUST_VERSION.zip
unzip -o *.zip
rm *.zip
cd -

3. create a directory with the name sb_pre and clone xous-core into it, such that you now have a path prefix of /home/z/sb_pre/xous-core/
4. checkout this commit: https://github.com/betrusted-io/xous-core/commit/353311b9872e8dba6f54cdb3cf39f12cc35e114d
5. Burn both the loader and kernel that correspond to this commit.

You should see that the system gets past parsing the arguments and then panics in the codec process. This is because the exact combination of username and path cause .sdata and .data to end up on just exactly one page after .text, and that page is not marked as x. This will lead to a rather odd exception:

PROGRAM HALT: CPU Exception on PID 17: Instruction page fault of 0x00032000 at 0x00031ff2
Current thread 2:
Thread 2:
PC:00031ff2   SP:7ffe3550   RA:0001c334
GP:00000000   TP:60000000
T0:7ffe3680   T1:00024540   T2:00000000
T3:00000000   T4:00000000   T5:00000000   T6:00000000
S0:7ffefb30   S1:7ffe3680   S2:00000002   S3:7fffbf8c
S4:73756f78   S5:676f6c2d   S6:7265732d   S7:20726576
S8:00000010   S9:00000001  S10:00000006  S11:00000009
A0:7ffefb30   A1:00000000   A2:00000048   A3:00001000
A4:00000003   A5:7ffebb20   A6:00000000   A7:00001000
Memory Maps for PID 17:
       0 Superpage for 00000000 @ 40b20000 (flags: VALID)
          16 00010000 -> 40cee000 (flags: VALID | R | USER | A | D)
          17 00011000 -> 40ced000 (flags: VALID | R | USER | A | D)
          18 00012000 -> 40cec000 (flags: VALID | R | USER | A | D)
          19 00013000 -> 40ceb000 (flags: VALID | R | USER | A | D)
          20 00014000 -> 40cea000 (flags: VALID | R | USER | A | D)
          21 00015000 -> 40ce9000 (flags: VALID | R | USER | A | D)
          22 00016000 -> 40ce8000 (flags: VALID | R | USER | A | D)
          23 00017000 -> 40ce7000 (flags: VALID | R | X | USER | A | D)
          24 00018000 -> 40ce6000 (flags: VALID | R | X | USER | A | D)
          25 00019000 -> 40ce5000 (flags: VALID | R | X | USER | A | D)
          26 0001a000 -> 40ce4000 (flags: VALID | R | X | USER | A | D)
          27 0001b000 -> 40ce3000 (flags: VALID | R | X | USER | A | D)
          28 0001c000 -> 40ce2000 (flags: VALID | R | X | USER | A | D)
          29 0001d000 -> 40ce1000 (flags: VALID | R | X | USER | A | D)
          30 0001e000 -> 40ce0000 (flags: VALID | R | X | USER | A | D)
          31 0001f000 -> 40cdf000 (flags: VALID | R | X | USER | A | D)
          32 00020000 -> 40cde000 (flags: VALID | R | X | USER | A | D)
          33 00021000 -> 40cdd000 (flags: VALID | R | X | USER | A | D)
          34 00022000 -> 40cdc000 (flags: VALID | R | X | USER | A | D)
          35 00023000 -> 40cdb000 (flags: VALID | R | X | USER | A | D)
          36 00024000 -> 40cda000 (flags: VALID | R | X | USER | A | D)
          37 00025000 -> 40cd9000 (flags: VALID | R | X | USER | A | D)
          38 00026000 -> 40cd8000 (flags: VALID | R | X | USER | A | D)
          39 00027000 -> 40cd7000 (flags: VALID | R | X | USER | A | D)
          40 00028000 -> 40cd6000 (flags: VALID | R | X | USER | A | D)
          41 00029000 -> 40cd5000 (flags: VALID | R | X | USER | A | D)
          42 0002a000 -> 40cd4000 (flags: VALID | R | X | USER | A | D)
          43 0002b000 -> 40cd3000 (flags: VALID | R | X | USER | A | D)
          44 0002c000 -> 40cd2000 (flags: VALID | R | X | USER | A | D)
          45 0002d000 -> 40cd1000 (flags: VALID | R | X | USER | A | D)
          46 0002e000 -> 40cd0000 (flags: VALID | R | X | USER | A | D)
          47 0002f000 -> 40ccf000 (flags: VALID | R | X | USER | A | D)
          48 00030000 -> 40cce000 (flags: VALID | R | X | USER | A | D)
          49 00031000 -> 40ccd000 (flags: VALID | R | X | USER | A | D)
          50 00032000 -> 40ccc000 (flags: VALID | R | W | USER | A | D)

That means that the program counter was at address 0x00031ff2 when it tried to execute an instruction that it didn't have access to execute.

This is what the ELF layout looks like:

(gdb) compare-sections
Section .rodata, range 0x100f4 -- 0x14d73: matched.
Section .eh_frame_hdr, range 0x14d74 -- 0x15518: matched.
Section .eh_frame, range 0x15518 -- 0x16f60: matched.
Section .text, range 0x17f60 -- 0x32000: matched.
Section .sdata, range 0x32000 -- 0x32028: matched.
Section .data, range 0x32028 -- 0x32050: matched.
(gdb)

You can see that .sdata is perfectly aligned to a page (0x32000), not by design, but because it so happens all the strings in the build conspired to align it to a page. It's not marked as executable, because ELF doesn't say to do that, but, this leads to a crash.

This is the function that's being executed:

(gdb) disassemble 0x31ff2
Dump of assembler code for function memset:
   0x00031ff0 <+0>:     beqz    a2,0x31ffe <memset+14>
   0x00031ff2 <+2>:     mv      a3,a0
   0x00031ff4 <+4>:     sb      a1,0(a3)
   0x00031ff8 <+8>:     addi    a2,a2,-1
   0x00031ffa <+10>:    addi    a3,a3,1
   0x00031ffc <+12>:    bnez    a2,0x31ff4 <memset+4>
   0x00031ffe <+14>:    ret
End of assembler dump.
(gdb)

So this is crashing on a mv instruction, which doesn't touch data (this is weird and not expected).

Marking .sdata/.data as executable fixes this:

https://github.com/betrusted-io/xous-core/commit/0c372263ffa2d668bfe71fec645e6b9a477c41a7

Weird!

[bunnie]

Turns out that .sdata sections add 4096 bytes of padding from the end of .text, so, when mapped into virtual memory, the tail of code does not need to be writeable. Thus this is a non-issue and closing.