briansmith
Illegal instruction on arm-a72
NOTE: this is compiled with -target-cpu=native and --release, and target triple stable-aarch64-unknown-linux-gnu. But it fails with default target-cpu anyways
backtrace:
Thread 1 "musicalcat" received signal SIGILL, Illegal instruction.
ring_core_0_17_7_aes_hw_set_encrypt_key ()
at /home/katbot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.7/pregenerated/aesv8-armx-linux64.S:122
122 aese v6.16b,v0.16b
(gdb) bt full
#0 ring_core_0_17_7_aes_hw_set_encrypt_key ()
at /home/katbot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.7/pregenerated/aesv8-armx-linux64.S:122
No locals.
#1 0x0000005555f71bec in ring::aead::aes_gcm::init_256 ()
No symbol table info available.
#2 0x0000005555f6f72c in <ring::aead::unbound_key::UnboundKey as core::convert::From<ring::hkdf::Okm<&ring::aead::Algorithm>>>::from ()
No symbol table info available.
#3 0x0000005555f4e71c in rustls::tls13::key_schedule::KeySchedule::derive_decrypter::he91ce5cdc238f2b2 ()
No symbol table info available.
#4 0x0000005555f4db68 in rustls::tls13::key_schedule::KeyScheduleHandshakeStart::derive_client_handshake_secrets ()
No symbol table info available.
#5 0x0000005555f56a08 in rustls::client::tls13::handle_server_hello ()
No symbol table info available.
#6 0x0000005555f404a0 in <rustls::client::hs::ExpectServerHello as rustls::common_state::State<rustls::client::client_conn::ClientConnectionData>>::handle ()
No symbol table info available.
#7 0x0000005555f409f0 in <rustls::client::hs::ExpectServerHelloOrHelloRetryRequest as rustls::common_state::State<rustls::client::client_conn::ClientConnectionData>>::handle ()
No symbol table info available.
#8 0x0000005555eb7950 in rustls::common_state::CommonState::process_main_protocol ()
lscpu:
Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Vendor ID: ARM
Model: 3
Model name: Cortex-A72
Stepping: r0p3
CPU max MHz: 1500.0000
CPU min MHz: 600.0000
BogoMIPS: 108.00
L1d cache: 128 KiB
L1i cache: 192 KiB
L2 cache: 1 MiB
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Vulnerable
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fp asimd evtstrm crc32 cpuid
@jupyterkat Thanks for the bug report. Which operating system (kernel version and distribution version) are you using?
@jupyterkat Could you help with some additional info:
- What is the exact cargo/rustc version you are using (cargo --version / rustc --version)? Is this the standard toolchain you get from rustup.sh or a custom one?
- What is the output of
rustc --target=aarch64-unknown-linux-gnu --print cfgusing the rustc with which you are experiencing this issue? - What hardware is this? Cortex-A72 usually does have AES enabled but not for Raspberry Pi and a few others.
- Do you have any idea how I could emulate this configuration using QEMU?
- 1.74.1, standard toolchain from rustup.sh
- print cfg returns below
debug_assertions
panic="unwind"
target_arch="aarch64"
target_endian="little"
target_env="gnu"
target_family="unix"
target_feature="neon"
target_has_atomic="128"
target_has_atomic="16"
target_has_atomic="32"
target_has_atomic="64"
target_has_atomic="8"
target_has_atomic="ptr"
target_os="linux"
target_pointer_width="64"
target_vendor="unknown"
unix
- This is a
Raspberry Pi 4 Model B Rev 1.1with a BCM2835
not sure about emulating raspi4, qemu should already support it, or there is a patch to that.
I also get illegal instruction on Raspberry Pi 4 but only when using target-cpu=native.
Code:
use ring::signature::{EcdsaKeyPair, ECDSA_P256_SHA256_ASN1_SIGNING};
fn main() {
let rng = ring::rand::SystemRandom::new();
let pkcs8 = EcdsaKeyPair::generate_pkcs8(&ECDSA_P256_SHA256_ASN1_SIGNING, &rng).unwrap();
EcdsaKeyPair::from_pkcs8(&ECDSA_P256_SHA256_ASN1_SIGNING, pkcs8.as_ref(), &rng).unwrap();
}
Run with RUSTFLAGS='-C target-cpu=native' cargo run --release
Result from gdb:
Program terminated with signal SIGILL, Illegal instruction.
#0 sha256_block_armv8 () at /home/pi/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.17.7/pregenerated/sha256-armv8-linux64.S:1087
1087 .inst 0x5e2828a4 //sha256su0 v4.16b,v5.16b
pi@raspberrypi:~/dev/server $ rustc --version
rustc 1.77.0-nightly (fb5ed726f 2023-12-28)
pi@raspberrypi:~/dev/server $ rustc --target=aarch64-unknown-linux-gnu --print cfg
debug_assertions
overflow_checks
panic="unwind"
relocation_model="pic"
target_abi=""
target_arch="aarch64"
target_endian="little"
target_env="gnu"
target_family="unix"
target_feature="neon"
target_has_atomic
target_has_atomic="128"
target_has_atomic="16"
target_has_atomic="32"
target_has_atomic="64"
target_has_atomic="8"
target_has_atomic="ptr"
target_has_atomic_equal_alignment="128"
target_has_atomic_equal_alignment="16"
target_has_atomic_equal_alignment="32"
target_has_atomic_equal_alignment="64"
target_has_atomic_equal_alignment="8"
target_has_atomic_equal_alignment="ptr"
target_has_atomic_load_store
target_has_atomic_load_store="128"
target_has_atomic_load_store="16"
target_has_atomic_load_store="32"
target_has_atomic_load_store="64"
target_has_atomic_load_store="8"
target_has_atomic_load_store="ptr"
target_os="linux"
target_pointer_width="64"
target_thread_local
target_vendor="unknown"
unix
pi@raspberrypi:~/dev/server $ cat /proc/cpuinfo
processor : 0
BogoMIPS : 108.00
Features : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd08
CPU revision : 3
...
Model : Raspberry Pi 4 Model B Rev 1.4
Operating System: Debian GNU/Linux 12 (bookworm)
Kernel: Linux 6.1.0-rpi7-rpi-v8
I also get illegal instruction on Raspberry Pi 4 but only when using
target-cpu=native.
Thank you for solving the problem. target-cpu=native seems to do the wrong thing on Raspberry Pi on a Cortex-A72 CPU:
$ rustc --print cfg --target=aarch64-unknown-linux-gnu -C target_cpu=cortex-a72 | grep feature
target_feature="aes"
target_feature="crc"
target_feature="neon"
target_feature="pmuv3"
target_feature="sha2"
Without -C target_cpu=cortex-a72 we get the correct feature flags:
$ rustc --print cfg --target=aarch64-unknown-linux-gnu | grep feature
target_feature="neon"
This is a bug in rustc/LLVM or user error.
Coincidentally, for aarch64-apple targets, we always force dynamic detection of SHA512, and coincidentally PR #2040 generalizes that mechanism to allow us to force dynamic detection of any set of features on a per-OS basis. One way we could work around this issue with the compiler would be to merge PR #2040, then change the src/cpu/arm/linux.rs to have const FORCE_DYNAMIC_DETECTION: u32 = !NEON.mask; (or whatever works) to force all features except NEON to be dynamically detected for Linux and Android. If we were to do that then we'd need to document in a comment above that definition why we are doing this.
Related issues:
- Similar issue reported against BoringSSL: https://bugs.chromium.org/p/boringssl/issues/detail?id=460 ...
- ...which triggered https://bugzilla.yoctoproject.org/show_bug.cgi?id=14641
- ...and https://github.com/agherzan/meta-raspberrypi/issues/964
PR #2050 has been merged. It would be great if somebody could confirm that this fixes the issue for them.