crossSystem compiling does not work as expected

[dysinger]

Setting a crossSystem configuration in your nixpkgs setup so you can cross-compile for armv7l works well. Take for example a nixpkgs file like so:

let
  # NIXPKGS RELEASE-19.03 (STABLE) BRANCH @ 2019-07-20
  nixpkgs = builtins.fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/archive/6291c0c5ff52991f11ae14a64bcd4464c4ff6dd4.tar.gz";
    sha256 = "sha256:169iyph6qk9nq80w60whn65083z69ascl4046y9gdlgybh1lmr0p";
  };
  mozillaOverlay =
    # MOZILLA (RUST) OVERLAY (MASTER BRANCH @ 2019-07-20)
    (import
      (builtins.fetchTarball
        https://github.com/mozilla/nixpkgs-mozilla/archive/200cf0640fd8fdff0e1a342db98c9e31e6f13cd7.tar.gz));
  localOverlay = (super: self: rec {
    rustc = self.rustChannels.stable.rustc;
    cargo = self.rustChannels.stable.cargo;
  });
  overlays = [ mozillaOverlay localOverlay ];
  crossSystem = {
    config = "armv7l-unknown-linux-gnueabihf";
    platform = {
      name = "remarkable-cortex-a9";
      kernelMajor = "2.6";
      kernelBaseConfig = "multi_v7_defconfig";
      kernelArch = "arm";
      gcc = { cpu = "cortex-a9"; fpu = "neon"; };
    };
in import nixpkgs {
  inherit crossSystem overlays;
}

with such a file I can easily build any package for the remarkable tablet. take for example htop which compiles and links correctly and runs even (armv7l under qemu-system):

vega% nix build -f default.nix htop
vega% file ./result/bin/htop 
./result/bin/htop: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /nix/store/hyljfhyblnklp6iacs9aviiggd5d38hy-glibc-2.27-armv7l-unknown-linux-gnueabihf/lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, not stripped
vega% ./result/bin/htop --help
htop 2.2.0 - (C) 2004-2018 Hisham Muhammad
Released under the GNU GPL.

-C --no-color               Use a monochrome color scheme
-d --delay=DELAY            Set the delay between updates, in tenths of seconds
-h --help                   Print this help screen
-s --sort-key=COLUMN        Sort by COLUMN (try --sort-key=help for a list)
-t --tree                   Show the tree view by default
-u --user=USERNAME          Show only processes of a given user
-p --pid=PID,[,PID,PID...]  Show only the given PIDs
-v --version                Print version info

Long options may be passed with a single dash.

Press F1 inside htop for online help.
See 'man htop' for more information.
vega% 

Now let's try the same nixpkgs file above and try to install rustc from the overlay. Ooops it downloaded the x86 rustc and linked it with the arm libraries (!?). That'll never work.

vega% nix build -f default.nix rustc
vega% file ./result/bin/rustc              
./result/bin/rustc: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/store/hyljfhyblnklp6iacs9aviiggd5d38hy-glibc-2.27-armv7l-unknown-linux-gnueabihf/lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.9, with debug_info, not stripped
vega% ./result/bin/rustc          
zsh: accessing a corrupted shared library: ./result/bin/rustc
vega% 

[nbp]

@dysinger I have not noticed any issue when using the rust-overlay natively on arm64 devices.

Here is the location where the mapping between the system and the tarball to be downloaded is made: https://github.com/mozilla/nixpkgs-mozilla/blob/master/rust-overlay.nix#L38

Can you investigate what would have to be fixed to cross-download properly the version of Rust, as well as patching it correctly?

[dysinger]

Natively works fine. If you ask for an x86 rustc on x86 you get just that. If you ask for aaarch64 rust on aarch64 you get the right rust too. If you ask for armv7l binaries on x86 via normal Nixos you get just that ( see Beginner's Guide to Cross-Compiling blog post for a hello-world one-liner example.) If you ask for armv7l (arm32) 'rustc' on x86 you get this blob of binary that was downloaded x86_64 and patchelf-patched with linking armv7l libraries. It won't run (obviously) under any architecture (hosted or emulated.)

What needs to be fixed is, if binary downloads need to continue as opposed to compiling from source for Nixos, then we need to download an ARM binary when the stdenv.targetPlatform.config is ARM (not the host's architecture.)