Custom target files

[b-NC]

Hi, I have started working on trying to implement a way to feed the builder a path to a target JSON file.. This went about as well as I wanted it to, until I ran into this :

     Running `/home/infrandomness/CLionProjects/Arc/target/debug/xtask image /home/infrandomness/CLionProjects/Arc/target/test`
target : /home/infrandomness/CLionProjects/Arc/xtask/../targets/x86_64-arc-uefi.json
error: failed to run custom build command for `bootloader v0.10.12 (/home/infrandomness/CLionProjects/bootloader)`

Caused by:
  process didn't exit successfully: `/home/infrandomness/CLionProjects/bootloader/../kernel/target/release/build/bootloader-84b80c089ed2fd1a/build-script-build` (exit status: 101)
  --- stderr
  thread 'main' panicked at 'The UEFI bootloader must be compiled for the `x86_64-unknown-uefi` target.', build.rs:41:13
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'assertion failed: cmd.status()?.success()', src/bin/builder.rs:111:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'Build failed', xtask/src/main.rs:67:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Here are the changes I brought to the builder binary :

Index: src/bin/builder.rs
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/bin/builder.rs b/src/bin/builder.rs
--- a/src/bin/builder.rs	(revision 66637c11f5d459cfb4aa5d4f9538e70f012aa442)
+++ b/src/bin/builder.rs	(date 1645915730834)
@@ -1,14 +1,7 @@
 use anyhow::{anyhow, bail, Context};
 use argh::FromArgs;
 use bootloader::disk_image::create_disk_image;
-use std::{
-    convert::TryFrom,
-    fs::{self, File},
-    io::{self, Seek},
-    path::{Path, PathBuf},
-    process::Command,
-    str::FromStr,
-};
+use std::{convert::TryFrom, env, fs::{self, File}, io::{self, Seek}, path::{Path, PathBuf}, process::Command, str::FromStr};
 
 type ExitCode = i32;
 
@@ -46,6 +39,10 @@
     /// place the output binaries at the given path
     #[argh(option)]
     out_dir: Option<PathBuf>,
+
+    /// json target to build bootloader for
+    #[argh(option)]
+    target: Option<String>,
 }
 
 #[derive(Debug, Eq, PartialEq, Copy, Clone)]
@@ -96,7 +93,8 @@
         let mut cmd = Command::new(env!("CARGO"));
         cmd.arg(build_or_run).arg("--bin").arg("uefi");
         cmd.arg("--release");
-        cmd.arg("--target").arg("x86_64-unknown-uefi");
+        cmd.arg("--target")
+            .arg(&args.target.clone().unwrap_or(String::from("x86_64-unknown-uefi")));
         cmd.arg("--features")
             .arg(args.features.join(" ") + " uefi_bin");
         cmd.arg("-Zbuild-std=core");

I was wondering, if I were to dig more into the problem to implement this feature, would this work at all if the chose target is x86_64 UEFI or aarch64 UEFI ? I don't know a whole lot about UEFI so I am assuming that the code from an x86_64 UEFI bootloader and an ARM64 UEFI bootloader is pretty much the same; Maybe I'm wrong.

[bjorn3]

I just looked at your target spec (https://github.com/InfRandomness/Arc/blob/7aec1745ffaf568156a46894d4e3c45a8b976b38/targets/x86_64-arc-uefi.json) and it won't be able to produce a working uefi executable AFAIK. It doesn't specify the efi_application subsystem, it doesn't specify efi_main as entrypoint and it misses "abi-return-struct-as-int": true. Also youbmay want "exe-suffix": ".efi". I think it is best to use the official uefi target rather than a custom target.

[b-NC]

I just looked at your target spec (InfRandomness/Arc@7aec174/targets/x86_64-arc-uefi.json) and it won't be able to produce a working uefi executable AFAIK. It doesn't specify the efi_application subsystem, it doesn't specify efi_main as entrypoint and it misses "abi-return-struct-as-int": true. Also youbmay want "exe-suffix": ".efi".

Thanks for the quick reply. I will apply those changes.

I think it is best to use the official uefi target rather than a custom target.

Yeah, you're probably right, I'll think about this option as well, although I am keen on having my own spec.

[b-NC]

After a bit of researching, I found out that https://github.com/rust-osdev/bootloader/blob/main/build.rs#L25-L45 restricts the possibility to build the bootloader to those two targets only, is this because the bootloader itself has code that would not be compatible with custom specs (even x86_64 ones) ?

[phil-opp]

The bootloader crate currently only supports x86_64 unfortunately. While UEFI provides nice abstractions for lots of things, there are still platform-specific steps required, for example setting up a new page table to map the kernel into the virtual address space.

The bootloader is a completly separate executable separate from the kernel. So there should be no need to adjust the target that it is compiled for. In fact, the implementation relies on that exact target to work because a specific linker script is required to get the correct layout.

I'm currently working on a new v0.11 version of the bootloader that should make the build process more obvious. And it should also make it easier to port it to new architectures, even if this was not an explicit goal of the new design.

[b-NC]

The bootloader crate currently only supports x86_64 unfortunately. While UEFI provides nice abstractions for lots of things, there are still platform-specific steps required, for example setting up a new page table to map the kernel into the virtual address space.

The bootloader is a completly separate executable separate from the kernel. So there should be no need to adjust the target that it is compiled for. In fact, the implementation relies on that exact target to work because a specific linker script is required to get the correct layout.

I'm currently working on a new v0.11 version of the bootloader that should make the build process more obvious. And it should also make it easier to port it to new architectures, even if this was not an explicit goal of the new design.

Aww man, well, I will disable the aarch64 part in my kernel then. Please do let me know if I can be of any help on that task!

[b-NC]

The bootloader is a completly separate executable separate from the kernel. So there should be no need to adjust the target that it is compiled for. In fact, the implementation relies on that exact target to work because a specific linker script is required to get the correct layout.

I see; Therefore our kernel mustn't be compiled with an UEFI target, it must be compiled with a regular target. I was wondering if doing such a thing brings restrictions onto the kernel ? (like limited physical memory address space ?)

[bjorn3]

How do you define a "regular" target? What calling convention should it use? And what binary format?

[b-NC]

How do you define a "regular" target? What calling convention should it use? And what binary format?

Sorry I should've been more clear, in this context I meant a target like x86_64-unknown-none

[b-NC]

I have successfully made my kernel boot with the UEFI image produced by this crate. I was wondering if the https://os.phil-opp.com/ contained some instructions specific to UEFI (like how to use the FrameBuffer proposed by BootInfo).