esp-idf-sys
esp-idf-sys copied to clipboard
esp-rs
Fails to build on Nix
I have everything setup except for the esp-idf which this crate should handle. However, when attempting to build, I get following:
esp32-hello-world via (nix-shell-env) ❯ cargo build
Updating crates.io index
Compiling esp-idf-sys v0.32.1
error: failed to run custom build command for `esp-idf-sys v0.32.1`
Caused by:
process didn't exit successfully: `/home/odd/source/nix/temp/esp32-hello-world/target/debug/build/esp-idf-sys-abdf284afcdfa8df/build-script-build` (exit status: 1)
--- stdout
cargo:rerun-if-env-changed=ESP_IDF_TOOLS_INSTALL_DIR
cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG
cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG_DEFAULTS
cargo:rerun-if-env-changed=MCU
cargo:rerun-if-env-changed=ESP_IDF_SYS_ROOT_CRATE
cargo:rerun-if-env-changed=ESP_IDF_VERSION
cargo:rerun-if-env-changed=ESP_IDF_REPOSITORY
cargo:rerun-if-env-changed=ESP_IDF_CMAKE_GENERATOR
cargo:rerun-if-env-changed=IDF_PATH
cargo:rerun-if-env-changed=EXTRA-COMPONENTS
cargo:rerun-if-env-changed=ESP_IDF_COMPONENTS
PATH=/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4/tools:$PATH
--- stderr
Build configuration: BuildConfig {
esp_idf_tools_install_dir: None,
esp_idf_sdkconfig: None,
esp_idf_sdkconfig_defaults: None,
mcu: None,
native: NativeConfig {
esp_idf_version: Some(
Branch(
"release/v4.4",
),
),
esp_idf_repository: None,
esp_idf_cmake_generator: None,
idf_path: None,
extra_components: [],
esp_idf_components: None,
},
esp_idf_sys_root_crate: None,
}
Using managed esp-idf repository: RemoteSdk { repo_url: None, git_ref: Branch("release/v4.4") }
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
WARNING: Git describe was unsuccessful: b''
ERROR: This script was called from a virtual environment, can not create a virtual environment again
Error: Could not install esp-idf
Caused by:
command '"python3" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4/tools/idf_tools.py" "--idf-path" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4" "--non-interactive" "install-python-env"' exited with non-zero status code 1
All the code is located here.
ERROR: This script was called from a virtual environment, can not create a virtual environment again Error: Could not install esp-idf Caused by: command '"python3" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4/tools/idf_tools.py" "--idf-path" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4" "--non-interactive" "install-python-env"' exited with non-zero status code 1
Make sure that python3 doesn't come from a virtual env (see Determine if Python is running inside virtualenv for more info).
The esp-idf-sys will try to install the needed tools which includes creating a python virtualenv.
Ok, I've changed the shell now:
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
esp-dev = {
url = "github:thiskappaisgrey/nixpkgs-esp-dev-rust";
inputs.nixpkgs.follows = "nixpkgs";
};
};
outputs = {
self,
nixpkgs,
esp-dev,
}: let
pkgs = import nixpkgs {
system = "x86_64-linux";
overlays = [ esp-dev.overlay ];
};
in {
devShells.x86_64-linux.default = pkgs.mkShell {
buildInputs = with pkgs; [
gcc-riscv32-esp32c3-elf-bin
openocd-esp32-bin
# Tools required to use ESP-IDF.
git
wget
gnumake
flex
bison
gperf
pkg-config
cmake
ninja
ncurses5
python3
python3Packages.pip
python3Packages.virtualenv
];
shellHook = ''
export ESP_IDF_VERSION="v4.4"
export LD_LIBRARY_PATH="${pkgs.lib.makeLibraryPath [ pkgs.libxml2 pkgs.zlib pkgs.stdenv.cc.cc.lib ]}"
export LIBCLANG_PATH=${pkgs.libclang.lib}/lib
'';
};
};
}
Now I get this:
[939/941] Linking C executable libespidf.elf
[940/941] Generating binary image from built executable
esptool.py v3.2-dev
Merged 1 ELF section
Generated /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/libespidf.bin
[941/941] cd /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/esp-idf/esptool_py && /home/odd/source/nix/hello-esp32c3/.embuild/espressif/python_env/idf4.4_py3.10_env/bin/python /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/partition_table/check_sizes.py --offset 0x8000 partition --type app /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/partition_table/partition-table.bin /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/libespidf.bin
libespidf.bin binary size 0x21db0 bytes. Smallest app partition is 0x100000 bytes. 0xde250 bytes (87%) free.
cargo:root=/home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out
cargo:rerun-if-changed=/home/odd/.cargo/registry/src/index.crates.io-6f17d22bba15001f/esp-idf-sys-0.33.2/src/include/esp-idf/bindings.h
cargo:rustc-env=EMBUILD_GENERATED_BINDINGS_FILE=/home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/bindings.rs
--- stderr
Build configuration: BuildConfig {
esp_idf_tools_install_dir: None,
esp_idf_sdkconfig: None,
esp_idf_sdkconfig_defaults: None,
mcu: None,
native: NativeConfig {
esp_idf_version: Some(
Tag(
"v4.4",
),
),
esp_idf_repository: None,
esp_idf_cmake_generator: None,
idf_path: None,
extra_components: [],
esp_idf_components: None,
},
esp_idf_sys_root_crate: None,
}
Using managed esp-idf repository: RemoteSdk { repo_url: None, git_ref: Tag("v4.4") }
Using esp-idf v4.4.0 at '/home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4'
WARNING: directory for tool riscv32-esp-elf version esp-2021r2-8.4.0 is present, but tool was not found
WARNING: destination path already exists, removing
WARNING: directory for tool cmake version 3.20.3 is present, but tool was not found
WARNING: destination path already exists, removing
WARNING: directory for tool ninja version 1.10.2 is present, but tool was not found
WARNING: destination path already exists, removing
WARNING: directory for tool riscv32-esp-elf version esp-12.2.0_20230208 is present, but tool was not found
WARNING: destination path already exists, removing
/home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/tools/check_python_dependencies.py:12: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
CMake Warning:
Manually-specified variables were not used by the project:
CMAKE_TOOLCHAIN_FILE
Built components: esp_ringbuf, efuse, esp_ipc, driver, esp_pm, mbedtls, bootloader, esptool_py, partition_table, app_update, bootloader_support, spi_flash, nvs_flash, pthread, esp_gdbstub, espcoredump, esp_phy, esp_system, esp_rom, hal, vfs, esp_eth, tcpip_adapter, esp_netif, esp_event, wpa_supplicant, esp_wifi, ieee802154, console, openthread, lwip, log, heap, soc, esp_hw_support, riscv, esp32c3, esp_common, esp_timer, freertos, newlib, cxx, app_trace, asio, bt, cbor, unity, cmock, coap, nghttp, esp-tls, esp_adc_cal, esp_hid, tcp_transport, esp_http_client, esp_http_server, esp_https_ota, esp_https_server, esp_lcd, protobuf-c, protocomm, mdns, esp_local_ctrl, sdmmc, esp_serial_slave_link, esp_websocket_client, expat, wear_levelling, fatfs, freemodbus, idf_test, jsmn, json, libsodium, mqtt, openssl, spiffs, usb, tinyusb, wifi_provisioning
/nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:145:5: error: function-like macro '__GLIBC_USE' is not defined
/nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:184:5: error: function-like macro '__GLIBC_USE' is not defined
/home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/freertos/port/riscv/include/freertos/portmacro.h:439:20: warning: section does not match previous declaration [-Wsection]
/home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/freertos/port/riscv/include/freertos/portmacro.h:271:20: note: previous attribute is here
/home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/esp_hw_support/include/soc/esp_spiram.h:1:2: warning: esp_spiram.h has been replaced by esp32/spiram.h, please include esp32/spiram.h instead [-W#warnings]
clang diag: /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/freertos/port/riscv/include/freertos/portmacro.h:439:20: warning: section does not match previous declaration [-Wsection]
clang diag: /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/esp_hw_support/include/soc/esp_spiram.h:1:2: warning: esp_spiram.h has been replaced by esp32/spiram.h, please include esp32/spiram.h instead [-W#warnings]
Error: failed to generate bindings in file '/home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/bindings.rs'
Caused by:
clang diagnosed error: /nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:145:5: error: function-like macro '__GLIBC_USE' is not defined
/nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:184:5: error: function-like macro '__GLIBC_USE' is not defined
@knarkzel I am trying to make esp-idf for rust work on nix, same as you.
I took a step back and side-stepped the issue of the idf source not being a git repo, by unsetting IDF_PATH and letting esp-idf-sys download it at runtime (in a nix develop session).
Then, I stumbled on this __GLIBC_USE issue.
I think it has to do with nix wrapping clang and adding include paths to it. But esp-idf does not want to pull in host system headers, which are being mixed up with headers from the cross compiler.
So I removed clang from the shell inputs (but kept libclang), and it worked!
It would still be nice to be able to use the esp toolchain from the nix store though.
This is something I'm trying to get working too.
I've bumped a Flake I forked with the ESP-IDF, but encountering a Git repo error.
I think this will take some work.....
Does anyone have a working flake? I'm also getting the __GLIBC_USE issue.
@ChocolateLoverRaj i do with the dependencies of the flake above (actually without openocd-esp32-bin, flex, bison and gperf) but i put them in an FHS env with nix because the build process tried to execute it's own non-nix executables.
Its a two part process:
The first part is us installing essential stuff like the python virt_env + cmake + ninja + either riscv or xtensa gcc compiler needed for the specific ESP_IDF version, That could be more or less statically provided under the assumption that it follows the same layout ESP_IDF expacts.
The second part and the bigger problem is that ESP_IDF also uses pip and a bunch of python dependancys e.g kconfig wrapper and esp-idf-component manager written in python, that depends on other stuff it pulls in via pip. Keep in mind that upstream has this deeply coupled inside there system so just providing something similar is not enough, it will just fail on certain test that are run inline when the cmake buildsystem is executed.
So a complete solution would need to tackle the second part, the first part is more trivial.
@auroraanna can u provide your flake.nix? This is what I'm using atm: https://github.com/ChocolateLoverRaj/rust-esp32c3-examples/blob/main/flake.nix.
@ChocolateLoverRaj
{
inputs = {
nixpkgs.url = "nixpkgs/nixos-unstable";
};
outputs = {
self,
nixpkgs
}: let
pkgs = import nixpkgs {
system = "x86_64-linux";
};
fhs = pkgs.buildFHSUserEnv {
name = "fhs-shell";
targetPkgs = pkgs: with pkgs; [
gcc
pkg-config
libclang.lib
gnumake
cmake
ninja
git
wget
rustup
cargo-generate
espflash
python3
python3Packages.pip
python3Packages.virtualenv
];
};
in {
devShells.${pkgs.system}.default = fhs.env;
};
}
oh noes, just tried to delete all the build directories and see if it still works but somehow there's no compiler
updated the flake now and it works. it seems that with gcc-riscv32-esp32c3-elf-bin there's no compiler in the path and that openocd-esp32-bin isn't needed at all.
I tried your flake.nix but this is the error I get:
[rajas@nixos:~/Documents/rust-esp32c3-examples/VL53L0X]$ cargo r
Compiling std v0.0.0 (/home/rajas/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std)
Compiling esp-idf-hal v0.43.1
Compiling esp-idf-svc v0.48.1
Compiling esp32c3-vl53l0x v0.1.0 (/home/rajas/Documents/rust-esp32c3-examples/VL53L0X)
error[E0425]: cannot find function `dirfd` in crate `libc`
--> /home/rajas/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/pal/unix/fs.rs:898:37
|
898 | let fd = unsafe { libc::dirfd(self.0) };
| ^^^^^ not found in `libc`
For more information about this error, try `rustc --explain E0425`.
error: could not compile `std` (lib) due to 1 previous error
why is your toolchain for x86_64
Maybe because I have rustup installed with home-manager.
Hi all, hitting a similar issue, can't install using the toolchain provided with my Nix devenv.
It seems to be due to the fact that the folder where I have esp-idf installed (and compiled already) is not a Git repository. esp-idf is set up using the following overlay: https://github.com/mirrexagon/nixpkgs-esp-dev/tree/master
Code can be found here mostly under ./flake.nix and ./firmware: https://github.com/openmoto-org/kontroller/tree/3afc4c3d3f1719c5b877a401ff1d380883fd1d6b
The error log:
cargo build --target riscv32imc-esp-espidf
Compiling esp-idf-sys v0.34.1
Compiling firmware v0.1.0 (/Users/ar3s3ru/Code/openmoto-org/kontroller/firmware)
warning: function `main` is never used
--> build.rs:4:4
|
4 | fn main() {
| ^^^^
|
= note: `#[warn(dead_code)]` on by default
The following warnings were emitted during compilation:
warning: [email protected]: (esp-idf-sys) Removed env vars that might affect the ESP IDF C build: `CC=clang;CXX=clang++`
error: failed to run custom build command for `esp-idf-sys v0.34.1`
Caused by:
process didn't exit successfully: `/Users/ar3s3ru/Code/openmoto-org/kontroller/firmware/target/debug/build/esp-idf-sys-ba1c29dbb5b2bdf7/build-script-build` (exit status: 1)
--- stdout
cargo:warning=(esp-idf-sys) Removed env vars that might affect the ESP IDF C build: `CC=clang;CXX=clang++`
cargo:rerun-if-env-changed=ESP_IDF_TOOLS_INSTALL_DIR
cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG
cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG_DEFAULTS
cargo:rerun-if-env-changed=MCU
cargo:rerun-if-env-changed=ESP_IDF_SYS_ROOT_CRATE
cargo:rerun-if-env-changed=ESP_IDF_VERSION
cargo:rerun-if-env-changed=ESP_IDF_REPOSITORY
cargo:rerun-if-env-changed=ESP_IDF_CMAKE_GENERATOR
cargo:rerun-if-env-changed=IDF_PATH
cargo:rerun-if-env-changed=EXTRA-COMPONENTS
cargo:rerun-if-env-changed=ESP_IDF_COMPONENTS
cargo:rerun-if-env-changed=ESP_IDF_COMPONENT_MANAGER
--- stderr
Build configuration: BuildConfig {
esp_idf_tools_install_dir: None,
esp_idf_sdkconfig: None,
esp_idf_sdkconfig_defaults: Some(
[
"/Users/ar3s3ru/Code/openmoto-org/kontroller/firmware/sdkconfig.defaults",
],
),
mcu: Some(
"esp32c3",
),
native: NativeConfig {
esp_idf_version: None,
esp_idf_repository: None,
esp_idf_cmake_generator: None,
idf_path: Some(
"/nix/store/j7nh3a5rz8br2qligxl4xfav68rsn26q-esp-idf-v5.1.3",
),
extra_components: [],
esp_idf_components: None,
esp_idf_component_manager: None,
},
esp_idf_sys_root_crate: None,
}
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Error: '/nix/store/j7nh3a5rz8br2qligxl4xfav68rsn26q-esp-idf-v5.1.3' is not a git respository
Caused by:
0: command 'cd "/nix/store/j7nh3a5rz8br2qligxl4xfav68rsn26q-esp-idf-v5.1.3" && LANGUAGE="" LC_ALL="C.UTF-8" "git" "rev-parse" "--show-toplevel"' exited with non-zero status code 128
1: fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
warning: build failed, waiting for other jobs to finish...
I managed to make it work, here is a write-up of what was necessary: https://github.com/nix-community/fenix/issues/58#issuecomment-2156056797
Hope it helps!
why is your toolchain for x86_64
I removed rustup from my home-manager config, rm -rf ~/.rustup, and then used your Flake. And it worked. I just had to add ldproxy to it. This is what works:
{
inputs = {
nixpkgs.url = "nixpkgs/nixos-unstable";
};
outputs = {
self,
nixpkgs
}: let
pkgs = import nixpkgs {
system = "x86_64-linux";
};
fhs = pkgs.buildFHSUserEnv {
name = "fhs-shell";
targetPkgs = pkgs: with pkgs; [
gcc
pkg-config
libclang.lib
gnumake
cmake
ninja
git
wget
rustup
cargo-generate
espflash
python3
python3Packages.pip
python3Packages.virtualenv
ldproxy
];
};
in {
devShells.${pkgs.system}.default = fhs.env;
};
}
it's nice how your solution doesn't need to set any environmental variables so I'll be using it instead of my previous flake.
Another nice thing is that now I removed rustup and clang from my home-manager and just ran rust-rover . inside nix-develop. Now Rust Rover detects the toolchain I need and there's no NixOS issues with it.
Awesome stuff, seems to be solved for now then? Basically get an environment able to build esp-idf right?