Not building on esp-idf-sys master (5.1) from directory

[axos88]

Hi,

Setting

IDF_PATH = { value = "/[..]/esp-idf" }

in .cargo/config.toml, pointing to a recent clone (09f7589ef23a6b78339224efd372257a57e1be4b) of the esp-idf repository, esp-idf does not build:

error: failed to run custom build command for `esp-idf-sys v0.31.9`

Caused by:
  process didn't exit successfully: `/home/akos/projects/rust/esp-rs/esp-doorlock/target/release/build/esp-idf-sys-fce0880a7ca583e1/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=ESP_IDF_TOOLS_INSTALL_DIR
  cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG
  cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG_DEFAULTS
  cargo:rerun-if-env-changed=MCU
  cargo:rerun-if-env-changed=ESP_IDF_SYS_ROOT_CRATE
  cargo:rerun-if-env-changed=ESP_IDF_VERSION
  cargo:rerun-if-env-changed=ESP_IDF_REPOSITORY
  cargo:rerun-if-env-changed=ESP_IDF_CMAKE_GENERATOR
  cargo:rerun-if-env-changed=IDF_PATH
  cargo:rerun-if-env-changed=ESP_IDF_COMPONENTS
  cargo:rerun-if-changed=/home/akos/projects/rust/esp-rs/esp-doorlock/sdkconfig.defaults
  CMAKE_PREFIX_PATH_riscv32imc-esp-espidf = None
  CMAKE_PREFIX_PATH_riscv32imc_esp_espidf = None
  TARGET_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_riscv32imc-esp-espidf = None
  CMAKE_riscv32imc_esp_espidf = None
  TARGET_CMAKE = None
  CMAKE = None
  running: "cmake" "/home/akos/projects/rust/esp-rs/esp-doorlock/target/riscv32imc-esp-espidf/release/build/esp-idf-sys-740a94ad1cf0d2c8/out" "-G" "Ninja" "-DCMAKE_TOOLCHAIN_FILE=/home/akos/projects/rust/esp-rs/esp-idf/tools/cmake/toolchain-esp32c3.cmake" "-DCMAKE_BUILD_TYPE=" "-DPYTHON=/home/akos/.espressif/python_env/idf5.1_py3.10_env/bin/python" "-DCMAKE_INSTALL_PREFIX=/home/akos/projects/rust/esp-rs/esp-doorlock/target/riscv32imc-esp-espidf/release/build/esp-idf-sys-740a94ad1cf0d2c8/out" "-DCMAKE_C_FLAGS= -march=rv32imc -ffunction-sections -fdata-sections -march=rv32imc -mabi=ilp32 -mcmodel=medany" "-DCMAKE_CXX_FLAGS= -march=rv32imc -ffunction-sections -fdata-sections -march=rv32imc -mabi=ilp32 -mcmodel=medany" "-DCMAKE_ASM_FLAGS=  -ffunction-sections -fdata-sections -march=rv32imc -mabi=ilp32 -mcmodel=medany"
  -- Checking Python dependencies...
  pkg_resources cannot be imported. The most common cause is a missing pip or setuptools package. If you've installed a custom Python then these packages are provided separately and have to be installed as well. Please refer to the Get Started section of the ESP-IDF Programming Guide for setting up the required packages.
  Skipping the download of /home/akos/projects/rust/esp-rs/esp-doorlock/.embuild/espressif/espidf.constraints.v5.1.txt because it was downloaded recently.
  Constraint file: /home/akos/projects/rust/esp-rs/esp-doorlock/.embuild/espressif/espidf.constraints.v5.1.txt
  Requirement files:
   - /home/akos/projects/rust/esp-rs/esp-idf/tools/requirements/requirements.core.txt
  Python being checked: /home/akos/projects/rust/esp-rs/esp-doorlock/.embuild/espressif/python_env/idf5.1_py3.10_env/bin/python
  -- Configuring incomplete, errors occurred!
  See also "/home/akos/projects/rust/esp-rs/esp-doorlock/target/riscv32imc-esp-espidf/release/build/esp-idf-sys-740a94ad1cf0d2c8/out/build/CMakeFiles/CMakeOutput.log".

  --- stderr
  Build configuration: BuildConfig {
      esp_idf_tools_install_dir: None,
      esp_idf_sdkconfig: None,
      esp_idf_sdkconfig_defaults: None,
      mcu: None,
      native: NativeConfig {
          esp_idf_version: Some(
              Tag(
                  "v5.1",
              ),
          ),
          esp_idf_repository: None,
          esp_idf_cmake_generator: None,
          idf_path: Some(
              "/home/akos/projects/rust/esp-rs/esp-idf",
          ),
          extra_components: [],
          esp_idf_components: None,
      },
      esp_idf_sys_root_crate: None,
  }
  Using activated esp-idf v5.1.0 environment at '/home/akos/projects/rust/esp-rs/esp-idf'
  CMake Error at /home/akos/projects/rust/esp-rs/esp-idf/tools/cmake/build.cmake:345 (message):
    Failed to run Python dependency check.  Python:
    /home/akos/.espressif/python_env/idf5.1_py3.10_env/bin/python, Error: 255
  Call Stack (most recent call first):
    /home/akos/projects/rust/esp-rs/esp-idf/tools/cmake/build.cmake:480 (__build_check_python)
    CMakeLists.txt:14 (idf_build_process)


  thread 'main' panicked at '
  command did not execute successfully, got: exit status: 1

  build script failed, must exit now', /home/akos/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.48/src/lib.rs:975:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[N3xed]

Try using esp-idf-sys master (it could be related to this). Also, it seems that esp-idf-sys detected an activated esp-idf environment (i.e. after export.sh was sourced), is this desired?

[axos88]

Well I was trying whatever I could think of to make it compile. I got it working by sourcing the export.sh and setting the IDF toolchain env to fromenv.

Tbh I have no idea what needs to be set up to use a cloned version of the IDF, some more thorough documentation would be nice.

[axos88]

I'll try esp IDF sys master tomorrow. I did have an issue with time_t not being 64bits, but adding the flag did not help, so wound up forking that too, and removing the checks.... And the code compiles and runs correctly, but seems super fishy.

[N3xed]

To use your own esp-idf repository, you should only need to set IDF_PATH (without sourcing ./export.sh), and esp-idf-sys will install all the required tools in the location configured by ESP_IDF_TOOLS_INSTALL_DIR or the <workspace-dir>/.embuild/espressif folder by default. (Note though that you need llvm, git and python installed.)

If you run with cargo build -vv it will show the build script output.

If there is an error installing the tools (i.e. checking the python dependencies fails), it may be worth it to remove the <install dir> and let it install again.

[axos88]

Yeah that fixed it thanks! How can I invoke tools such as otatool though?

[axos88]

or idf itself?

[ivmarkov]

I'll try esp IDF sys master tomorrow. I did have an issue with time_t not being 64bits, but adding the flag did not help, so wound up forking that too, and removing the checks.... And the code compiles and runs correctly, but seems super fishy.

Do you need to use ESP IDF 5 or 5.1? See #137

[ivmarkov]

To use your own esp-idf repository, you should only need to set IDF_PATH (without sourcing ./export.sh), and esp-idf-sys will install all the required tools in the location configured by ESP_IDF_TOOLS_INSTALL_DIR or the <workspace-dir>/.embuild/espressif folder by default. (Note though that you need llvm, git and python installed.)

If you run with cargo build -vv it will show the build script output.

If there is an error installing the tools (i.e. checking the python dependencies fails), it may be worth it to remove the <install dir> and let it install again.

@N3xed Using an activated ESP IDF environment used to work for ESP IDF < 5. What changed in ESP IDF 5? I vaguely remember some python related issue. Can we fix that?

[N3xed]

Using an activated ESP IDF environment used to work for ESP IDF < 5. What changed in ESP IDF 5? I vaguely remember some python related issue. Can we fix that?

That issue where an incorrect python was used, is already fixed on master. I haven't released it yet since there could be some more changes related to esp-idf v5.

It only happens when ESP_IDF_TOOLS_INSTALL_DIR is unset

[ivmarkov]

Using an activated ESP IDF environment used to work for ESP IDF < 5. What changed in ESP IDF 5? I vaguely remember some python related issue. Can we fix that?

That issue where an incorrect python was used, is already fixed on master. I haven't released it yet since there could be some more changes related to esp-idf v5.

It only happens when ESP_IDF_TOOLS_INSTALL_DIR is unset

OK. Let's wait until #137 is fixed, we then test the time_t thing and if it works OK we release.

[axos88]

Nuking the .embuild fixed the issue, except the TIME64 stuff.

[axos88]

workaround for the time64 stuff: Manually edit ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/Cargo.toml to include latest libc:

libc = { git = "https://github.com/rust-lang/libc.git", default-features = false, features = ['rustc-dep-of-std'] }

I also updated my binary crate's Cargo.toml to include:

[patch.crates-io]
libc = { git = "https://github.com/rust-lang/libc.git" }

but I'm not 100% sure this is necessary.

You will also need to add the following to .cargo/config.toml

rustflags = "--cfg espidf_time64 -C default-linker-libraries "

[ivmarkov]

but I'm not 100% sure this is necessary

It is not. If the two checks (for STD and libc) don't trigger, all is OK.

[ivmarkov]

Fixed since quite some time ago.