Moving components locally for development purposes can cause wrong dependency resolution (PACMAN-1203)

[Delzye]

The Component Manager version

v2.4.2

ESP-IDF Version

v6,0 Beta (e8ba1ed59a2b227809aa1d222db6bc8bb0045312)

python Version

3.11

Operating System

Windows 11

Browser (for https://components.espressif.com Issues)

Firefox

Description

I have a component A v0.1.0 as a dependency in the idf_component.yml of my project. A depends on B, which is v1.5.0. Both are downloaded from the esp-registry. Now I want do change something in B, a fix or whatever, so I move B locally to the components directory and change the version to v1.5.1. This works as expected, the new version is found and used in the compilation process.

But if I then want to change something in A as well and move it to the components directory also, the problems start. Note that this also happens if I do not change anything in A, just moving it from managed_compoents to components leads to this issue:

When recompiling now (deleting dependencies.lock and build artifacts and building) it picks up both B v1.5.0 and v1.5.1, which leads to A using v1.5.0 and not using the fix I implemented in v1.5.1.

I was wondering if I did something wrong, but just moving a dependency from managed to local should not influence the dependency resolution, or does it?

To Reproduce

Create a new project, add espressif/bmi270_sensor: "^0.1.0" as a dependency in the idf_component.yml of the main component. 
Let it build, then move espressif__i2c_bus from managed_components to components and change the Version from 1.5.0 to 1.5.1. Delete dependencies.lock, remove build-artifacts, recompile. It picks up 1.5.1.

NOTICE: [3/4] espressif/i2c_bus (1.5.1) (C:\esp32\Projects\CCON\remote_management_test\components\espressif__i2c_bus)

Then move espressif__bmi270_sensor from managed_components to components.  Delete dependencies.lock, remove build-artifacts, recompile.
It picks up both espressif__i2c_bus v1.5.1 and 1.5.0. 

NOTICE: [3/5] espressif/i2c_bus (1.5.1) (C:\esp32\Projects\CCON\remote_management_test\components\espressif__i2c_bus)
NOTICE: [4/5] espressif/i2c_bus (1.5.0)

I had this happen with other components as well, but some are proprietary so I put together this example using official components.

Expected behaviour

A (espressif/bmi270_sensor) should use the local version 1.5.1 of espressif/i2c_bus and not download a second, older version from the registry.

Additional info

No response

I have checked existing issues and online Documentation

• [x] I confirm I have checked existing issues and online Documentation.

[Delzye]

I should also note that I cannot force the local version of espressif/bmi270_sensor to use espressif/i2c_bus v1.5.1 by changing the idf_component.yml, this leads to this error:

  ERROR: Because espressif/bmi270_sensor (0.1.0) depends on espressif/i2c_bus
  (1.5.1) which doesn't match any versions, espressif/bmi270_sensor is
  forbidden.

It seems like it is not even looking for local versions

[kumekay]

Hello @Delzye

Could you please try the current development version? https://github.com/espressif/idf-component-manager?tab=readme-ov-file#installing-a-development-version-of-the-component-manager

I believe it was fixed in https://github.com/espressif/idf-component-manager?tab=readme-ov-file#installing-a-development-version-of-the-component-manager

[Delzye]

Hi, I ran the two python commands, curiously the version from the main branch reports itself as 2.4.0. Unfortunately, the behavior stayed the same, both versions are still found by the system:

NOTICE: [3/5] espressif/i2c_bus (1.5.1) (C:\esp32\Projects\CCON\remote_management_test\components\espressif__i2c_bus)
NOTICE: [4/5] espressif/i2c_bus (1.5.0)

[Delzye]

Hey @kumekay, you posted the same link twice, I assume you meant this commit?

I ensured my local version has these changes, but it still does not work. Since I was unsure how to enable debug messages when building from VSCode (idf.py build VERBOSE=1 unfortunately does not enable component manager debug messages), I added some prints to the code myself to try and figure out what happens, this is what I got:

1. the manager finds the local version of bmi270_sensor.
2. It knows i2c_bus is available locally (Using component placed at <path>\components\espressif__i2c_bus for dependency "espressif/i2c_bus"(introduced by component "espressif/bmi270_sensor"), specified in <path>\remote_management_test\main\idf_component.yml)
3. It asks the registry for available versions of i2c_bus (Fetching versions of component "espressif/i2c_bus" with spec "*" from https://components-file.espressif.com Fetched versions: 1.5.0, 1.4.2, 1.4.1, 1.4.0, 1.3.0, 1.2.0, 1.1.0, 1.0.0, 0.1.0)
4. It finds the local version for i2c_bus (1.5.1)
5. It drops i2c_bus from the solved_components, as the source does not match (service != local)
6. It looks for the local version of i2c_bus and finds 1.5.0 in managed_components
7. It reconfigures, because I have an optional dependency with a Kconfig option
8. Repeat 1-6.
9. Build the project using both versions of i2c_bus

Do you have any insight, why this still happens in the main from five days ago?

[kumekay]

@Delzye I'm sorry for the wrong link, yes, I meant that commit. We will check and come back to you soon!