`core install` uses incorrect vendor folder name

[jantje]

trafficstars

Describe the problem

The documentation at https://github.com/arduino/arduino-cli/blob/master/docs/package_index_json-specification.md states:

3rd party vendors should use a single PACKAGE_XXXX

and

name: the folder used for the installed cores.

However when the arduino ide installs the esp8266 version 2.1.2 from the sparkfun provided json file https://raw.githubusercontent.com/sparkfun/Arduino_Boards/master/IDE_Board_Manager/package_sparkfun_index.json the arduino IDE does not use the package name (esp8266) that contains the platform but uses "sparkfun".

Note that the sparfun provided json file has 2 packages (against the Arduino advice) one is named "Sparkfun" the other is called "esp8266". I'm assuming Arduino uses the name of the first package in the json file which is not compliant with the documentation. Can you clarify this as a bug in the code or a bug in the documentation?

To reproduce

Install the esp8266 version 2.1.2 from the sparkfun provided json file https://raw.githubusercontent.com/sparkfun/Arduino_Boards/master/IDE_Board_Manager/package_sparkfun_index.json

The platform is installed in packages/sparkfun

Expected behavior

Expected the platform is installed in packages/esp8266

Arduino CLI version

all

Operating system

N/A

Operating system version

windows 11

Additional context

No response

Issue checklist

• [x] I searched for previous reports in the issue tracker
• [x] I verified the problem still occurs when using the nightly build
• [x] My report contains all necessary details

[per1234]

Thanks for your report @jantje. I am not able to reproduce it. When I install the platform it is installed to the expected path:

<data directory>/packages/esp8266/hardware/esp8266/2.1.2

I see you specified "all" in the "Arduino CLI version" field of the issue report form. Please tell us the specific version of Arduino CLI you reproduced the fault with. You can get it by running the following command:

arduino-cli version

the sparfun provided json file has 2 packages (against the Arduino advice) one is named "Sparkfun" the other is called "esp8266".

The "3rd party vendors should use a single PACKAGE_XXXX" in the Package Index Specification is a bit ambiguous and you have misinterpreted it. What this is intended to communicate is that a single package should be used for all platforms and tools from a given vendor. For example, Arduino uses the arduino package for all our platforms and tools. However, it is perfectly fine and 100% supported for a package index to contain multiple packages: one for each of the vendors that distribute their platforms and tools via that package index. Arduino even does this in the primary package index since, in addition to being used to distribute Arduino's platforms and tools, this package index is also used to distribute platforms and tools from some of Arduino's partners (each of which have their own package in that one index).

[jantje]

I just tried it again here using Arduino IDE 2.3.3 windows. The sparkfun is installed in C:\Users\jan\AppData\Local\Arduino15\packages\SparkFun\hardware\esp8266\2.1.2

[jantje]

Is there still more information you want from me?

[cmaglie]

Could you provide all the external 3rd party URLs you are using?

IIRC The package names esp32 and esp8266 are used by Espressif. They opted to use the same string for both the package name and the architecture name, ignoring the recommendations from the documentation. The FQBN for their boards starts with esp32:esp32:... or esp8266:esp8266:....

What you're seeing is in your install dir is:

• packages/esp8266/tools are tools in the esp8266 package (there are no platforms here, probably some of those tools are referenced from other platforms)
• packages/esp32/... are platforms and tools in the esp32 package
• packages/Sparkfun/hardware/esp8266 is the actual platform/tools installed from Sparkfun

[cmaglie]

I see that Sparkfun has something going on in their package index:

$ curl -s https://raw.githubusercontent.com/sparkfun/Arduino_Boards/master/IDE_Board_Manager/package_sparkfun_index.json | \
   jq '.packages[] | { "name":.name, "platforms":[ .platforms[].architecture ]|unique, "tools":[ .tools[]| .name+" "+.version  ]|unique }' 
{
  "name": "SparkFun",
  "platforms": [
    "apollo3",
    "avr",
    "esp32",
    "samd"
  ],
  "tools": [
    "arm-none-eabi-gcc 8-2018-q4-major",
    "bossac 1.4.0"
  ]
}
{
  "name": "esp8266",
  "platforms": [
    "esp8266"
  ],
  "tools": [
    "esptool 0.4.5",
    "esptool 0.4.6",
    "esptool 0.4.8",
    "mkspiffs 0.1.2",
    "xtensa-lx106-elf-gcc 1.20.0-26-gb404fb9",
    "xtensa-lx106-elf-gcc 1.20.0-26-gb404fb9-2"
  ]
}

so you're right @jantje they define a second package esp8266 (instead of extending the existing Sparkfun as usual). I don't know why they chose to do it this way, I suppose they had some reason to do so...

[jantje]

The problem I'm reporting is

When the arduino ide installs the esp8266 version 2.1.2 from the sparkfun provided json file https://raw.githubusercontent.com/sparkfun/Arduino_Boards/master/IDE_Board_Manager/package_sparkfun_index.json the arduino IDE does not use the package name (esp8266) that contains the platform but uses "sparkfun".

According to @per1234 the CLI installs in /packages/esp8266 (which is what Sloeber does) As far as I can see the Arduino IDE installs in packages/sparkfun

I just want to have this inconsistency confirmed and know what it should be; so 3rd party implementers of the Arduino framework know where to install these platforms.

PS Note that I found this while investigating an issue and I thought this was the root cause; but I was wrong. At the time of writing I do not know -nor suspect- anything to fail because of the different implementation (apart from when platforms are shared between board managers who implement this differently -like CLI and Arduino IDE) .

[cmaglie]

I just tried from a clean install, and the Arduino IDE 2.3.4 installed inside packages/esp8266 and not in packages/Sparkfun.

This is the full dir structure:

cmaglie:~/.arduino15/packages$ tree -d -L 4
.
├── builtin
│   └── tools
│       ├── ctags
│       │   └── 5.8-arduino11
│       ├── dfu-discovery
│       │   └── 0.1.2
│       ├── mdns-discovery
│       │   └── 1.0.9
│       ├── serial-discovery
│       │   └── 1.4.1
│       └── serial-monitor
│           └── 0.14.1
└── esp8266
    ├── hardware
    │   └── esp8266
    │       └── 2.1.2
    └── tools
        ├── esptool
        │   └── 0.4.8
        ├── mkspiffs
        │   └── 0.1.2
        └── xtensa-lx106-elf-gcc
            └── 1.20.0-26-gb404fb9-2

24 directories

Could it be that Sparkfun changed the package_index (moving the esp8266 platform out of the Sparkfun package) after you installed it?

[jantje]

Could it be that Sparkfun changed the package_index (moving the esp8266 platform out of the Sparkfun package) after you installed it?

As far as I see they renamed the second package to sparkfun

What I find really strange is that I can't find the esp8266 product anymore in Arduino IDE (I tried 2.3.3 and couldn't find it so I installed 2.3.4 and again can't find it)

I'm confused. Something is wrong (I hope it is not my brain 😕 )