Request to update ESPTool package

[nullstalgia]

The last time esptool.py supplied with PlatformIO recieved an update was over a year ago. Since then, there have been a lot of improvements, especially in regards to reliability of starting and maintaining connections.

I've been using the ESP32-S3 in some recent projects, and I've found that the chips native USB-Serial port works great for flashing and debugging, but works way more reliably with those newer releases.

Note: There has been some slight changes in dependencies, so it's not a simple git pull and done. But I was able to get flashing with only a single pip install IntelHex on the platformio pyenv.

[dlbogdan]

see https://github.com/platformio/platform-espressif32/issues/1225

[MichaelEischer]

In case someone needs this (e.g. to flash an esp32c6), you can use the following snippets to automatically setup esptool 4.8.1:

Add the platform_packages and extra_scripts to your platformio.ini.

[env:seeed_xiao_esp32c6]
platform = https://github.com/platformio/platform-espressif32.git
board = seeed_xiao_esp32c6
framework = espidf
platform_packages =
    tool-esptoolpy@~1.40801.0
extra_scripts = extra_deps.py

Create extra_deps.py with the following content.

Import("env")

# Install missed package
try:
    import intelhex
except ImportError:
    env.Execute("$PYTHONEXE -m pip install intelhex")

[Jason2866]

or this

platform_packages = tool-esptoolpy @ https://github.com/pioarduino/esptool/releases/download/v4.8.9/esptool.zip

[grisharav]

I have an ESP32 IDF project which I now upgraded to latest platform = [email protected] and I got (during compilation):

ModuleNotFoundError: No module named 'intelhex'

Added

platform_packages = tool-esptoolpy @ https://github.com/pioarduino/esptool/releases/download/v4.8.11/esptool.zip

Seemed to work, but when I wanted to upgrade to latest 5.0.2, I got:

ModuleNotFoundError: No module named 'rich_click'

Any ideas what's the issue?

[Jason2866]

See the post above. You have to install the missing Python Module manually.

[grisharav]

See the post above. You have to install the missing Python Module manually.

I understand, the real question is if it can be bundled "out-of-the-box" in platformio.

[Jason2866]

No, not with the current espressif32 Platform. There are changes needed to work with esptool v5.0.2 (to get no deprecated warnings).

[valeros]

@grisharav It should work out of the box. Could you share here a larger portion of the log (preferably in verbose mode) so we can see exactly where the issue occurred?

[grisharav]

@grisharav It should work out of the box. Could you share here a larger portion of the log (preferably in verbose mode) so we can see exactly where the issue occurred?

Sure, without the line platform_packages = tool-esptoolpy@https://github.com/pioarduino/esptool/releases/download/v4.8.11/esptool.zip:

Traceback (most recent call last):
  File "/home/greg/.platformio/packages/tool-esptoolpy/esptool.py", line 41, in <module>
    import esptool
  File "/home/greg/.platformio/packages/tool-esptoolpy/esptool/__init__.py", line 42, in <module>
Compiling .pio/build/gpt_rev2/libad6/sgtcm_lib/esp32/network/esp32_wifi_mesh_network_manager.o
    from esptool.bin_image import intel_hex_to_bin
  File "/home/greg/.platformio/packages/tool-esptoolpy/esptool/bin_image.py", line 16, in <module>
    from intelhex import HexRecordError, IntelHex
ModuleNotFoundError: No module named 'intelhex'
*** [.pio/build/gpt_rev2/bootloader.bin] Error 1

With the line compiles just fine.

[valeros]

@grisharav can you share here also the header of the log?

[Jason2866]

@grisharav Why not using pioarduino? There is esptool v5.x default. @valeros Using esptool v5.x needs changes to use it. It needs the Python modules intelhex and rich_click Furthermore to avoid "deprecated" messages esptool needs to be installed, calling via python esptool.py is outdated and the possibility to do will be removed in future

[valeros]

@Jason2866 I understand, but @grisharav is trying to use the default v4 package that should work as expected.

[Jason2866]

@valeros Latest available esptool v4 version needs Python module "intelhex". In the past i forked esptool (for pioarduino use) and added the intelhex code directly in esptool. For more actual versions i adopted the espressif32 Platform to install all needed Python esptool modules

[valeros]

@Jason2866 I know, there was a workaround added to the package, so it should work by default.

[grisharav]

@grisharav Why not using pioarduino? There is esptool v5.x default. @valeros Using esptool v5.x needs changes to use it. It needs the Python modules intelhex and rich_click Furthermore to avoid "deprecated" messages esptool needs to be installed, calling via python esptool.py is outdated and the possibility to do will be removed in future

You suggested this here Is there another source for the 4.X version of esptool?

[Jason2866]

@grisharav just do not add manually a esptool version via platform_packages. Latest Platform will use an actual esptool v4.x version

[grisharav]

@grisharav just do not add manually a esptool version via platform_packages. Latest Platform will use an actual esptool v4.x version

without it:

Verbose mode can be enabled via `-v, --verbose` option
Commit ID="21c80e2fb"
Version="7.3.2.0"
Extended Version="7.3.2.0+dirty"
Generating log messages methods...
Generating proto file...
Proto file is up-to-date
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/pico32.html
PLATFORM: Espressif 32 (6.12.0) > ESP32 Pico Kit
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-espidf @ 3.50500.0 (5.5.0) 
 - tool-cmake @ 3.30.2 
 - tool-esp-rom-elfs @ 0.0.1+20241011 
 - tool-esptoolpy @ 2.40900.0 (4.9.0) 
 - tool-ninja @ 1.7.1 
 - tool-riscv32-esp-elf-gdb @ 12.1.0+20221002 
 - tool-xtensa-esp-elf-gdb @ 12.1.0+20221002 
 - toolchain-esp32ulp @ 1.23800.240113 (2.38.0) 
 - toolchain-xtensa-esp-elf @ 14.2.0+20241119

...

Traceback (most recent call last):
  File "/home/greg/.platformio/packages/tool-esptoolpy/esptool.py", line 41, in <module>
    import esptool
  File "/home/greg/.platformio/packages/tool-esptoolpy/esptool/__init__.py", line 42, in <module>
    from esptool.bin_image import intel_hex_to_bin
  File "/home/greg/.platformio/packages/tool-esptoolpy/esptool/bin_image.py", line 16, in <module>
    from intelhex import HexRecordError, IntelHex
ModuleNotFoundError: No module named 'intelhex'

[valeros]

@grisharav Could you please try the latest esptoolpy package? Just run pio pkg update -g -t tool-esptoolpy if it's already installed.

[grisharav]

@grisharav Could you please try the latest esptoolpy package? Just run pio pkg update -g -t tool-esptoolpy if it's already installed.

pio pkg update -g -t tool-esptoolpy
Tool Manager: [email protected] is already up-to-date

[valeros]

@grisharav You either installed a fork of this platform (the official platform doesn't use the original versions like 4.8.11) or you have several versions of the espressif32 platform in your system. What's the output of pio pkg list -g -p espressif32?

[grisharav]

@grisharav You either installed a fork of this platform (the official platform doesn't use the original versions like 4.8.11) or you have several versions of the espressif32 platform in your system. What's the output of pio pkg list -g -p espressif32?

greg@greg-lap-3581:~/control/local-control/SGTCM-PlatformIo$ pio pkg list -g -p espressif32
Platforms
├── espressif32 @ 6.12.0 (required: platformio/espressif32)

And I have used platform_packages = tool-esptoolpy@https://github.com/pioarduino/esptool/releases/download/v4.8.11/esptool.zip in the ini file, without it there is the error I described before

[valeros]

@grisharav you need to remove the tool-esptoolpy package platform_packages and run pio pkg update -g -t tool-esptoolpy again.

[grisharav]

I have removed it, and ran pio pkg update -g -t tool-esptoolpy, same result:

pio pkg update -g -t tool-esptoolpy
Tool Manager: [email protected] is already up-to-date

[valeros]

That's strange. OK, can you remove this folder /home/YOUR_USER/.platformio/packages/tool-esptoolpy and then run your project again?

[grisharav]

That's strange. OK, can you remove this folder /home/YOUR_USER/.platformio/packages/tool-esptoolpy and then run your project again?

Indeed compiled fine on my machine after the removal. What could be the root cause for this?

[valeros]

@grisharav

What could be the root cause for this?

Hard to guess, probably a mix up of different esptoolpy packages.