platform-espressif32 icon indicating copy to clipboard operation
platform-espressif32 copied to clipboard

Apple Silicon support without Rosetta

Open mailinger-mate opened this issue 2 years ago • 9 comments

Hello,

ESP-IDF v5.0 release should now have native support of macos-arm64 for Apple Silicon: https://github.com/espressif/esp-idf/releases/tag/v5.0-beta1

Until recently we still needed Rosetta to build a project on M1: https://github.com/platformio/platform-espressif32/issues/724

Would it possible to roll out native support without Rosetta on PlatformIO too?

Thank you!

mailinger-mate avatar Aug 31 '22 06:08 mailinger-mate

Hi @mailinger-mate ! Will be added once a stable IDF v5.0 is available.

valeros avatar Aug 31 '22 14:08 valeros

@mailinger-mate If you do not want to wait you can use my fork. I have compiled the espressif toolchains ([8.4.0-2021r2-patch3) for Apple ARM. The are used for Arduino and IDF 4.4. Performance gain between 15 and 25%. You can use with:

platform  = https://github.com/jason2866/platform-espressif32.git#Tasmota/205

No more Rosetta :-)

PACKAGES: 
 - framework-arduinoespressif32 @ 2.0.4 
 - framework-espidf @ 3.40403.20220830 (4.4.3) 
 - tool-cmake @ 3.16.4 
 - tool-esptoolpy @ 1.40201.220809 (4.2.1) 
 - tool-mklittlefs @ 1.203.210628 (2.3) 
 - tool-mkspiffs @ 2.230.0 (2.30) 
 - tool-ninja @ 1.9.0 
 - toolchain-riscv32-esp-arm @ 8.4.0+2021r2-patch3 
 - toolchain-xtensa-esp32s3-arm @ 8.4.0+2021r2-patch3

Jason2866 avatar Aug 31 '22 15:08 Jason2866

Processing esp32-s3 (board: esp32s3; platform: https://github.com/Jason2866/platform-espressif32.git#Tasmota/205; framework: arduino, espidf)
-----------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32s3.html
PLATFORM: Espressif 32 (2.0.5+sha.c1f5957) > Espressif Generic ESP32-S3 8M Flash, Tasmota 2944k Code/OTA, 2112k FS
HARDWARE: ESP32S3 240MHz, 320KB RAM, 8MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-builtin, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 2.0.4 
 - framework-espidf @ 3.40403.20220830 (4.4.3) 
 - tool-cmake @ 3.16.4 
 - tool-esptoolpy @ 1.40201.220809 (4.2.1) 
 - tool-mklittlefs @ 1.203.210628 (2.3) 
 - tool-mkspiffs @ 2.230.0 (2.30) 
 - tool-ninja @ 1.9.0 
 - toolchain-riscv32-esp-arm @ 8.4.0+2021r2-patch3 
 - toolchain-xtensa-esp32s3-arm @ 8.4.0+2021r2-patch3
Warning! Couldn't extract the list of installed Python packages.
Installing ESP-IDF's Python dependencies
Collecting cryptography<35.0.0,>=2.1.4
  Using cached cryptography-3.4.8-cp36-abi3-macosx_11_0_arm64.whl (1.9 MB)
Collecting future>=0.15.2
  Using cached future-0.18.2-py3-none-any.whl
Collecting pyparsing<2.4.0,>=2.0.3
  Using cached pyparsing-2.3.1-py2.py3-none-any.whl (61 kB)
Collecting kconfiglib==13.7.1
  Using cached kconfiglib-13.7.1-py2.py3-none-any.whl (145 kB)
Collecting cffi>=1.12
  Using cached cffi-1.15.1-cp39-cp39-macosx_11_0_arm64.whl (174 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: pyparsing, pycparser, kconfiglib, future, cffi, cryptography
  Attempting uninstall: pyparsing
    Found existing installation: pyparsing 2.3.1
    Uninstalling pyparsing-2.3.1:
      Successfully uninstalled pyparsing-2.3.1
  Attempting uninstall: pycparser
    Found existing installation: pycparser 2.21
    Uninstalling pycparser-2.21:
      Successfully uninstalled pycparser-2.21
  Attempting uninstall: kconfiglib
    Found existing installation: kconfiglib 13.7.1
    Uninstalling kconfiglib-13.7.1:
      Successfully uninstalled kconfiglib-13.7.1
  Attempting uninstall: future
    Found existing installation: future 0.18.2
    Uninstalling future-0.18.2:
      Successfully uninstalled future-0.18.2
  Attempting uninstall: cffi
    Found existing installation: cffi 1.15.1
    Uninstalling cffi-1.15.1:
      Successfully uninstalled cffi-1.15.1
  Attempting uninstall: cryptography
    Found existing installation: cryptography 3.4.8
    Uninstalling cryptography-3.4.8:
      Successfully uninstalled cryptography-3.4.8
Successfully installed cffi-1.15.1 cryptography-3.4.8 future-0.18.2 kconfiglib-13.7.1 pycparser-2.21 pyparsing-2.3.1

[notice] A new release of pip available: 22.1.2 -> 22.2.2
[notice] To update, run: pip install --upgrade pip
Warning! Arduino framework as an ESP-IDF component doesn't handle the `variant` field! The default `esp32` variant will be used.
Reading CMake configuration...
Generating assembly for certificate bundle...
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 0 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode
Compiling .pio/build/esp32-s3/src/Blink.o
Generating LD script .pio/build/esp32-s3/memory.ld
Compiling .pio/build/esp32-s3/app_trace/app_trace.o
Compiling .pio/build/esp32-s3/app_trace/app_trace_util.o
Compiling .pio/build/esp32-s3/app_trace/host_file_io.o
Compiling .pio/build/esp32-s3/app_trace/gcov/gcov_rtio.o
Compiling .pio/build/esp32-s3/app_update/esp_ota_ops.o
Compiling .pio/build/esp32-s3/app_update/esp_app_desc.o
Compiling .pio/build/esp32-s3/asio/asio/asio/src/asio.o
Compiling .pio/build/esp32-s3/bootloader_support/src/bootloader_common.o
Archiving .pio/build/esp32-s3/esp-idf/app_update/libapp_update.a
Archiving .pio/build/esp32-s3/esp-idf/app_trace/libapp_trace.a
Compiling .pio/build/esp32-s3/bootloader_support/src/bootloader_common_loader.o
Indexing .pio/build/esp32-s3/esp-idf/app_trace/libapp_trace.a
.
snip
snip
.
Indexing .pio/build/esp32-s3/bootloader/esp-idf/spi_flash/libspi_flash.a
Archiving .pio/build/esp32-s3/bootloader/esp-idf/xtensa/libxtensa.a
Archiving .pio/build/esp32-s3/bootloader/esp-idf/micro-ecc/libmicro-ecc.a
Indexing .pio/build/esp32-s3/bootloader/esp-idf/xtensa/libxtensa.a
Indexing .pio/build/esp32-s3/bootloader/esp-idf/micro-ecc/libmicro-ecc.a
Linking .pio/build/esp32-s3/bootloader.elf
Building .pio/build/esp32-s3/bootloader.bin
esptool.py v4.2.1
Creating esp32s3 image...
Merged 1 ELF section
Successfully created esp32s3 image.
Linking .pio/build/esp32-s3/firmware.elf
Retrieving maximum program size .pio/build/esp32-s3/firmware.elf
Checking size .pio/build/esp32-s3/firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [          ]   3.4% (used 11260 bytes from 327680 bytes)
Flash: [=         ]   5.1% (used 155661 bytes from 3080192 bytes)
Building .pio/build/esp32-s3/firmware.bin
esptool.py v4.2.1
Creating esp32s3 image...
Merged 2 ELF sections
Successfully created esp32s3 image.
====================================== [SUCCESS] Took 50.50 seconds ======================================

Environment    Status    Duration
-------------  --------  ------------
esp32-s3       SUCCESS   00:00:50.501
======================================= 1 succeeded in 00:00:50.501 =======================================

Jason2866 avatar Aug 31 '22 15:08 Jason2866

Good news, espressif released new 8.4.0 toolchains too. Now with native Apple Silicon support :-)

Jason2866 avatar Sep 13 '22 17:09 Jason2866

Rosetta is nowhere used anymore.

Jason2866 avatar Jan 17 '23 19:01 Jason2866

A fresh install still leads to the following output:

UserSideException: Processing esp32-c3-wican (platform: espressif32; board: esp32-c3-devkitm-1; framework: espidf)
--------------------------------------------------------------------------------
/Users/mickey/.platformio/penv/lib/python3.9/site-packages/urllib3/__init__.py:34: NotOpenSSLWarning: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
  warnings.warn(
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32-c3-devkitm-1.html
PLATFORM: Espressif 32 (6.4.0) > Espressif ESP32-C3-DevKitM-1
HARDWARE: ESP32C3 160MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-builtin) External (cmsis-dap, esp-bridge, esp-builtin, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-espidf @ 3.50101.230828 (5.1.1) 
 - tool-cmake @ 3.16.4 
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - tool-ninja @ 1.9.0 
 - tool-openocd-esp32 @ 2.1100.20220706 (11.0) 
 - tool-riscv32-esp-elf-gdb @ 12.1.0+20221002 
 - tool-xtensa-esp-elf-gdb @ 12.1.0+20221002 
 - toolchain-esp32ulp @ 1.23500.220830 (2.35.0) 
 - toolchain-riscv32-esp @ 12.2.0+20230208
Reading CMake configuration...
OSError: [Errno 86] Bad CPU type in executable: '/Users/mickey/.platformio/packages/tool-cmake/bin/cmake':
  File "/Users/mickey/.platformio/penv/lib/python3.9/site-packages/platformio/builder/main.py", line 173:
    env.SConscript("$BUILD_SCRIPT")
  File "/Users/mickey/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/Users/mickey/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/Users/mickey/.platformio/platforms/espressif32/builder/main.py", line 346:
    target_elf = env.BuildProgram()
  File "/Users/mickey/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/Users/mickey/.platformio/penv/lib/python3.9/site-packages/platformio/builder/tools/piobuild.py", line 61:
    env.ProcessProgramDeps()
  File "/Users/mickey/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/Users/mickey/.platformio/penv/lib/python3.9/site-packages/platformio/builder/tools/piobuild.py", line 121:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/Users/mickey/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/Users/mickey/.platformio/penv/lib/python3.9/site-packages/platformio/builder/tools/piobuild.py", line 342:
    SConscript(env.GetFrameworkScript(name), exports="env")
  File "/Users/mickey/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 662:
    return method(*args, **kw)
  File "/Users/mickey/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/Users/mickey/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/Users/mickey/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 1341:
    project_codemodel = get_cmake_code_model(
  File "/Users/mickey/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 222:
    run_cmake(src_dir, build_dir, extra_args)
  File "/Users/mickey/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 757:
    run_tool(cmd)
  File "/Users/mickey/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 712:
    result = exec_command(cmd, env=idf_env)
  File "/Users/mickey/.platformio/penv/lib/python3.9/site-packages/platformio/proc.py", line 114:
    with subprocess.Popen(*args, **kwargs) as p:
  File "/Applications/Xcode-15.0.0-Beta.6.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 951:
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Applications/Xcode-15.0.0-Beta.6.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 1821:
    raise child_exception_type(errno_num, err_msg, err_filename)
========================== [FAILED] Took 0.54 seconds ==========================

mickeyl avatar Sep 06 '23 10:09 mickeyl

So, for some reason, the host detection seems to be broken, since all tools for the x86_64 platform are installed.

Or is this a matter of not yet shipping the latest espressif-tools?

mickeyl avatar Sep 06 '23 10:09 mickeyl

I am getting the same error on a brand new MacBook Pro M3 Pro CPU with new fresh install. Does it still an issue?

astorun avatar Mar 07 '24 04:03 astorun

I didn't pursue it any further, since I moved away from platformio towards pure ESP-IDF. If you're using that, there are no dependencies on Rosetta.

mickeyl avatar Mar 07 '24 08:03 mickeyl