Support Arduino ESP32 v3.0 based on ESP-IDF v5.1

[ivankravets]

The ESP32 Core for Arduino 2.x is the most recent major version currently recommended for use with PlatformIO. The decision to discontinue support was made by the Espressif company, as indicated in their official statement: espressif/arduino-esp32#8606

If you're an Espressif user who relies on PlatformIO for your projects, kindly share your thoughts in the comments below and cast your vote on this matter. Once we gather enough interest from the Espressif community, we plan to reach out to the Espressif company and request a reconsideration of support for PlatformIO.

[thijstriemstra]

I've been trying this with following config:

; see https://docs.espressif.com/projects/arduino-esp32/en/latest/installing.html#how-to-update-to-the-latest-code
[env:arduino-latest]
platform = https://github.com/platformio/platform-espressif32.git
board = az-delivery-devkit-v4
framework = arduino
platform_packages =
    framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32#master

but the build fails with KeyError: 'framework-arduinoespressif32-libs':

Processing arduino-latest (platform: https://github.com/platformio/platform-espressif32.git; board: az-delivery-devkit-v4; framework: arduino)
--------------------------------------------------------------------------------
Platform Manager: Installing git+https://github.com/platformio/platform-espressif32.git
git version 2.42.0
Cloning into '/home/runner/.platformio/.cache/tmp/pkg-installing-uz1yxjz0'...
Platform Manager: espressif32@6.4.0+sha.f6ec392 has been installed!
Tool Manager: Installing git+https://github.com/espressif/arduino-esp32#master
git version 2.42.0
Cloning into '/home/runner/.platformio/.cache/tmp/pkg-installing-kih232ry'...
Tool Manager: framework-arduinoespressif32@3.0.0+sha.bf822ef has been installed!
Library Manager: Installing fortyseveneffects/MIDI Library @ ^5.0.2
Unpacking 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Library Manager: MIDI Library@5.0.2 has been installed!
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/az-delivery-devkit-v4.html
PLATFORM: Espressif 32 (6.4.0+sha.f6ec392) > AZ-Delivery ESP-32 Dev Kit C V4
HARDWARE: ESP32 240MHz, 520KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 3.0.0+sha.bf822ef 
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Converting edrumulus.ino
/tmp/tmpj5gstour/src/edrumulus.ino:45: warning: "MYMIDI" redefined
 #  define MYMIDI                     usbMIDI
 
/tmp/tmpj5gstour/src/edrumulus.ino:39: note: this is the location of the previous definition
 #  define MYMIDI                     MIDI
 
/tmp/tmpj5gstour/src/edrumulus.ino:46: warning: "MIDI_CONTROL_CHANGE_TYPE" redefined
 #  define MIDI_CONTROL_CHANGE_TYPE   usbMIDI.ControlChange
 
/tmp/tmpj5gstour/src/edrumulus.ino:40: note: this is the location of the previous definition
 #  define MIDI_CONTROL_CHANGE_TYPE   midi::ControlChange
 
/tmp/tmpj5gstour/src/edrumulus.ino:47: warning: "MIDI_SEND_AFTER_TOUCH" redefined
 #  define MIDI_SEND_AFTER_TOUCH      sendAfterTouchPoly
 
/tmp/tmpj5gstour/src/edrumulus.ino:41: note: this is the location of the previous definition
 #  define MIDI_SEND_AFTER_TOUCH      sendAfterTouch
 
KeyError: 'framework-arduinoespressif32-libs':
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/main.py", line 173:
    env.SConscript("$BUILD_SCRIPT")
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/runner/.platformio/platforms/espressif32@src-5f117260f75b328038ec9d3fd0e14a68/builder/main.py", line 346:
    target_elf = env.BuildProgram()
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 61:
    env.ProcessProgramDeps()
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 121:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Util/envs.py", line 242:
    return self.method(*nargs, **kwargs)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 342:
    SConscript(env.GetFrameworkScript(name), exports="env")
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 662:
    return method(*args, **kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/runner/.platformio/platforms/espressif32@src-5f117260f75b328038ec9d3fd0e14a68/builder/frameworks/arduino.py", line 41:
    SConscript(
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 662:
    return method(*args, **kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 598:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/runner/.platformio/packages/tool-scons/scons-local-4.5.2/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/runner/.platformio/packages/framework-arduinoespressif32/tools/platformio-build.py", line 40:
    FRAMEWORK_LIBS_DIR = platform.get_package_dir("framework-arduinoespressif32-libs")
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/platform/_packages.py", line 32:
    pkg = self.get_package(name)
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/platform/_packages.py", line 29:
    return self.pm.get_package(spec or self.get_package_spec(name))
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/platformio/platform/_packages.py", line 21:
    owner=self.packages[name].get("owner"),
========================== [FAILED] Took 4.31 seconds ==========================

[ivankravets]

See the updated comment #1225 (comment)

[Jason2866]

Adding Arduino 3.0.0 support is the only way to make the ESP32-C6 available for Arduino projects.
The needed changes are not that many, i have added Arduino 3.0.x alpha support in my Fork and bulding Arduino and IDF projects with the C6 are working.
We have ported Arduino project Tasmota to the C6 and it runs already surprisingly stable.

[Jason2866]

If you're an Espressif user who relies on PlatformIO for your projects, kindly share your thoughts in the comments below and cast your vote on this matter. Once we gather enough interest from the Espressif community, we plan to reach out to the Espressif company and request a reconsideration of support for PlatformIO.

@ivankravets Does this mean that Espressif has dropped any support? Looking in current Arduino 3.0.0 code (branch master) everything regarding Platformio support is there.

[VojtechBartoska]

Hello all,

I do work in Arduino Core ESP32 project as a Project Manager so I would like to clarify this from our Team point of view.

Please take a look on this comment in original Pull request.

We also update a warning note in our documentation which should be a bit confusing.

To not duplicate what is written in original PR, we will be glad if PlatformIO supports our latest version and we are open to any contributions.

@ivankravets Feel free to contact us any time directly, we will be happy to discuss this if you see any points we can do differently.

Thanks

[0x0fe]

Adding Arduino 3.0.0 support is the only way to make the ESP32-C6 available for Arduino projects.
The needed changes are not that many, i have added Arduino 3.0.x alpha support in my Fork and bulding Arduino and IDF projects with the C6 are working.
We have ported Arduino project Tasmota to the C6 and it runs already surprisingly stable.

@Jason2866
Do you know how i can setup the .ini file in a new project for using your 3.0.0 fork and 5.1 IDF at the same time (to keep the SDK menuConfig)?

[0x0fe]

@ivankravets
there is something i dont understand, why is there a separate platformespressif32 core? It is based on arduinoespressif32, but always lags behind, and arduinoesp32 already lags behind IDF. So what is the point of this separate core? Why is PIO not using arduinoesp32 core as is directly?

I think a reason why many users use PIO is also to keep some of the arduinoesp32 APIs while having the possibility to modify the IDF SDK configuration (menuconfig) which is necessary in many cases and not possible in arduino, unless using the libbuilder (buggy and not convenient).

Maybe it would be wise for PIO to directly use arduinoesp32 core as is, and just focus on allowing users to run arduinoesp32 core along with IDF to keep access to menuconfig.

[mhaberler]

I am concerned about the viability of this repo long-term as folks like @Jason2866 have started to roll their own and this repo not catching up

I am currently using his https://github.com/tasmota/platform-espressif32/tree/Arduino/IDF5 branch and it works mostly great for me - even if I feel uneasy about relying on a private effort with unclear future - how do you see the longer-term perspective of your fork?

This fork has diverged from this repo massively - not sure this can brought back here easily

the other issue I have is the abysmal debugging experience esp32/pio/arduino compared to using straight IDF

[0x0fe]

@mhaberler i did the same yesterday and used https://github.com/tasmota/platform-espressif32/tree/Arduino/IDF5

platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#3.0.0-alpha2
framework = 
	arduino
	espidf
board = esp32dev

But then when building it complained of missing SPIFFS, i added it manually from previous version (in components directory) but then i get a dependancy error with esp_partition.h, despite it is well present in "components".

   17 | #include "esp_partition.h"
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
Compiling .pio\build\esp32dev\tcp_transport\transport.o
Compiling .pio\build\esp32dev\tcp_transport\transport_ssl.o
Compiling .pio\build\esp32dev\tcp_transport\transport_internal.o
*** [.pio\build\esp32dev\spiffs\esp_spiffs.o] Error 1
*** [.pio\build\esp32dev\spiffs\spiffs_api.o] Error 1

Did you met this issue too?