Improv Wi-Fi over Serial fails with ESP32 devices

[bwims]

trafficstars

Hello,

I am having a real problem with using the web based utility to install ESPHome on ESP32 devices.

I can install on a Wemos D1 (8266) with no trouble, but when I try with an ESP32-S2 or ESP32-S3, the software installs, but then I get the message “An error occurred. Improv Wi-Fi Serial not detected.”

Can you confirm whether this is expected, and, if not, how to debug it?

Thanks!

B.

[balloob]

Make sure you run the latest version of ESPHome as we have made some fixes for those models. Our voice assistant ready-made project works just fine with ESP32-S3 boxes https://esphome.io/projects/

[bwims]

Thanks for your speedy help! One last question, I'm currently at ESPHome 2023.8.3 / Home Assistant 2023.10.3

Is it ok to go straight to the latest version of ESPHome, or should I upgrade HA first?

[EDIT] I took a chance and went to the latest Docker version of ESPHome - 2023.12.9 but the problem is still there. Do I need a later version?

Thanks again!

B.

[balloob]

Try the dev version. We skipped a release and it might not have been out yet.

[bwims]

Sadly no. Running Version: 2024.1.0-dev I still the error “An error occurred. Improv Wi-Fi Serial not detected.”

Just to make sure, I am trying to use https://web.esphome.io/?dashboard_wizard "Prepare For First Use"

with an ESP32 S2 Mini WIFI Board Based ESP32-S2FN4R2 4MB FLASH 2MB PSRAM

that looks like this:

I know there are workarounds, but if I can help debug it, I am willing. If there are any log files I can send, let me know where they are.

Thanks!

B.

[balloob]

I don't know about that board. In your initial report you mentioned that S3 doesn't work, but that we have working.

Try this URL https://esphome.io/projects/?type=empty

[bwims]

I did also mention S2

I can install on a Wemos D1 (8266) with no trouble, but when I try with an ESP32-S2 or ESP32-S3, the software installs, but then I get the message “An error occurred. Improv Wi-Fi Serial not detected.”

Since then I moved over to OpenHASP for the S3 device

However, I had tried the url referenced before. It performs the install ok, but afterwards there is not even an attempt at wifi configuration. The only choices offered are to reinstall or to look at the logs as shown in the screenshot below.

I can get the device working and adopted by creating an uploadable .bin fil but it is a clunky way to go.

(I actually used a tip I got from Reddit here to get the board specification right:

esp32:
  board: lolin_s2_mini
  variant: esp32s2
  framework:
    type: esp-idf

So I take it that the S2 is not supported yet by web tools? If so, can I make a plea for inclusion? I am willing to beta test!

Thanks!

B.

[bwims]

Hmm.... so is the S2 supported? Or not?

[sblantipodi]

I have the same problem on ESP32-S3 running the Glow Worm Luciferin firmware

There is a weird thing... If I click the install button, select the device, I get the error: Improv initialization failed. Error: Improv Wi-Fi Serial not detected and there is no option to configure WiFi:

but if then I click Logs and console and then Back the option to connect to WiFi is always there

@balloob sorry if I quote you but what does the back button that the install button does not?

[ldijkman]

I have the same problem on ESP32-S3 running the [Glow Worm Luciferin firmware]( but if then I click Logs and console and then Back the option to connect to WiFi is always there

same for me on ESP32 switch to logs and console back and forth several times until configure wifi apears not on esp8266 version only on esp32 thought it was my arduino coding not good

https://ldijkman.github.io/async-esp-fs-webserver/

[TD-er]

I've also noticed this behavior. Today I just looked into this a bit more and it looks like some kind of timeout issue in combination with not resetting the ESP right after flashing.

For example on an ESP32-C3 (USB via HWCDC, not USB to serial chip) I clicked to open the console in the webflasher menu right after flashing and I didn't even get my console output until I sent a reset via the terminal window. (thus not pressing RST button)

This could indicate the board may still have been in flash mode? So maybe the webflasher should try a few times more to get the IMPROV response and/or need to wait a bit longer for sending the reset signal?

[sblantipodi]

@TD-er have you tried a workaround for this problem? it is very hard to configure WiFi in this way since requires a lot of trials and errors and I don't know if all users may be able to do it without constantly reflashing the device.

[TD-er]

It is still a bit of trial-and-error as you need to toggle log-view, click "back" and see if IMPROV details are shown.

You can try and see for yourself with my ESPEasy web flasher: https://td-er.nl/ESPEasy/latest/

[sblantipodi]

I need to admit that the Tasmota version of the Web Installer works much better than this. Sadly it's impossible to flash a lot of devices with the esphome version...

[omar10wahab]

I've also noticed this behavior. Today I just looked into this a bit more and it looks like some kind of timeout issue in combination with not resetting the ESP right after flashing.

For example on an ESP32-C3 (USB via HWCDC, not USB to serial chip) I clicked to open the console in the webflasher menu right after flashing and I didn't even get my console output until I sent a reset via the terminal window. (thus not pressing RST button)

This could indicate the board may still have been in flash mode? So maybe the webflasher should try a few times more to get the IMPROV response and/or need to wait a bit longer for sending the reset signal?

I want to emphasize what youre saying cause thats what worked for me. Power cycling/resetting/pulling usb from the computer and plugging back in multiple times eventually got everything to work for me. Weird thing was i could flash the ESPhome firmware with little to no issue using whatever steps needed but the wifi is finnicky. I thought i had a bad ESP-01S and then i went and tried a ESP-32-C3 Super mini (all these from alibaba) and it gave me the same weird wifi over serial fail message. Resetting the device multiple times until i saw logs populating with no errors and then trying made it finally work for me. I then went back to my ESP-01s and tried a similar process of unplugging and plugging back in and it finally took the wifi and connected.

Another thing to is sometimes ungplugging and restarting after everything is set still makes things work weird. Theres something to these devices like you said related to powering off incorrectly or timing out weirdly but i encourage people to try several times before giving up. especially if your board is being recognized and flashing ESP home properly.

[mjhags]

Just been banging my head on this one. To get it to work for me I literally just went to the Logs view (from https://web.esphome.io/?dashboard_wizard) which showed a blank log screen, hit Reset Device and the Logs sprang into life. After that the Wifi install worked fine.

[scpotter]

Also got the error “improv wifi serial not detected” flashing Sonoff SV v1 (esp8266) on Web.ESPhome.

Similar symptoms: Logs showed as blank (reset did nothing), select configure wifi from menu repeats the error.

Resolution/workaround: Pulled the 3.3 power to board from TTL, plugged back in, selecting configure wifi from the menu was a success.

After that I was able to take control in ESPHome builder, edit and download.bin. Attempted to Install through Web.ESPhome, TTL was unable to connect, not sure if that’s expected. The TTL being used: https://a.co/d/2XODQYg

OTA from ESPHome builder worked fine.

[xbmcnut]

I spent hours trying to get my generic ESP32-C3's working and finally from info on this forum, I hit the reset device and everything sparked into life.

[ajfriesen]

I have an es32-c3 and flash it via a cp2102n.

It is the esp32-c3-mini-1u-h4 in a custom board.

However I always: get this message in the browser for the improv serial:

Initializing Improv Serial
install-dialog-CQzvAv3p.js:823 Improv initialization failed. Error: Improv Wi-Fi Serial not detected
    at install-dialog-CQzvAv3p.js:491:1210

Flashing works fine when I hit the boot button on my board, but the wifi setup does not.

Any idea how I can debug this further?

I have added the this to my esphome.yaml:

improv_serial:

---

Edit:

I figured out my problem. I am using a cp2101n for uart. I had to specify that with uart0 and then the wifi flow did start:

# Enable logging
logger:
  level: VERY_VERBOSE
  hardware_uart: uart0

esp32_improv:
  authorizer: none

[TheOrangeStuff]

@ajfriesen Newb here. I'm having the same issue. Can you elaborate on your solution? You had to specify uart0 ... where? That looks like a yaml? Sporting a Huzzah Esp32 v2 feather, connected via USB, trying to flash from web.esphome.io

Thanks in advance!

[ajfriesen]

@ajfriesen Newb here. I'm having the same issue. Can you elaborate on your solution? You had to specify uart0 ... where? That looks like a yaml? Sporting a Huzzah Esp32 v2 feather, connected via USB, trying to flash from web.esphome.io

Thanks in advance!

I have built a custom board myself. And I was not using the esp32 to flash the board.

My board was additionally connected to a uart chip. And in order to have the wifi improv to work, you have to setup the above mentioned optien. Because the logger and the wifi improv are using the same uart then.

If your board has a second uart chip you can try the same. If not, I can not help.

[the78mole]

I also have a strange problem with my custom board. I have esp32_improv an improv_serial in my yaml, but neither serial improv nor BLE improv work.

https://github.com/the78mole/ESPhome-KM271-WiFi/blob/main/buderus-km271_en.factory.yaml

I spent hours trying to get it working. If I just use the example/demo firmware from web tools (https://esphome.github.io/esp-web-tools/), everything works (serial and BLE). If I do a minimal config in ESPhome in my home assistant add the improv lines and delete the WiFi client lines (keeping the fallback hotspot) it also works. But as soon as it comes to km271-config, it doesn't. I also checked, which configuration has been applied in the GitHub actions (creating the release files), but also there, the improv lines appear to be active.

I'm totally out of ideas, how I can fix this problem 😭

Edit: Maybe I'm getting closer... The log shows some problems, because it can not connect to the heating controller via serial and after the error messages, it shows Improv,...,Enabling BLE,..., abort, a backtrace :

[...]
[21:01:39][E][km271:219]: Adding sensor with type param 0 to parameter 0x883B
[21:01:39][E][km271:219]: Adding sensor with type param 0 to parameter 0x893E
[21:01:39][E][km271:219]: Adding sensor with type param 1 to parameter 0x893F
[21:01:39][I][app:029]: Running through setup()...
[21:01:39][C][uart.arduino_esp32:077]: Setting up UART...
[21:01:39][D][binary_sensor:034]: 'USER 2': Sending initial state OFF
[21:01:39][C][switch.gpio:011]: Setting up GPIO Switch 'LED2_Green'...
[21:01:39][D][switch:016]: 'LED2_Green' Turning OFF.
[21:01:39][D][switch:055]: 'LED2_Green': Sending state OFF
[21:01:39][D][switch:016]: 'LED2_Green' Turning OFF.
[21:01:39][C][km271:142]: Setup was called
[21:01:39][D][KM271:029]: Sending Telegram [3]: 0xEE 00 00 
[21:01:39][C][adc.esp32:025]: Setting up ADC 'KM217 5V Supply'...
[21:01:39][C][esp32_ble:035]: Setting up BLE...
[21:01:39][C][wifi:048]: Setting up WiFi...
[21:01:39][C][wifi:061]: Starting WiFi...
[21:01:39][C][wifi:062]:   Local MAC: 90:15:06:4F:74:B0
[21:01:39][C][wifi:263]: Setting up AP...
[21:01:39][C][wifi:265]:   AP SSID: 'Fallback Hotspot'
[21:01:39][C][wifi:266]:   AP Password: 'Z8zfajgxVvNw'
[21:01:40][C][wifi:275]:   IP Address: 192.168.4.1
[21:01:40][D][wifi:482]: Starting scan...
[21:01:40][D][esp32_improv.component:252]: Setting Improv to start
[21:01:40][C][api:026]: Setting up Home Assistant API server...
[21:01:40][D][wifi:482]: Starting scan...
[21:01:40][I][app:062]: setup() finished successfully!
[21:01:40][D][sensor:094]: 'KM217 WiFi Signal Sensor': Sending state 0.00000 dBm with 0 decimals of accuracy
[21:01:40][D][esp32_ble:295]: Enabling BLE...
[21:01:43][W][component:237]: Component esp32_ble took a long time for an operation (2598 ms).
[21:01:43][W][component:238]: Components should block for at most 30 ms.
[21:01:43]
[21:01:43]abort() was called at PC 0x401ec8a7 on core 1
[21:01:43]
[21:01:43]
[21:01:43]Backtrace:0x40083829:0x3ffcd8900x4009320d:0x3ffcd8b0 0x400987d1:0x3ffcd8d0 0x401ec8a7:0x3ffcd950 0x401ec8ee:0x3ffcd970 0x401ec9e7:0x3ffcd990 0x401ec946:0x3ffcd9b0 0x401eca51:0x3ffcd9d0 0x401f0733:0x3ffcd9f0 0x400f6039:0x3ffcda40 0x400d9def:0x3ffcda80 0x400d9dfa:0x3ffcdaa0 0x40206199:0x3ffcdac0 0x40206235:0x3ffcdae0 0x400e7a44:0x3ffcdb00 0x400ea1fe:0x3ffcdb30 0x400f8ad1:0x3ffcdb50 
[21:01:43]
[21:01:43]
[21:01:43]
[21:01:43]
[21:01:43]ELF file SHA256: 0000000000000000
[21:01:43]
[21:01:43]Rebooting...
[21:01:43]ets Jun  8 2016 00:22:57
[21:01:43]
[21:01:43]rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[21:01:43]configsip: 0, SPIWP:0xee
[21:01:43]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[21:01:43]mode:DIO, clock div:2
[21:01:43]load:0x3fff0030,len:1184
[21:01:43]load:0x40078000,len:13132
[21:01:43]load:0x40080400,len:3036
[21:01:43]entry 0x400805e4
[21:01:43][I][logger:171]: Log initialized
[...and so forth...]

Could this be the root of all evil, that improv is not working properly?

[CoookieTM]

I spent hours trying to get my generic ESP32-C3's working and finally from info on this forum, I hit the reset device and everything sparked into life.

I have the same esp as OP and hitting this console reset twice fixed my issue.

[jwidess]

I was having the same issue with my ESP32-C3 Super Mini board. I tried so many things, external power supply, resetting, flashing from different methods and directly with pytool, etc. The fix ended up being as simple as adding the following to my YAML config,

wifi:
  output_power: 8.5dB

This seemed to have both fixed my improv serial issue alongside my Event: Disconnected ssid='SSID' bssid=[redacted] reason='Auth Expired' problem.

EDIT: While this did allow for WiFi to work properly, I realized that likely the real issue was the horrible range of the WiFi on the ESP32-C3 Super Mini, or at least this version, as moving the device next to my router alleviated all of the problems. And for context I was having these problems about 10ft from my router through open air...

Credits for this info:

• https://github.com/espressif/arduino-esp32/issues/2144#issuecomment-1212591202
• https://community.home-assistant.io/t/unable-to-connect-to-wifi-auth-expired-and-association-expired/678570/2?u=jwidess

[gopro2027]

I was having this issue with my device. I switched from chrome to edge and it worked fine. I've noticed in the past too, that sometimes chrome struggles more with serial connections and edge is much more consistent. Very odd.

[mjhags]

Oddly for me its really not working on all existing/older devices much either in my experience.

No idea what's actually changed, it could be my side of course, but I've not knowingly changed anything - but I'd provisioned multiple D1 Mini (8266) devices easily before, but had an issue with a newer device (as above, which was an esp32). But now I can't seem to provision the older devices either, and no workarounds seemed to help. So in the end I gave up and manually flashed the device with my own built image and pretty much forget the whole esp web convenience approach.