Unable to update Shelly Dimmer 2 to stock 1.10.x firmware

[CReimer]

Just wanted to let you know that it is not possible to update stock Shelly firmware beyond 1.9.6 after using tasmota-to-mgos. It's stuck on an "Index of" page.

Tried that with 4 different Shelly Dimmer 2 devices. Flashing a full binary backup from before installing Tasmota the first time, fixes it on all these devices.

Something is different between Stock Shelly Firmware and "Returned-To-Stock" Fimware using tasmota-to-mgos.

BTW: I tried replacing hwinfo_struct.json with the correct one for my Shelly Dimmer 2, but still unable to update to 1.10.

[jaamarcelis]

I have the same problem. @CReimer Can you provide me the binary backup of your Shelly Dimmer 2? Or is it not possible to flash an other binary backup to an other device?

[CReimer]

I used one of these. https://github.com/Mollayo/Shelly-Dimmer-2-Reverse-Engineering/tree/master/shelly%20stock%20firmware

I knew that my hw_revision was 20200629, but @Mollayo also provides one for hw_revision 20200904. I don't think it actually matters much, but after having trouble updating to 1.10 I think it's best to use one which fits your hardware the best

[jaamarcelis]

Thanks!

[joracadu]

Hello, I have been stuck in the same issue. I have downloaded one of the firmwares, but now I don't know how to upload it to the Shelly. How is it done? Thanks!

[CReimer]

You need to connect your Shelly via Serial connection to your PC and use a tool like Tasmotizer. You can select a custom bin file and flash the original Shelly firmware that way

[megakid]

I also came to this point, bummer - will extract from wall and reflash!

[ydeschoe]

anybody a solution for this, I have the same, latest version I can use is 1.9.4, when I go to 1.10.1 I always get the index page and not the dimmer start page.

[borft]

Nope, had the same problem. In the end solved it by connecting a serial cable and flashing with esptool. This, however, doesn't really solve the issue, especially for shellies that are already in the wall. So it would be great if this issue can be solved!

[To0wnn]

Hi,

mgos512k-0x7000.bin I can flash en wifi hotspot comes online I follow the instructions and it is not working. If I flash the 1.7 or 1.8 bin it is not working. Also tried the guide for shelly 1 to create firmware.bin also not working

Any suggestions?

[Dilergore]

I have the same issue. Would be good to be able to flash it back in an OTA way.

[To0wnn]

Found a way! Made a backup of different dimmer (that i already installed in the ceiling). esptool.py --port /dev/ttyUSB0 --baud 115200 read_flash 0x00000 0x400000 ~/Dimmer2.bin

Download and extract: Dimmer2.zip

Install esptool pip3 install esptool

First: esptool.py --port /dev/ttyUSB0 --baud 115200 erase_flash

Second: esptool.py --port /dev/ttyUSB0 --baud 115200 write_flash --flash_freq 80m 0x000000 Dimmer2.bin

Connect to the dimmer wifi, do a factory reset (not sure if nessesary) and your all good to go for a new setup!

[Dilergore]

Found a way! Made a backup of different dimmer (that i already installed in the ceiling). esptool.py --port /dev/ttyUSB0 --baud 115200 read_flash 0x00000 0x400000 ~/Dimmer2.bin

Download and extract: Dimmer2.zip

Install esptool pip3 install esptool

First: esptool.py --port /dev/ttyUSB0 --baud 115200 erase_flash

Second: esptool.py --port /dev/ttyUSB0 --baud 115200 write_flash --flash_freq 80m 0x000000 Dimmer2.bin

Connect to the dimmer wifi, do a factory reset (not sure if nessesary) and your all good to go for a new setup!

I would rather not remove the device from its current location, I would really appreciate if there would be a way to do this without plugging it to my computer :)

[To0wnn]

I would rather not remove the device from its current location, I would really appreciate if there would be a way to do this without plugging it to my computer :)

If that happens let me know 😊 Some hobby cost time 😋

[optilude]

Just to elaborate on this a little bit - and see if anyone might have a solution.

I have a new Shelly Dimmer 2. I tried to flash it OTA to Tasmota, which worked, but I couldn't get the Tasmota firmware to behave (the light would turn on and then off again after a few seconds). I pulled it out of the wall and used another Dimmer 2, with the stock firmware, and that works fine. So I decided to forgo Tasmota and stick with standard Shelly firmware.

I was going to USB-flash it, but in the process of putting the flasher pins into the device, the black plastic around the pin sockets disintegrated (I must've done something wrong) so it's now nearly impossible to get a connection to the USB flasher.

I re-connected the Shelly to power, and managed to get it to tasmota-minimal.bin, then used tasmota-to-mgos to get it into an intermediate state, then ran curl -i -F filedata=@./SHDM-2.zip http://10.42.42.44/update where SHDM-2.zip is a file I downloaded from the Shelly website after inspecting the API. This completed successfully.

At this point, a wifi network shellydimmer2-XYZ appeared as expected. On connecting to it and then to 192.168.33.1 I get a file listing that looks like this:

That looks like a web server directory listing to me. So I imagine that Mongoose 6.18 is running as expected, but whatever files were put there with the curl command do not conform to the required format.

Re-flashing from serial isn't an option for me, so if there is any way to rescue it, that would be amazing. I can ping the device for example, so it's responsive. I'd need some way of getting the right firmware to it though!

[Mollayo]

There are two MCU in the Shelly dimmer 2: the ESP8266 and the STM32. The web page with the file listing that you see is because the ESP8266 is not able to talk with the STM32. When you flash the dimmer with the Tasmota firmware, the firmwares of both MCU (i.e. ESP8266 and STM32) are modified. Then when you flash the dimmer with the Shelly firmware, only the ESP8266 firmware is modified. The firmware of the STM32 MCU is still the Tasmota version.

One solution would be:

• Flash the Shelly with this firmware https://github.com/Mollayo/Shelly-Dimmer-2/blob/master/shellyDimmer2.ino.generic.bin using a serial connection.
• This firmware creates a WiFi access point. You can connect to this AP and access the web page of the device. There is a button to update the STM32 firmware. This allows you to flash the STM32 with this firmware: https://github.com/Mollayo/Shelly-Dimmer-2/blob/master/STM32%20Firmware/stm_3F_02.bin. This stm_3F_02 firmware is from the Shelly firmware.
• Then flash the Shelly dimmer with the Shelly firmware as you did before. Your device should work again.

[optilude]

Thank you @Mollayo - very insightful.

In my particular case, a serial flash isn’t an option. I am fairly confident MongooseOS is running on it, and whilst I haven’t tested it yet I think it’s likely that the /update POST end point is still operational.

Is there any way to recover it OTA?

[Mollayo]

I see. Then you might be able to change the firmware from Shelly to Tasmota with the mgos-to-tasmota tool: https://github.com/yaourdt/mgos-to-tasmota. With your Internet browser: http://shellyip/ota?url=http://dl.dasker.eu/firmware/mg2tasmota-ShellyDimmer2.zip

Once the Tasmota firmware has been installed, you can then flash this firmware https://github.com/Mollayo/Shelly-Dimmer-2/blob/master/shellyDimmer2.ino.generic.bin.gz. You will then be able to change the STM32 firmware as described above.

[optilude]

Thank you again @Mollayo!

Just to check I got this right:

Originally, I went from stock firmware to Tasmota using mgos-to-tasmota. I then installed the template and custom STM32-aware firmware as per https://templates.blakadder.com/shelly_Dimmer_2.html. That didn’t work well with my light (but worked fine in terms of Tasmota itself, networking etc) so I used tasmota-to-mgos as per the instructions on this package to flash (using the /update HTTP endpoint) SHDM-2.zip I downloaded from the Shelly website.

What you suggest is:

• Use mgos-to-tasmota again as before to get to a basic Tasmota setup
• Use the normal Tasmota web interface to flash your shellyDimmer2.ino.generic.bin.gz file
• Connect to new AP and use it to flash STM32 with your stm_3F_02.bin
• Then use the /update endpoint with curl to flash SHDM-2.zip again
• Owe you a coffee

Did I get that right?

[optilude]

Wait - no, this doesn't work, because at present my Shelly is not yet connected to my wifi network. It exposes its own AP (shellydimmer2-XYZ) and I can reach it on 192.168.33.1 but I don't have the UI to configure wifi, only the aforementioned directory listing.

The mgos-to-tasmota approach relies on the Shelly itself being able to download firmware from an external website, so it needs to have a route to the internet, which means I'd need some way of configuring its networking before I could proceed as above.

[Mollayo]

You can actually setup a web server on your computer to serve files. See here: https://stackabuse.com/serving-files-with-pythons-simplehttpserver-module/.

Actually, I am trying to reproduce the steps. The conversion from the broken Shelly to Tasmota using the mgos-to-tasmota utility does not seem to work. The firmware is uploaded. But after reboot, the device becomes unreachable. It does not create the AP. I am investigating the problem.

[optilude]

Thank you! I’ll hold fire then as without an AP the device will be bricked for me (given no viable serial connection).

On Sun, 16 Jan 2022 at 10:47, Mollayo @.***> wrote:

You can actually setup a web server on your computer to serve files. See here: https://stackabuse.com/serving-files-with-pythons-simplehttpserver-module/ .

Actually, I am trying to reproduce the steps. The conversion from the broken Shelly to Tasmota using the mgos-to-tasmota utility does not seem to work. The firmware is uploaded. But after reboot, the device becomes unreachable. It does not create the AP. I am investigating the problem.

— Reply to this email directly, view it on GitHub https://github.com/yaourdt/tasmota-to-mgos/issues/7#issuecomment-1013852174, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABHJCX4B2J4HT7LDBHVAH3UWKO3RANCNFSM4ZV56ZLQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

[Mollayo]

Ok. It might take one or two days.

[Mollayo]

I think I found a way to change the malfunctioning shelly firmware to the tasmota firmware.

Actually, the mgos-to-tasmota tool assumes that the device is already connected to the Internet; this is required to get the tamosta binary to complete the conversion from ngos to tasmota. But since the WiFi credentials are not available (the shelly device works as a WiFi AP), the device does not connect to the home router; it does not create an AP neither. The only way to recover is to flash it with serial connection.

One solution would be to use a modified version of mgos-to-tasmota firmware in which the SSID and password of your WiFi network have been saved. This can be done by following the instructions here: https://github.com/yaourdt/mgos-to-tasmota#build-the-firmware-yourself The modifications to be done are in the file https://github.com/yaourdt/mgos-to-tasmota/blob/main/mos.yml:

• Delete the three lines:
  • ["wifi.ap.enable", false ]
  • ["wifi.sta1.ssid", "mgos-recover" ] # fallback SSID
  • ["wifi.sta1.pass", "RJoPuKC3u5" ] # fallback password
• Add the three following lines
  • ["wifi.ap.enable", true ]
  • ["wifi.sta.ssid", "Your SSID" ]
  • ["wifi.sta.pass", "Your password" ]

Once the mgos-to-tasmota firmware has been compiled, you need to setup a web server to make this firmware available to the shelly device. See here: https://stackabuse.com/serving-files-with-pythons-simplehttpserver-module/.

Then the update is done with the following instruction: curl http://192.168.33.1/ota?url=http://192.168.33.2:8000/fw.zip. 192.168.33.1 is the IP address of your Shelly device and 192.168.33.2 is the IP address of your computer. Your computer is assumed to be connected to the AP of the Shelly device. fw.zip is the file name of the mgos-to-tasmota firmware that has been previously compiled.

The steps are very tricky. It would be better to try first on another device that you can flash with serial connection in case things go wrong.

I can also provide you the modified version of the mgos-to-tasmota firmware.

[optilude]

Worth a try.

~~Any idea what the MODEL build var should be for a Dimmer 2?~~

Figured it out. It’s referenced in mos.yml

[Mollayo]

I think you better wait. I was doing the tests with an esp8266. When I tried on a real shelly dimmer, it did not work as expected. There are still some problems.

[optilude]

Ok. I'll hold fire. In the meantime, I've installed mos, cloned the mgos-to-tasmota firmware, and edited the mos.yml wifi config as suggested. Currently trying to figure out how to build it (it's failing with Error: Post "https://build.mongoose-os.com/api/fwbuild/2.20.0/build": net/http: TLS handshake timeout at the moment, but I'm still reading the docs on how the mos build command actually works).

Update: Installing docker and running mos build with the --local flag works. I now have build/fw.zip but I'm not going to attempt the "local" OTA mgos-to-tasmota yet per your last message.

[Mollayo]

I found that my Shelly dimmer is dead. It is just sending some garbage data over the serial line. Actually, I saw a flash light when I was uploading the firmware. I should have removed the bulb. That problem might not be related to the mgos-to-tasmota firmware. The dimmer was a spare device on which I was doing some experiments with the STM32 firmware. This could be the reason of the bricked device.

Most probably, the above method works but I cannot guaranty this. If you decide to go, please let me know the result. I can help you for the next steps (which hopefully will be easier).

[optilude]

I mean, it's not going to get any more useless than it already is, so why not try...

So I did this:

$ curl http://192.168.33.1/ota?url=http://192.168.33.2:8000/fw.zip
{"status":"updating","has_update":false,"new_version":"","old_version":"20211109-131033/v1.11.7-g682a0db"}

I've confirmed that http://192.168.33.2:8000/fw.zip resolves, i.e.:

$ curl -O http://192.168.33.2:8000/fw.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  859k  100  859k    0     0  49.3M      0 --:--:-- --:--:-- --:--:--  167M

$ unzip fw.zip 
Archive:  fw.zip
 extracting: dimmer-l51-0.4.6/manifest.json  
 extracting: dimmer-l51-0.4.6/dimmer-l51.bin  
 extracting: dimmer-l51-0.4.6/esp_init_data_default_v08.bin  
 extracting: dimmer-l51-0.4.6/fs.bin  
 extracting: dimmer-l51-0.4.6/rboot.bin  

$ ls dimmer-l51-0.4.6/
dimmer-l51.bin                  esp_init_data_default_v08.bin   fs.bin                          manifest.json                   rboot.bin

I've also confirmed that this zip file is similar in layout the "official" one fromhttp://dl.dasker.eu/firmware/mg2tasmota-ShellyDimmer2.zip. If I put my extract in a folder custom and the official one in official:

$ diff custom/ official/
Binary files custom/dimmer-l51.bin and official/dimmer-l51.bin differ
Binary files custom/fs.bin and official/fs.bin differ
diff custom/manifest.json official/manifest.json
2,3c2,3
<   "build_id": "20220116-175045/v0.4.6-gf2f935a-main-dirty",
<   "build_timestamp": "2022-01-16T17:50:45Z",
---
>   "build_id": "20210118-130502",
>   "build_timestamp": "2021-01-18T13:05:02Z",
22,23c22,23
<       "cs_sha1": "27db0d41678994fcf1fa01cad5c0e1f854c9cd7f",
<       "cs_sha256": "ddf1d02e745af6854550c60ee653602dae76c65665bb9f32e8319e708a001351",
---
>       "cs_sha1": "14028087936bd9ef52444bd5ceb7589c70fca073",
>       "cs_sha256": "aa1808f2eade39df7a88106ab70ad8a5ff4ddacd93d9e4e5a491416c489221cd",
32,34c32,34
<       "cs_sha1": "202b3ef817078f2be0e88bc14f75b71a74660fb3",
<       "cs_sha256": "ed7aa7ef56ab296957574e1b78ead0da497d3f5c3b632d75ce48244ea5631f00",
<       "size": 613248,
---
>       "cs_sha1": "597853f133fc0628b9d7339a97bcbef7ae852afe",
>       "cs_sha256": "8c8f9c973f0e72ea3408bd2d57135f6c929794346a85ac03232f83a21620d58c",
>       "size": 621120,

However, the device never reboots or does anything noticeable, i.e. I'm still connected to its shellydimmer2-XYZ access point.

Is there any way to get a log or any error messages out of it?

[Mollayo]

The shellydimmer2-XYZ AP is the AP of your dimmer ? If every thing goes well, your dimmer should connect to your home router. You should be able to see it in your home network.

[optilude]

Red herring. My Mac firewall was refusing the inbound connection.

The command completed and the shellydimmer2-XYZ AP disappeared. But I can't find it on my normal wifi network :(

This is what was in my mos.yml

# Custom configuration
config_schema:
  #### sys ####
  - ["debug.level", 2]                     # 0 = ERROR, 1 = WARN, 2 = INFO, 3+ = DEBUG
  #### wifi ####
  # - ["wifi.ap.enable",             false ]
  - ["wifi.sta.enable",             true ] # join wifi as client
  - ["sta_connect_timeout",           30 ] # timeout for wifi connection / seconds
  # - ["wifi.sta1.ssid",    "mgos-recover" ] # fallback SSID
  # - ["wifi.sta1.pass",      "RJoPuKC3u5" ] # fallback password
  - ["mg2x", "o", {"title": "Settings for this firmware"}]
  - ["wifi.ap.enable", true ]
  - ["wifi.sta.ssid", "MyNetwork" ]
  - ["wifi.sta.pass", "MyPassword" ]

I've double-checked the the SSID and password are correct (obviously modified in the paste above).

One question - why is it wifi.ap.enable = true? Could it be trying to broadcast its own AP with the same SSID as my home network?

If not, I imagine it's now unrecoverable. But nevermind. A huge, huge thank you for doing so much to help figure out a path!