Local API with Protocol Buffers

[drobtravels]

I recently had a SE10000H-US installed and noticed when I visit the IP address of the inverter it has a status page with some high level information (although no specifics about each optimizer). I see its using an unsecured API by doing a GET on http://<inverter_ip>/web/v1/status, with the response in protocol buffers.

I would be interested in reverse engineering this response so I could monitor the status in realtime without setting up the additional hardware involved in the ethernet passive approach. Has anyone done this?

I parsed the response using protoc --decode_raw and have a pretty good idea of what the structure of the data looks like, but am unclear how its actually converted to the proper number. For example it seems likely that 0x481b4600 corresponds to 159 kWh. I have no experience with protocal buffers, so I may be missing something obvious, but am not seeing how to get 159 or 159,000 from what I assume is the hex representation of 481b4600

I apologize if this is out of scope for this particular package, but thought some people here might be interested. I can post more details and examples if people are interested.

[drobtravels]

Ok just realized its a floating point number! This calculator can do the conversion.

Is there any interest in me posting samples or .proto files for working with this API?

[Jerrythafast]

That's quite interesting! Would you mind sharing your firmware version numbers?

[AndyRPH]

I'm interested in this. Just got my SE10000H online yesterday. Need to wait two weeks for meter swap out though before it's producing power. I've got the inverter mirrored to another switch port, and an rPi that just does a file a day of tcpdump.

If the inverters are changing their encryption keys a few times a day, being able to get all the same data your way would be much simpler!

[drobtravels]

I don't see anywhere that explicitly says firmware version, but I'm assuming this is that you're looking for?

I have probably 80% of the API response mapped which includes

• Kwh today
• Kwh this month
• Kwh this year
• Total lifetime kwh
• Current power from production meter
• Total kwh from production meter
• Current power
• Voltage
• Frequency
• number of optimizers communicating

[Jerrythafast]

Yes, that's what I was referring to, thanks! I think that's the first time I see a CPU version number starting with 4. Would be interesting to find out whether it offers more API functions than the one you discovered. More specifically, whether it would offer one that provides optimizer status! If we could get our hands on a recent firmware image we might be able to do some string search on it to find out?

[drobtravels]

Just clicking around in chrome with Dev tools on, I see several different API endpoints being used, but the only optimizer specific information I see displayed in browser is the number online.

I was looking at responses at night when it wasn't producing, so I'll try it again during the day in case the information is just in the API.

The weirdest thing about this dashboard is it's totally unauthenticated and appears to allow you to change any setting you want on the inverter, including grid protection, etc. I haven't tried changing any setting since I don't know what I'm doing. I wonder if the installer enabled it by mistake?

[AndyRPH]

I believe that's by design. My installer used a connection this way to do all the setup and provisioning of the inverter. My versions match yours exactly. Fingers crossed this provides more data than the existing public api.

[jbuehl]

SolarEdge was promoting this capability recently for installers https://marketing.solaredge.com/acton/rif/8801/s-144b-1902/-/l-0045:1133/l-0045/showPreparedMessage?sid=TV2:cfByisFlu.

If this interface can provide information about individual optimizers it is very interesting because the lack of that capability was the original motivation for this project. Hopefully it does and can be available on older inverters with a firmware upgrade.

[drobtravels]

Unfortunately I'm not seeing any details about optimizes (under then number communicating vs paired). Reading through the installer instructions, it has them pair via phone app, then go to the monitoring tool to setup the rest of the optimizer details. That makes me think the local API might not have these details implemented.

On the plus side, I realized that if you leave the dev console open, it actually logs out the parsed API responses as you're browsing the dashboard, so it should be very easy to use the APIs. I'll try to publish details of the APIs over the weekend.

[drobtravels]

I've been digging into this and am pulling out a lot of details. I'll post specifics soon.

I've found some optimizer specific information via the maintenance API

• Module
• Last Telemetry
• Output Voltage
• Input Voltage
• Input Current

Is that useful? I was hoping to see watts / power or something similar.

[jbuehl]

That's very good. Optimizers don't directly report power, but the watts can be calculated by multiplying the input voltage by the input current. The only other thing that would be useful would be the optimizer temperature.

[drobtravels]

Oh great! It does have the temperature in the API actually.

I'm not sure where you want this information posted, so I created my own repo. Happy to merge into your project if you prefer. https://github.com/drobtravels/solaredge-local

I went ahead and mapped what looked to me the most interesting data here

[dragoshenron]

Great job @drobtravels ! Thanks for sharing and for your effort. I made a pull request on your parallel repository with some addition on which inverters (US and European) are supported.

[drobtravels]

Just FYI, here's the information I'm getting back per optimizer

optimizer {
        serialNumber: "xxxx"
        online: true
        lastReport {
          year: 2019
          month: 3
          day: 11
          hour: 10
          minute: 20
          second: 29
        }
        outputV: 23
        inputV: 31
        inputC: 8
        temperature {
          value: 34
        }
      }

[AndyRPH]

Any idea when this might be available? I can pull the data off with your example command but not get it into a csv or such to put into a database nor homeassistant. Very cool to get all the optimizer data on a local api though!

[drobtravels]

I have it working on Home Assistant with a custom component. I'll hopefully be able to submit a PR this weekend.

I don't plan to develop anything which outputs to CSV, as that would require keeping a service running to constantly call the API and store the results. I'm happy to offer guidance if you'ld like to develop that though. You can also leverage the python library.

[AndyRPH]

Awesome! Looking forward to the PR so I can implement it locally until it's merged. I had hoped there was an easy way to convert the protoc output to csv so I could keep the data stored apart from HA, but if it gets into HA than its easy to pull out of HA. Thanks!

[drobtravels]

@AndyRPH There is no issue converting to CSV, but I assume you're looking for historical data in that case? The local API only gives you a snapshot of the current state. Typically CSVs contain lines of data by minute, hour, day, etc. To get historical data you'll need something to poll the API and constantly collect data.

[AndyRPH]

Yeah I just wanted to log it somewhere that was more robust than my rpi running home assistant. The maintenance page seems to include a time stamp in the data. I had hoped there was an easy way to convert the protocol buffer message output with the message definitions as the first row then all the data below, each query to the api being a row, since the "column" numbers should not change unless I change optimizer quantities. At the moment I just threw together a cronjob and bash script that outputs it to a unique log file every 15min. I can always just figure out a regex and clean the file up later on. It looks similar to but not quite json. I didn't see any good documentation on the --decode options for the protoc command line tool.

If it gets reliably into HA I'll just replicate that section of the database to another device to maintain it all.

[drobtravels]

Gotcha. Do you know any python? It should be relatively simple to write a python script using the library I made to create that.

[AndyRPH]

Nah, not a coder by trade but picked up enough knowledge to tinker, as my day job had me learn a relatively language for their hospital software.

Once your stuff is available I'll probably give python a good faith try since I'm better if I can reverse something working as a template to learning.

[jshank]

This API doesn't appear to be available for the LCD-based SE-10000H-US. I opened a ticket with SolarEdge and they pushed out the latest firmware on the same day but I'm not seeing a web interface or any major changes in behavior.

CPU Version: 3.2468.0 (updated per the request) DSP 1 Version: 1.0.751

I may have lost access to the data using semonitor though. All messages are now coming through with Decryption key not yet available.

[ContextPerspective]

@jshank

I think the web interface is only on the non-LCD models. SE marketing failed to rename the models based on this key design change, so it makes it fun with threads like this. ;)

I can personally confirm the same lack of web functionality on LCD Models SE6000H-USRGM AND SE3000H-USRGM. No access to the interface and refused message when attempting to connect.

[dragoshenron]

@jshank Indeed any inverter with display doesn't support these APIs (at least not as of today).

Model number is the same (with and without SetApp) but part number is different (see datasheet): SExxxxH-XXXXBXX4

[oliv3r]

The 4 in the 'cpu version' stands for the new software gen. The LCD only versions use '3' as it's software gen 3, as we see 'portia 3' throughout the codebases and firmwares. The gen 4 is called 'Portia-NG' as in net generation. These guys really need to work on naming and versioning, as the platform is really incomparable. I do wonder how the inside looks like, if they only changed the 'control' PCB. Also I wonder if this solution is cheaper then the LCD (more expensive development for the higher-level software).

Initially, when searching for firmware files for the version 3, I stumbled upon the software for the version 4 and was very pleasantly surprised that it was running Linux (our version 3's are not). As such, I started to take it apart and analyze it.

Sadly, I stumble upon this thread after you spend some painful amount to RE it. The protobuf protocol, and endpoints are available in the firmware quite visibly in python (well pyc, but that was easily resolved). :)

Once I realized however that this firmware was not four our devices, but only for the non-LCD ones, I was sad in that I could not run Linux on my inverter and kinda gave up on the project.

What was peculiar, is that they must have been working on this for a while, as they where using an extremely outdated and slow ARM chip ...

However, all the RE data is still there, with most of my findings. https://gitlab.com/centraluniverse/portia-ng_analysis

P.S. If someone can point me to a new firmware, ideally in chronological order, I don't mind extracting the firmware and adding it to the repo, to nicely track the changes between versions. Btw, I think I'm pretty sure I extracted the firmware image from the android apk, as it's part of the apk image. If I can find an easy way to download older versions (in between) I should be able to get historical data. This assumes that the app gets updated in sync with the inverter software, which is not unreasonable to think, as the app is intended for installers. Edit: This is where I found someone analyzing the App: https://gitlab.com/sehacker/inverter_configuration_tool/tree/setapp

[kingfisher63]

The current SetApp apk (v2.3.0) for Android does not contain any inverter firmware (the apk file is way to small for this anyway). Comparing the proto files from solaredge-local (apparently based on 4.4.67 or later) with the protobuf descriptors in 4.0.58 (from the Portia NG analysis by @oliv3r) shows that SolarEdge did not change essential parts. This may not be true in the future. We will need to find some other way to get to the protocol buffer message descriptors so that we can keep up with SolarEdge developments.

[AndyRPH]

If it's anything like the iOS app, you have to login to Setapp with an installer account and then it pulls new firmwares from their server. I just updated mine yesterday.

[kingfisher63]

Is there any evidence that updating firmware leaves files on the mobile device that can be recovered? (If so, this will probably be easier on Android than on iOS)

[AndyRPH]

The fact that it downloaded firmware to my phone before I connected it to my inverter means that it probably is there before. Maybe not after. I'm not sure how to get a look at what the iOS app may have downloaded since it's not jailbroken. Nor do I have an android device.

[oliv3r]

I've been looking at the code as well; and the URL is reasonably easy, the problem it's behind a login, e.g. you need an authentication token. I didn't spend time yet to see if a) your username needs some special level and b) making it it work :)

As for the file, it is guaranteed to be left on the device, for the simple reason, your inverter can't pull it itself (or you wouldn't use setapp) and thus the app first downloads it, and then uploads it to the device.

So after figuring out the rather simple curl command with the authentication token; it should be easy to get new firmware versions automatically downloaded.

That said, I've extracted the firmware from all 'old' versions (thank apkmirror) and will update the repo with the information where needed :)