plattform esp32-ble esphome - ENHANCEMENT

[riker65]

Hi as this is a perfect implementation for pulling data from the Renogy BT I am wondering if this can be migrated to esphome esp32-ble as well.

Thanks for checking

[mavenius]

FYI it looks like bleak isn't supported for micropython yet: https://github.com/orgs/micropython/discussions/13215

It has this though: https://github.com/micropython/micropython-lib/tree/master/micropython/bluetooth/aioble

[mateuszdrab]

@riker65

I had another go at this project yesterday having learnt more about Modbus recently and I've managed to get a somewhat very basic implementation of Renogy BLE Battery support on ESPhome, at the moment it just reads the basics.

Happy to share the code if this would meet your needs.

[riker65]

Hi Thanks a lot Looks great

Thanks for sharing the code? Is it in git? Thomas

[mavenius]

@mateuszdrab that looks great! I'd love to see the code as well, so I could play around with it. Thanks!

[mateuszdrab]

Hey guys, I dropped my existing code into https://gist.github.com/mateuszdrab/922c760582fce29d63608a1a405c541b for now.

Feel free to play around with it, just ensure to change the mac address field to your one.

As mentioned, it is nowhere near as functional as this repo as I was pretty much learning about interacting with the BT stack and Modbus as I was doing it but it does what I care about the most.

I think ultimately, the goal would be to replicate the same depth of data retrieved as this repo does and perhaps make it into an esphome component.

[mavenius]

0x30 in renogy_battery_bc is your battery ID, right? So 48 in base 10?

[mateuszdrab]

0x30 in renogy_battery_bc is your battery ID, right? So 48 in base 10?

Yeah, it's the modbus slave id. It might be the same on your battery, or it might be different. I am not sure since I've only got one of those.

 unsigned char newVal[8] = {
   0x30,       // device 48
   0x03,       // 3 READ - 6 WRITE
   0x13, 0xB2, // Register
   0x00, 0x06, // Word
   0x65, 0x4A  // CRC
 };

[mavenius]

Perfect; thanks! I have three batteries daisy chained that I can access as 48, 49, and 50 via renogy-bt. Having the breakdown of the bytes is super helpful.

On Mon, Jul 29, 2024, 07:15 Mateusz Drab @.***> wrote:

0x30 in renogy_battery_bc is your battery ID, right? So 48 in base 10?

Yeah, it's the modbus slave id. It might be the same on your battery, or it might be different. I am not sure since I've only got one of those.

unsigned char newVal[8] = {

0x30, // device 48

0x03, // 3 READ - 6 WRITE

0x13, 0xB2, // Register

0x00, 0x06, // Word

0x65, 0x4A // CRC

};

— Reply to this email directly, view it on GitHub https://github.com/cyrils/renogy-bt/issues/49#issuecomment-2255656435, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO65BDTTV2NGE545FND7MTZOYP3NAVCNFSM6AAAAABLC3VGL2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJVGY2TMNBTGU . You are receiving this because you commented.Message ID: @.***>

[mateuszdrab]

Perfect; thanks! I have three batteries daisy chained that I can access as 48, 49, and 50 via renogy-bt. Having the breakdown of the bytes is super helpful.

On Mon, Jul 29, 2024, 07:15 Mateusz Drab @.***> wrote:

0x30 in renogy_battery_bc is your battery ID, right? So 48 in base 10?

Yeah, it's the modbus slave id. It might be the same on your battery, or it might be different. I am not sure since I've only got one of those.

unsigned char newVal[8] = {

0x30, // device 48

0x03, // 3 READ - 6 WRITE

0x13, 0xB2, // Register

0x00, 0x06, // Word

0x65, 0x4A // CRC

};

— Reply to this email directly, view it on GitHub https://github.com/cyrils/renogy-bt/issues/49#issuecomment-2255656435, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO65BDTTV2NGE545FND7MTZOYP3NAVCNFSM6AAAAABLC3VGL2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJVGY2TMNBTGU . You are receiving this because you commented.Message ID: @.***>

Yeah, I think you'll need to recalculate the CRC though when you change them. Let me know how it goes for you.

Definitely want to put this in a repo so we consolidate the improvements.

[mavenius]

Yep, I figured that. I'll play with it ASAP; possibly today. Thanks again!

On Mon, Jul 29, 2024, 09:57 Mateusz Drab @.***> wrote:

Perfect; thanks! I have three batteries daisy chained that I can access as 48, 49, and 50 via renogy-bt. Having the breakdown of the bytes is super helpful.

On Mon, Jul 29, 2024, 07:15 Mateusz Drab @.***> wrote:

0x30 in renogy_battery_bc is your battery ID, right? So 48 in base 10?

Yeah, it's the modbus slave id. It might be the same on your battery, or it might be different. I am not sure since I've only got one of those. unsigned char newVal[8] = { 0x30, // device 48 0x03, // 3 READ - 6 WRITE 0x13, 0xB2, // Register 0x00, 0x06, // Word 0x65, 0x4A // CRC };

— Reply to this email directly, view it on GitHub #49 (comment) https://github.com/cyrils/renogy-bt/issues/49#issuecomment-2255656435, or unsubscribe

https://github.com/notifications/unsubscribe-auth/ABO65BDTTV2NGE545FND7MTZOYP3NAVCNFSM6AAAAABLC3VGL2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJVGY2TMNBTGU . You are receiving this because you commented.Message ID: @.***>

Yeah, I think you'll need to recalculate the CRC though when you change them. Let me know how it goes for you.

Definitely want to put this in a repo so we consolidate the improvements.

— Reply to this email directly, view it on GitHub https://github.com/cyrils/renogy-bt/issues/49#issuecomment-2256021205, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO65BEPSLNFT75MMJTZRSLZOZC5VAVCNFSM6AAAAABLC3VGL2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJWGAZDCMRQGU . You are receiving this because you commented.Message ID: @.***>

[riker65]

Hey guys, I dropped my existing code into https://gist.github.com/mateuszdrab/922c760582fce29d63608a1a405c541b for now.

Feel free to play around with it, just ensure to change the mac address field to your one.

As mentioned, it is nowhere near as functional as this repo as I was pretty much learning about interacting with the BT stack and Modbus as I was doing it but it does what I care about the most.

I think ultimately, the goal would be to replicate the same depth of data retrieved as this repo does and perhaps make it into an esphome component.

thanks a lot, I am on traveling at the moment, will test when I am back. T

[mavenius]

I have played with the code a bit and pulled some of the logic into a function that can be reused per-battery. Sorry for the possibly-garbage code; I'm very rusty with C++, having only used it in college and here and there for Arduino stuff.

interval:
  - interval: 30s
    then:
      - ble_client.ble_write: 
          characteristic_uuid: "FFD1"
          service_uuid: "FFD0"
          
          id: renogy_battery_esp32_bc
          # todo: update 0x30 and the last value to be the battery number and the checksum, resp.
          value: !lambda |-
                      vector<uint8_t> request = GetBatteryRequest(0x30);
                      for (size_t i = 0; i < request.size(); ++i) {
                        ESP_LOGD("main", "Request Byte %d: 0x%02X", i, request[i]);
                      }
                      return request;

Where GetBatteryRequest is defined as such:

#include <vector>
using namespace std;

uint16_t GetCRC16(vector<uint8_t> data);

vector<uint8_t> GetBatteryRequest(uint8_t batteryNumber) {
    vector<uint8_t> dataBytes = { batteryNumber, 0x03, 0x13, 0xB2, 0x00, 0x06 };

    // add checksum to the end
    uint16_t checksum = GetCRC16(dataBytes);

    // checksum needs to be split into 2 bytes
    dataBytes.push_back((checksum >> 0) & 0xFF);
    dataBytes.push_back((checksum >> 8) & 0xFF);

    return dataBytes;
}


uint16_t GetCRC16(vector<uint8_t> data) {
    uint16_t crc = 0xFFFF;
    int i;

	for(auto item = data.begin(); item != data.end(); ++item){
        crc ^= (uint16_t)*item;
        for (i = 0; i < 8; ++i) {
            if (crc & 1)
                crc = (crc >> 1) ^ 0xA001;
            else
                crc = (crc >> 1);
        }
    }
    ESP_LOGD("utilities", "GetCRC16 %d", crc);
    
	return crc;
}

I haven't looked yet at how to push the data into different sensors for each battery (e.g. renogy_battery_30_esp32_current instead of renogy_battery_esp32_current) but making those generic is on my list.

[mavenius]

Well I found a way not to do it.

Setting up three ble_clients is not the way to make it work. But as I think about it now, that's obviously not needed; the data passed back when querying includes the battery number. So we can just switch based on that to decide which entities to update.

[mavenius]

@cyrils is this something you want as part of this project, or should it be spun off? It's obviously related (and heavily influenced) by what you've done, but it's also a big departure from what you have built.

[mavenius]

Actually, that was easy and reliable:

interval:
  - interval: 30s
    then:
      - ble_client.ble_write: 
          characteristic_uuid: "FFD1"
          service_uuid: "FFD0"
          
          id: renogy_battery_esp32_bc
          value: !lambda |-
                    vector<uint8_t> request = GetBatteryRequest(0x30);
                    return request;

      - delay: 5s
      - ble_client.ble_write: 
          characteristic_uuid: "FFD1"
          service_uuid: "FFD0"
          
          id: renogy_battery_esp32_bc
          value: !lambda |-
                    vector<uint8_t> request = GetBatteryRequest(0x31);
                    return request;

      - delay: 5s
      - ble_client.ble_write: 
          characteristic_uuid: "FFD1"
          service_uuid: "FFD0"
          
          id: renogy_battery_esp32_bc
          value: !lambda |-
                    vector<uint8_t> request = GetBatteryRequest(0x32);
                    return request;                   

Now I just need to change the parsing logic to update sensors for the correct battery

[mavenius]

BTW, huge thanks to @mateuszdrab for getting this moving; without what you shared, I wouldn't have been anywhere near getting this working.

[mavenius]

Oh, snap:

sensor:
  # Renogy battery 48
  - platform: template
    name: "Renogy Battery 48 Current"
    id: renogy_battery_48_current
    device_class: current
    unit_of_measurement: A
    accuracy_decimals: 1

  - platform: template
    name: "Renogy Battery 48 Voltage"
    id: renogy_battery_48_voltage
    device_class: voltage
    unit_of_measurement: V
    accuracy_decimals: 1

  - platform: template
    name: "Renogy Battery 48 Present Capacity"
    id: renogy_battery_48_present_capacity
    unit_of_measurement: Ah
    accuracy_decimals: 1

  - platform: template
    name: "Renogy Battery 48 Total Capacity"
    id: renogy_battery_48_total_capacity
    unit_of_measurement: Ah
    accuracy_decimals: 1    

  - platform: template
    name: "Renogy Battery 48 Charge Level"
    id: renogy_battery_48_charge_level
    icon: mdi:percent
    unit_of_measurement: "%"
    accuracy_decimals: 1    

  # Renogy battery 49
  - platform: template
    name: "Renogy Battery 49 Current"
    id: renogy_battery_49_current
    device_class: current
    unit_of_measurement: A
    accuracy_decimals: 1

  - platform: template
    name: "Renogy Battery 49 Voltage"
    id: renogy_battery_49_voltage
    device_class: voltage
    unit_of_measurement: V
    accuracy_decimals: 1

  - platform: template
    name: "Renogy Battery 49 Present Capacity"
    id: renogy_battery_49_present_capacity
    unit_of_measurement: Ah
    accuracy_decimals: 1

  - platform: template
    name: "Renogy Battery 49 Total Capacity"
    id: renogy_battery_49_total_capacity
    unit_of_measurement: Ah
    accuracy_decimals: 1    

  - platform: template
    name: "Renogy Battery 49 Charge Level"
    id: renogy_battery_49_charge_level
    icon: mdi:percent
    unit_of_measurement: "%"
    accuracy_decimals: 1    

  # Renogy battery 50
  - platform: template
    name: "Renogy Battery 50 Current"
    id: renogy_battery_50_current
    device_class: current
    unit_of_measurement: A
    accuracy_decimals: 1

  - platform: template
    name: "Renogy Battery 50 Voltage"
    id: renogy_battery_50_voltage
    device_class: voltage
    unit_of_measurement: V
    accuracy_decimals: 1

  - platform: template
    name: "Renogy Battery 50 Present Capacity"
    id: renogy_battery_50_present_capacity
    unit_of_measurement: Ah
    accuracy_decimals: 1

  - platform: template
    name: "Renogy Battery 50 Total Capacity"
    id: renogy_battery_50_total_capacity
    unit_of_measurement: Ah
    accuracy_decimals: 1    

  - platform: template
    name: "Renogy Battery 50 Charge Level"
    id: renogy_battery_50_charge_level
    icon: mdi:percent
    unit_of_measurement: "%"
    accuracy_decimals: 1    

  - platform: ble_client
    ble_client_id: renogy_battery_esp32_bc
    id: renogy_battery_esp32_sensor
    internal: true
    type: characteristic
    service_uuid: FFF0
    characteristic_uuid: FFF1
    notify: true
    update_interval: never

    # on_notify: 
    #   then:
    #     - lambda: |-
    #         ESP_LOGD("ble_client.notify", "x: %.2f", x);

    lambda: |-      
      // A variable x of type esp32_ble_tracker::ESPBTDevice is passed to the automation for use in lambdas. (from docs)
      int receivedSize = x.size();
      ESP_LOGD("ble_client_lambda", "Received bytes size: %d", receivedSize);
      // Log each byte in the array
      for (size_t i = 0; i < receivedSize; ++i) {
        ESP_LOGD("main", "Response Byte %d: 0x%02X", i, x[i]);
      }
      if (receivedSize < 17) return NAN;
        
      HandleBatteryData(x);
      return 0.0; // this sensor isn't actually used other than to hook into raw value and publish to template sensors

interval:
  - interval: 30s
    then:
      - ble_client.ble_write: 
          characteristic_uuid: "FFD1"
          service_uuid: "FFD0"
          
          id: renogy_battery_esp32_bc
          value: !lambda |-
                    vector<uint8_t> request = GetBatteryRequest(48);
                    return request;

      - delay: 5s
      - ble_client.ble_write: 
          characteristic_uuid: "FFD1"
          service_uuid: "FFD0"
          
          id: renogy_battery_esp32_bc
          value: !lambda |-
                    vector<uint8_t> request = GetBatteryRequest(49);
                    return request;

      - delay: 5s
      - ble_client.ble_write: 
          characteristic_uuid: "FFD1"
          service_uuid: "FFD0"
          
          id: renogy_battery_esp32_bc
          value: !lambda |-
                    vector<uint8_t> request = GetBatteryRequest(50);
                    return request;    

void HandleBatteryData(vector<uint8_t> x) {
    uint8_t batteryId;
    std::memcpy(&batteryId, &x[0], sizeof(batteryId));
    ESP_LOGD("HandleBatteryData", "battery Id: %d", batteryId);
    
    // Parse the function
    uint8_t function;
    std::memcpy(&function, &x[1], sizeof(function));
    // function = ntohs(function); // Convert from network byte order to host byte order
    ESP_LOGD("HandleBatteryData", "function: %d", function);
    // Parse the current
    int16_t current;
    std::memcpy(&current, &x[3], sizeof(current));
    current = ntohs(current); // Convert from network byte order to host byte order
    ESP_LOGD("HandleBatteryData", "current: %d", current);
    // Parse the voltage
    uint16_t voltage;
    std::memcpy(&voltage, &x[5], 2);
    voltage = ntohs(voltage); // Convert from network byte order to host byte order
    ESP_LOGD("HandleBatteryData", "voltage: %d", voltage);
    // Parse the present capacity
    uint32_t presentCapacity;
    std::memcpy(&presentCapacity, &x[7], 4);
    presentCapacity = ntohl(presentCapacity); // Convert from network byte order to host byte order
    ESP_LOGD("HandleBatteryData", "presentCapacity: %d", presentCapacity);
    // Parse the total capacity
    uint32_t totalCapacity;
    std::memcpy(&totalCapacity, &x[11], 4);
    totalCapacity = ntohl(totalCapacity); // Convert from network byte order to host byte order
    ESP_LOGD("HandleBatteryData", "totalCapacity: %d", totalCapacity);
    // Convert the values to the appropriate units
    float currentFloat = static_cast<float>(current) / 100.0f;
    float voltageFloat = static_cast<float>(voltage) / 10.0f;
    float presentCapacityFloat = static_cast<float>(presentCapacity) / 1000.0f;
    float totalCapacityFloat = static_cast<float>(totalCapacity) / 1000.0f;
    float chargeLevelFloat = (presentCapacityFloat / totalCapacityFloat) * 100.0f; 

    ESP_LOGD("HandleBatteryData", "currentFloat: %.1f", currentFloat);
    ESP_LOGD("HandleBatteryData", "voltageFloat: %.1f", voltageFloat);
    ESP_LOGD("HandleBatteryData", "presentCapacityFloat: %.1f", presentCapacityFloat);
    ESP_LOGD("HandleBatteryData", "totalCapacityFloat: %.1f", totalCapacityFloat);
    ESP_LOGD("HandleBatteryData", "chargeLevelFloat: %.1f", chargeLevelFloat);

    // use battery_id to decide which to update
    switch (batteryId){
        case 48:
            id(renogy_battery_48_current).publish_state(currentFloat);
            id(renogy_battery_48_voltage).publish_state(voltageFloat);
            id(renogy_battery_48_present_capacity).publish_state(presentCapacityFloat);
            id(renogy_battery_48_total_capacity).publish_state(totalCapacityFloat);
            id(renogy_battery_48_charge_level).publish_state(chargeLevelFloat);
        break;
        case 49:
            id(renogy_battery_49_current).publish_state(currentFloat);
            id(renogy_battery_49_voltage).publish_state(voltageFloat);
            id(renogy_battery_49_present_capacity).publish_state(presentCapacityFloat);
            id(renogy_battery_49_total_capacity).publish_state(totalCapacityFloat);
            id(renogy_battery_49_charge_level).publish_state(chargeLevelFloat);
        break;
        case 50:
            id(renogy_battery_50_current).publish_state(currentFloat);
            id(renogy_battery_50_voltage).publish_state(voltageFloat);
            id(renogy_battery_50_present_capacity).publish_state(presentCapacityFloat);
            id(renogy_battery_50_total_capacity).publish_state(totalCapacityFloat);
            id(renogy_battery_50_charge_level).publish_state(chargeLevelFloat);
        break;

    }
}

BTW these C++ functions (GetBatteryRequest and HandleBatteryData) are in a file I called renogy_utilities.h and refereced in the esphome config like so:

esphome:
  includes: 
    - renogy_utilities.h

[mavenius]

As soon as I get some more esp32s, I'm going to configure one for pulling Rover data.

[mateuszdrab]

BTW, huge thanks to @mateuszdrab for getting this moving; without what you shared, I wouldn't have been anywhere near getting this working.

You're welcome, I'm happy to have been able to help you out - I've had this battery for a year and it's been bugging me constantly as I had power cuts due to my RCD tripping and wasn't able to get any insights into the battery.

Now I can finally react to this situation and shut the server down gracefully.

I think it might be better to have this code sanitized into an esphome component/library in its own repo for easy reusability.

However, my C++ knowledge is way rustier than yours 😂

Next, I need to try connecting to the renogy inverter charger which has an actual rs485 connector.

[cyrils]

@cyrils is this something you want as part of this project, or should it be spun off? It's obviously related (and heavily influenced) by what you've done, but it's also a big departure from what you have built.

@mavenius Feel free to create a new repo. Just give a link back, I'll also add a link to your repo.

[mavenius]

Sounds good; that's what I figured you'd say. I'll let you know when it's set up.

On Thu, Aug 8, 2024, 22:41 Cyril Sebastian @.***> wrote:

@cyrils https://github.com/cyrils is this something you want as part of this project, or should it be spun off? It's obviously related (and heavily influenced) by what you've done, but it's also a big departure from what you have built.

@mavenius https://github.com/mavenius Feel free to create a new repo. Just give a link back, I'll also add a link to your repo.

— Reply to this email directly, view it on GitHub https://github.com/cyrils/renogy-bt/issues/49#issuecomment-2277039654, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO65BGSPGZJ47IM5HHR2CLZQQT6RAVCNFSM6AAAAABLC3VGL2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZXGAZTSNRVGQ . You are receiving this because you were mentioned.Message ID: @.***>

[mavenius]

I spun off https://github.com/mavenius/renogy-bt-esphome and made some of the config generic. It has some more work to go to make it easier to use, but it's out there for you all to play with and/or PR into.

Thanks again for all that you all did to get this here; I truly couldn't have done it without you.

[mateuszdrab]

I have updated the code in my fork https://github.com/mateuszdrab/renogy-bt-esphome with support for obtaining sensor temperatures and cell voltages (though did not add sensors for temperatures)

I also updated the examples, but I have not tried the setup with multiple batteries as my yaml file is for a single battery and uses no prefixes in their naming. However, any changes I made should be backwards compatible.

PS. All those hours invested, and I still don't know why the battery is self-discharging at 1 Ah 😁