ahoy
ahoy copied to clipboard
lumapu
Discussion: Stand-alone Hoymiles lib / Support for RTL-SDR and SX127x/RFM9x chips
I'd like to integrate communications with my Hoymiles HMS more flexibly into other projects, i.e. to recv messages.
I'd also like to be able to do this using an RTL-SDR dongle.
It looks like it could also be possible to use an SX1276 (aka RFM95) for this, which supports LoRa, FSK, GFSK, MSK, GMSK and OOK. Even the SX1231/RFM69 might be supportable. Are there any obvious reasons these chips CANNOT be used that I'm missing?
Has anyone considered adding a Hoymiles HMS decoder to https://github.com/merbanan/rtl_433 ?
SX1276 enablement could then happen in https://github.com/NorthernMan54/rtl_433_ESP, which would make the decoder more widely usable in the Arduino/ESP32 ecosystem.
Also ref: https://github.com/tbnobody/OpenDTU/issues/1993 for the sibling issue on the OpenDTU repo
Sorry I don't know the mentioned radio modules. We use the same module as Hoymiles uses in their DTUs.
This is a snipped of the configuration of the CMT2300A radio module:
click to expand
;---------------------------------------
; CMT2300A Configuration File
; Generated by CMOSTEK RFPDK 1.53_Update3
; 2023.03.24 17:29
;---------------------------------------
; Mode = Advanced
; Part Number = CMT2300A
; Frequency = 860.000 MHz
; Xtal Frequency = 26.0000 MHz
; Demodulation = GFSK
; AGC = On
; Data Rate = 20.0 kbps
; Deviation = 20.0 kHz
; Tx Xtal Tol. = 19 ppm
; Rx Xtal Tol. = 20 ppm
; TRx Matching Network Type = 20 dBm
; Tx Power = +20 dBm
; Gaussian BT = 0.5
; Bandwidth = Auto-Select kHz
; CDR Type = Counting
; CDR DR Range = NA
; AFC = On
; AFC Method = Auto-Select
; Data Representation = 0:F-low 1:F-high
; Rx Duty-Cycle = Off
; Tx Duty-Cycle = Off
; Sleep Timer = Off
; Sleep Time = NA
; Rx Timer = Off
; Rx Time T1 = NA
; Rx Time T2 = NA
; Rx Exit State = STBY
; Tx Exit State = STBY
; SLP Mode = Disable
; RSSI Valid Source = PJD
; PJD Window = 8 Jumps
; LFOSC Calibration = On
; Xtal Stable Time = 155 us
; RSSI Compare TH = NA
; Data Mode = Packet
; Whitening = Disable
; Whiten Type = NA
; Whiten Seed Type = NA
; Whiten Seed = NA
; Manchester = Disable
; Manchester Type = NA
; FEC = Enable
; FEC Type = x^3+x^2+1
; Tx Prefix Type = 0
; Tx Packet Number = 1
; Tx Packet Gap = 32
; Packet Type = Variable Length
; Node-Length Position = First Node, then Length
; Payload Bit Order = Start from msb
; Preamble Rx Size = 2
; Preamble Tx Size = 30
; Preamble Value = 170
; Preamble Unit = 8-bit
; Sync Size = 4-byte
; Sync Value = 1296587336
; Sync Tolerance = None
; Sync Manchester = Disable
; Node ID Size = NA
; Node ID Value = NA
; Node ID Mode = None
; Node ID Err Mask = Disable
; Node ID Free = Disable
; Payload Length = 32
; CRC Options = IBM-16
; CRC Seed = 0 crc_seed
; CRC Range = Entire Payload
; CRC Swap = Start from MSB
; CRC Bit Invert = Normal
; CRC Bit Order = Start from bit 15
; Dout Mute = Off
; Dout Adjust Mode = Disable
; Dout Adjust Percentage = NA
; Collision Detect = Off
; Collision Detect Offset = NA
; RSSI Detect Mode = At PREAM_OK
; RSSI Filter Setting = 32-tap
; RF Performance = High
; LBD Threshold = 2.4 V
; RSSI Offset = 26
; RSSI Offset Sign = 0
As you know such a request is not our scope, so you have to do the research by you own.
@lumapu thank you for your reply (and for creating this repo!)! And yes, this may obviously be out of scope, so feel free to close too. I'll continue to reference this issue for discoverability nonetheless :)
@LorbusChris the original MI- & HM-models used to employ the NRF24L01+ chips for radio communication because they were using the so called Enhanced Shockburst (ESB) protocol. So there were no other chips available that would allow us to send and receive packets via ESB, that is why we used the same modules as in the Hoymiles hardware (DTU and Inverter).
Same with the HMS modules we also considered RFM95 initially, but then quickly settled on the CMT2300A because it is the same chip as in the Hoymiles DTU Pro/Lite S models, which are used with the HMS-/HMT-models. But I think there is even lesser reasons why this should not work with any other model / even with an RTL-SDR encoder/decoder virtual model.
Did you already achieve some success with the details from the CMT2300A initialization that lumapu shared with you ?
@stefan123t thanks for that context!
In the meantime, https://github.com/esphome/esphome/pull/7490 has merged into esphome, making the following now possible:
esphome:
name: "hoymiles-gateway"
friendly_name: Hoymiles Inverter Gateway
esp32:
board: adafruit_feather_esp32_v2
framework:
type: arduino
spi:
clk_pin: GPIO5
mosi_pin: GPIO19
miso_pin: GPIO21
sx127x:
dio0_pin: GPIO27 # IRQ
cs_pin: GPIO33
rst_pin: GPIO15
frequency: 865000000
modulation: FSK
shaping: GAUSSIAN_BT_0_5
bitrate: 20000
deviation: 20000
packet_mode: true
#crc_enable: true
rx_floor: -90
rx_start: true
bitsync: true
sync_value: [0x4D, 0x48, 0x5A, 0x48]
preamble_size: 30
preamble_detect: 2
preamble_errors: 8
preamble_polarity: 0xAA
payload_length: 0
on_packet:
then:
- lambda: |-
ESP_LOGD("lambda", "packet %s", format_hex(x).c_str());
# Enable logging
logger:
# Enable Home Assistant API
api:
encryption:
key: !secret api_encryption_key
ota:
- platform: esphome
password: !secret ota_password
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
which yields (likely from both my OpenDTU and HMS-800-2T):
[05:47:19][D][lambda:040]: packet 706ae8c69cb42e8cb9a8c006cb00d00468c0005c0001a46697119a
[05:47:19][D][lambda:040]: packet 725ae8c69cb42e8cb9a8c006cb00d00460034000003400dd12c00d
[05:47:19][D][lambda:040]: packet 725ae8c69cb42e8cb9a8c006cb00d0046005f94b8c8c00000d1cd1
[05:47:19][D][lambda:040]: packet e65ae8c69cb42e8cb9a8c006cb00d00468c68000000000000680
Without digging into the code too much, is the protocol/data structure documented anywhere? https://github.com/lumapu/ahoy/wiki/Protocol seems to be mostly about HM, not HMS.
I'd appreciate any pointers :)
The data received does look very different from what OpenDTU reports:
18:50:57.710 > Fetch inverter: 114483762423
18:50:57.727 > TX RealTimeRunData 865.00 MHz --> 15 83 76 24 23 80 19 06 08 80 0B 00 68 6D 4C 71 00 00 00 00 00 00 00 00 81 AF ED
18:50:57.830 > Interrupt received
18:50:57.836 > RX 865.00 MHz --> 95 83 76 24 23 80 19 06 08 01 00 01 01 F9 01 E9 00 28 00 28 00 CA 00 C5 00 05 EA | -70 dBm
18:50:57.840 > Interrupt received
18:50:57.941 > RX 865.00 MHz --> 95 83 76 24 23 80 19 06 08 02 EC A4 00 06 7D D4 02 50 02 4E 09 30 13 8A 01 7A D0 | -70 dBm
18:50:57.947 > Interrupt received
18:50:57.951 > RX 865.00 MHz --> 95 83 76 24 23 80 19 06 08 83 00 00 00 10 03 E8 00 DB 00 0A 95 61 AD | -70 dBm
18:50:58.227 > RX Period End
18:50:58.227 > Success
Do I need to decode the data in some way? FEC?
@LorbusChris I see you already linked your Feature Request on ESPHome side https://github.com/esphome/feature-requests/issues/3203 This looks very promising indeed!
The protocol description is mostly on the above Wiki page https://github.com/lumapu/ahoy/wiki/Protocol.
@tbnobody has added the documentation directly into the OpenDTU Code, e.g. here the most common / frequent RealTimeRunData Command 0x15 MainCmd and 0x0b SubCmd for starters: https://github.com/tbnobody/OpenDTU/blob/master/lib/Hoymiles/src/commands/RealTimeRunDataCommand.cpp
You may want to compare the Wiki with the initial attempts to document the Gen2 (basic start/stop commands and mostly single Channel requests for MI-inverters) / Gen3 Protocol (mostly multi-frame commands for HM- 433MHz/NRF24L01+ based, HMS-/HMT-inverters 865MHz/CMT2300A based). https://github.com/lumapu/ahoy/blob/main/doc/hoymiles-format-description.md I have started to format this using WaveDrom for readability / diagrams and you may prefer the PDF version of the Markdown instead of the Github rendering. Though this is still work in progress, but I have too little time to devout to it 🥹
And last but not least the initial, groundbreaking and somewhat lenghty Microcontroller Forum Thread which started the whole: https://www.mikrocontroller.net/topic/525778
For reversing some of the commands we used the Hoymiles DTU Pro code drop which occurred on gitee.com:
https://gitee.com/iotloves/hoymiles-DTU-PRO
Give it a search on our Discord if you want to take a look at the main classes usart_nrf.c/h (Gen2) / usart_nrf3.c/h (Gen3)
for which we machine-translated all the comments from Chinese to English ;)
For other commands we had some users listen in to the communication between the GD32 mcu and the NRF24L01+ / CMT2300A rc chips by tucking in a USART interface or two on the RX/TX lines. See https://github.com/tbnobody/OpenDTU/issues/1650 for the setup by @broth-itk who has been very helpful in creating traces e.g. for Grid Profile and Firmware Updates.