jasonacox
pyPowerwall logs repeated errors [proxy] [ERROR] Socket broken sending response [doGET]
Unknown trigger causes errors to start being logged in in pyPowerwall:
11/16/2022 12:00:15 AM [proxy] [ERROR] Socket broken sending response [doGET]
----------------------------------------
Exception happened during processing of request from ('172.18.0.2', 38078)
Traceback (most recent call last):
File "/usr/local/lib/python3.8/socketserver.py", line 683, in process_request_thread
self.finish_request(request, client_address)
File "/usr/local/lib/python3.8/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/lib/python3.8/socketserver.py", line 747, in __init__
self.handle()
File "/usr/local/lib/python3.8/http/server.py", line 434, in handle
self.handle_one_request()
File "/usr/local/lib/python3.8/http/server.py", line 422, in handle_one_request
method()
File "server.py", line 195, in do_GET
for device in vitals:
TypeError: 'NoneType' object is not iterable
----------------------------------------
----------------------------------------
Exception happened during processing of request from ('172.18.0.2', 38080)
Traceback (most recent call last):
File "/usr/local/lib/python3.8/socketserver.py", line 683, in process_request_thread
self.finish_request(request, client_address)
File "/usr/local/lib/python3.8/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/lib/python3.8/socketserver.py", line 747, in __init__
self.handle()
File "/usr/local/lib/python3.8/http/server.py", line 434, in handle
self.handle_one_request()
File "/usr/local/lib/python3.8/http/server.py", line 422, in handle_one_request
method()
File "server.py", line 216, in do_GET
for device in vitals:
TypeError: 'NoneType' object is not iterable
----------------------------------------
11/16/2022 12:00:20 AM [proxy] [ERROR] Socket broken sending response [doGET]
11/16/2022 12:00:20 AM [proxy] [ERROR] Socket broken sending response [doGET]
11/16/2022 12:00:20 AM [proxy] [ERROR] Socket broken sending response [doGET]
Whilst logged, Powerwall web gui is responsive, and pypowerwall scan finds Powerwall:
/app $ python3 -m pypowerwall scan
pyPowerwall Network Scanner [0.6.0]
Scan local network for Tesla Powerwall Gateways
Your network appears to be: 172.18.0.0/24
Enter Network or press enter to use 172.18.0.0/24: 192.168.1.0/24
Running Scan...
Host: 192.168.1.1 ... OPEN - Not a Powerwall
Host: 192.168.1.30 ... OPEN - Found Powerwall nnnnnnn-01-F--T17Hnnnnnnnn
[Firmware 22.26.2 8cd8cac4]
Done
Discovered 1 Powerwall Gateway
192.168.1.30 [nnnnnnn-01-F--T17Hnnnnnnnn] Firmware 22.26.2 8cd8cac4
/app $ python3 -m pypowerwall scan
pyPowerwall Network Scanner [0.6.0]
Scan local network for Tesla Powerwall Gateways
Your network appears to be: 172.18.0.0/24
Enter Network or press enter to use 172.18.0.0/24:
Running Scan...
Done
Discovered 0 Powerwall Gateway
/app $
While attempting to debug, have restarted docker. Problem still re-occurs - pyPowerwall starts, telegraf and and grafana are stuck in a restarting loop.
As requested, the following pypowerwall URLs report: http://localhost:8675/stats
{"pypowerwall": "0.6.0 Proxy t19", "gets": 0, "errors": 0, "timeout": 0, "uri": {}, "ts": 1668519546, "start": 1668519392, "clear": 1668519392, "uptime": "0:02:34", "mem": 27984} and http://localhost:8675/vitals
{
"NEURIO--OBBnnnnnnnnnn": {
"NEURIO_CT0_InstRealPower": 29.3700008392334,
"NEURIO_CT0_Location": "site",
"NEURIO_CT3_InstRealPower": -4.079999923706055,
"NEURIO_CT3_Location": "solar",
"componentParentDin": "STSTSM--nnnnnnn-01-F--T17Hnnnnnnn",
"firmwareVersion": "1.7.2-Tesla",
"lastCommunicationTime": 1668519560,
"manufacturer": "NEURIO",
"meterAttributes": {
"meterLocation": [
1,
4
]
},
"serialNumber": "OBBnnnnnnnnnn"
},
"STSTSM--nnnnnnn-01-F--T17Hnnnnnnn": {
"STSTSM-Location": "Gateway",
"alerts": [
"GridCodesWrite",
"FWUpdateSucceeded",
"PodCommissionTime",
"SystemConnectedToGrid"
],
"firmwareVersion": "2022-10-24-g64e8c689f9",
"lastCommunicationTime": 1668519560,
"manufacturer": "TESLA",
"partNumber": "nnnnnnn-01-F",
"serialNumber": "T17Hnnnnnnn",
"teslaEnergyEcuAttributes": {
"ecuType": 207
}
},
"TEPINV--nnnnnnn-79-J--T18Annnnnnn": {
"PINV_EnergyCharged": 8435040.0,
"PINV_EnergyDischarged": 7454520.0,
"PINV_Fout": 50.009,
"PINV_GridState": "Grid_Compliant",
"PINV_HardwareEnableLine": true,
"PINV_PllFrequency": 49.995000000000005,
"PINV_PllLocked": true,
"PINV_Pout": 0.45,
"PINV_PowerLimiter": "PWRLIM_No_Power_Limit",
"PINV_Qout": 0.0,
"PINV_ReadyForGridForming": true,
"PINV_State": "PINV_GridFollowing",
"PINV_VSplit1": 121.5,
"PINV_VSplit2": 123.4,
"PINV_Vout": 244.9,
"componentParentDin": "TETHC--nnnnnnn-03-G--TGnnnnnnnTN",
"firmwareVersion": "8cd8cac4b9cb48",
"lastCommunicationTime": 1668519560,
"manufacturer": "TESLA",
"partNumber": "nnnnnnn-79-J",
"serialNumber": "T18Annnnnnn",
"teslaEnergyEcuAttributes": {
"ecuType": 253
}
},
"TEPOD--nnnnnnn-79-J--T18Annnnnnn": {
"POD_ActiveHeating": false,
"POD_CCVhold": false,
"POD_ChargeComplete": false,
"POD_ChargeRequest": false,
"POD_DischargeComplete": false,
"POD_PermanentlyFaulted": false,
"POD_PersistentlyFaulted": false,
"POD_available_charge_power": 7000.0,
"POD_available_dischg_power": 7000.0,
"POD_enable_line": true,
"POD_nom_energy_remaining": 5223.0,
"POD_nom_energy_to_be_charged": 8866.0,
"POD_nom_full_pack_energy": 13488.0,
"POD_state": "POD_ACTIVE",
"alerts": [
"POD_w109_SW_Self_Test_Request_Not_Serviced"
],
"componentParentDin": "TETHC--nnnnnnn-03-G--TGnnnnnnnTN",
"firmwareVersion": "8cd8cac4b9cb48",
"lastCommunicationTime": 1668519560,
"manufacturer": "TESLA",
"partNumber": "nnnnnnn-79-J",
"serialNumber": "T18Annnnnnn",
"teslaEnergyEcuAttributes": {
"ecuType": 226
}
},
"TESYNC--nnnnnnn-01-F--T17Hnnnnnnn": {
"ISLAND_FreqL1_Load": 50.01,
"ISLAND_FreqL1_Main": 50.01,
"ISLAND_FreqL2_Load": 60.0,
"ISLAND_FreqL2_Main": 60.0,
"ISLAND_FreqL3_Load": 0.0,
"ISLAND_FreqL3_Main": 0.0,
"ISLAND_GridConnected": true,
"ISLAND_GridState": "ISLAND_GridState_Grid_Compliant",
"ISLAND_L1L2PhaseDelta": -256.0,
"ISLAND_L1L3PhaseDelta": -256.0,
"ISLAND_L1MicrogridOk": true,
"ISLAND_L2L3PhaseDelta": -256.0,
"ISLAND_L2MicrogridOk": false,
"ISLAND_L3MicrogridOk": false,
"ISLAND_PhaseL1_Main_Load": 0.0,
"ISLAND_PhaseL2_Main_Load": -13.0,
"ISLAND_PhaseL3_Main_Load": -256.0,
"ISLAND_ReadyForSynchronization": true,
"ISLAND_VL1N_Load": 240.5,
"ISLAND_VL1N_Main": 240.5,
"ISLAND_VL2N_Load": 0.0,
"ISLAND_VL2N_Main": 0.0,
"ISLAND_VL3N_Load": 0.0,
"ISLAND_VL3N_Main": 0.0,
"SYNC_ExternallyPowered": false,
"SYNC_SiteSwitchEnabled": true,
"alerts": [
"SYNC_a044_IslanderDisconnectWithin2s"
],
"componentParentDin": "STSTSM--nnnnnnn-01-F--T17Hnnnnnnn",
"firmwareVersion": "8cd8cac4b9cb48",
"lastCommunicationTime": 1668519560,
"manufacturer": "TESLA",
"partNumber": "nnnnnnn-01-F",
"serialNumber": "T17Hnnnnnnn",
"teslaEnergyEcuAttributes": {
"ecuType": 259
}
},
"TETHC--nnnnnnn-03-G--TGnnnnnnnTN": {
"THC_AmbientTemp": 14.5,
"THC_State": "THC_STATE_AUTONOMOUSCONTROL",
"componentParentDin": "STSTSM--nnnnnnn-01-F--T17Hnnnnnnn",
"firmwareVersion": "8cd8cac4b9cb48",
"lastCommunicationTime": 1668519560,
"manufacturer": "TESLA",
"partNumber": "1092170-03-G",
"serialNumber": "TGnnnnnnnTN",
"teslaEnergyEcuAttributes": {
"ecuType": 224
}
}
}
Problem still re-occurs - pyPowerwall starts, telegraf and and grafana are stuck in a restarting loop.
This actually points to a problem that may not be pypowerwall. Grafana is not dependent upon pypowerwall but it does require influxdb. Can you pull the logs for influxdb, telegraf and grafana?
docker logs influxdb
docker logs telegraf
docker logs grafana
The log errors you see from pypowerwall indicate a problem with clients talking to it (the doGET part) which would likely indicate a problem with telegraf crashing when trying to pull data from pypowerwall. That does not mean it has a problem talking to your Powerwall which is why you are getting successful results hitting pypowerwall endpoints.
Without more information, I would dig into the logs of the other services to see if it points to a problem. I would also suspect something odd happening on you docker host. What system are you running?
What system are you running?
Ubuntu 22.04.1 LTS (GNU/Linux 5.10.102.1-microsoft-standard-WSL2 x86_64)
I'll work out tomorrow (late in the evening now, and I had a couple of nice 9% NEIPAs tonight) how to truncate the logs so I can start fresh so I can capture this when it next occurs.
Nice!! :)
You can reset the logs by removing and restarting containers:
docker stop pypowerwall influxdb telegraf grafana
docker rm pypowerwall influxdb telegraf grafana
./compose-dash.sh up -d