core icon indicating copy to clipboard operation
core copied to clipboard

ABB photovoltaic "status unavailable"

Open Yosha89 opened this issue 2 years ago • 27 comments

The problem

After some days working good the data fixed in a value e don't change at all. The serial connection is ok properly connected. The only way is to reboot HA system host for solve. This happened always with every version of HA

What version of Home Assistant Core has the issue?

9.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ABB photovoltaic

Link to integration documentation on our website

https://www.home-assistant.io/integrations/aurora_abb_powerone/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

Yosha89 avatar Nov 30 '22 12:11 Yosha89

Hey there @davet2001, mind taking a look at this issue as it has been labeled with an integration (aurora_abb_powerone) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of aurora_abb_powerone can trigger bot actions by commenting:

  • @home-assistant close Closes the issue.
  • @home-assistant rename Awesome new title Change the title of the issue.
  • @home-assistant reopen Reopen the issue.
  • @home-assistant unassign aurora_abb_powerone Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


aurora_abb_powerone documentation aurora_abb_powerone source (message by IssueLinks)

home-assistant[bot] avatar Nov 30 '22 12:11 home-assistant[bot]

@Yosha89 Please could you paste any log messages from the time when the communication first stops working?

There is a PR in progress to improve the communications (#72363) which could help with this because it reduces the amount of stop/start cycles of the RS485 interface.

Its possible (but a bit of a guess) that the separate sensor entities are trying to access the RS485 interface before it's fully ready, which could lead to it getting into an unusable state.

But in any case we need to know what the error is. If the number is not updating, there should be at least a timeout exception somewhere in the logs.

davet2001 avatar Nov 30 '22 12:11 davet2001

No such file or directory: '/dev/ttyUSB0' Means that the OS thinks the com port doesn't exist any more. If that is the case, HA core can't do anything about it.

Regarding multiple access, sorry if I caused confusion, I meant that in the current release, each sensor access is independent within HA core and there is no particular synchronisation. (Not that you have more than one inverter).

I think the PR mentioned should be merged before further debugging because that eliminates a possible cause.

Are you in a position to try out the PR loaded as a custom component?

davet2001 avatar Dec 01 '22 07:12 davet2001

@Yosha89 Since you are running on virtual box, in the failed state is the com port still visible :

  • on the host OS?
  • on the guest OS?

You can test the PR by downloading this url: https://github.com/davet2001/home-assistant/tree/aurora_dataupdatecoordinator/homeassistant/components/aurora_abb_powerone

Maybe this tool works if you just want a zip to download: https://minhaskamal.github.io/DownGit/#/home?url=https:%2F%2Fgithub.com%2Fdavet2001%2Fhome-assistant%2Ftree%2Faurora_dataupdatecoordinator%2Fhomeassistant%2Fcomponents%2Faurora_abb_powerone

You then have to copy the contents into config/custom_components/aurora_abb_powerone then restart your HA instance.

HA will use the added aurora_abb_powerone instead of the standard integration. You can revert this by deleting it out of the custom_components directory. The PR makes no changes to config so it's fully reversible.

davet2001 avatar Dec 01 '22 12:12 davet2001

Everything should work like it did before. Just keep using it and see if the problem occurs. Thanks for trying it!

davet2001 avatar Dec 03 '22 17:12 davet2001

Are you sure that the inverter is still running? In low power it shuts down, so could be borderline. You can check by looking at the front panel.

davet2001 avatar Dec 17 '22 10:12 davet2001

i have the exact same problem. From today morning no response from the ABB integration, i already tried with reboot without success. I have enabled the debug mode for the logs but i don't think it can help much, because devs need the debug before the error. Now it only says : no response, it cloud be dark

xm4rcell0x avatar Jan 16 '23 11:01 xm4rcell0x

Thanks @Yosha89 and @xm4rcell0x for the information on this.

@xm4rcell0x you may need to fully power off your usb-serial adapter by unplugging and replugging it, then rebooting HA. In theory there is nothing in the HA state that can perpetuate serial comms failure through a reboot, but it's possible the adapter itself could be in an unknown state.

Looking at the logs, I think the problems start here:

2022-12-23 07:43:52.701 WARNING (SyncWorker_11) [homeassistant.components.aurora_abb_powerone.sensor] Communication with Total Energy lost
2022-12-23 07:43:52.711 ERROR (MainThread) [homeassistant.helpers.entity] Update for sensor.total_energy fails
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 592, in send_and_recv
    response += self.serline.readline(8 - len(response))
  File "/usr/local/lib/python3.10/site-packages/serial/serialposix.py", line 595, in read
    raise SerialException(
serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

Later on it complains that the port does not exist. Online there are several other cases of this, but hard to be sure what the real fault is. Possible related info: https://forum.dexterindustries.com/t/errno-5-input-output-error-when-trying-to-use-motors/2110 https://forums.raspberrypi.com/viewtopic.php?t=228315 https://github.com/pyserial/pyserial/issues/401

I have an idea that this is related to repeatedly opening and closing the serial port, which in theory isn't necessary. I am working on a branch to see if it helps: https://github.com/davet2001/home-assistant/tree/aurora_keep_tty_open This is now running locally - others are welcome to try it, but please bear in mind two caveats:

  • I don't know if this works at all as it's currently dark in my location (will know in the next day).
  • I don't know if it fixes anything, (hard to tell because I haven't been experiencing the same problems).

davet2001 avatar Jan 17 '23 23:01 davet2001

Update: The patch worked today - readings all ok. If you are willing to test it, please copy the aurora_abb_powerone directory from the above repo into custom_components, adding a version key e.g. “version”: “1” to manifest.json to make the file valid, then restart HA to try it.

davet2001 avatar Jan 18 '23 22:01 davet2001

it works!

xm4rcell0x avatar Jan 20 '23 11:01 xm4rcell0x

Worked for me yesterday, but today stopped, although that could have been a different problem I was dealing with. Please keep running with the patched one for as long as you can to see if it's really fixed.

davet2001 avatar Jan 20 '23 16:01 davet2001

Thanks! This is the relevant part of the log:

2023-03-08 06:36:47.069 ERROR (MainThread) [custom_components.aurora_abb_powerone] Unexpected error fetching aurora_abb_powerone data: device reports readiness to read but returned no data (device disconnected or multiple access on port?)
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 592, in send_and_recv
    response += self.serline.readline(8 - len(response))
  File "/usr/local/lib/python3.10/site-packages/serial/serialposix.py", line 595, in read
    raise SerialException(
serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

After that it never recovers.

It appears that this happens quite early in the day. Maybe it is the first reading of the day, and the inverter is sometimes only partially awake (e.g. corrupted or incomplete response that confuses communications).

If so a possible solution would be to retry up to n times on a SerialException with a delay between each attempt.

Please could you restart the integration (or HA) to get it working again, and continue to monitor? If it's repeatedly the same thing I can create something to work around it.

In the mean time it's been working fine for me since my last comment.

davet2001 avatar Mar 08 '23 12:03 davet2001

@Yosha89 Thanks for quick follow up. For info, the first failure is at the same point:

2023-03-09 06:40:44.331 ERROR (MainThread) [custom_components.aurora_abb_powerone] Unexpected error fetching aurora_abb_powerone data: device reports readiness to read but returned no data (device disconnected or multiple access on port?)
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 592, in send_and_recv
    response += self.serline.readline(8 - len(response))
  File "/usr/local/lib/python3.10/site-packages/serial/serialposix.py", line 595, in read
    raise SerialException(
serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

This happened just 3 minutes later in the day compared with yesterday. I tend to think this is an intermittent startup issue of the inverter, in which case I will add a catch for SerialException as described above.

davet2001 avatar Mar 09 '23 12:03 davet2001

I have some time later this week. Will try to set something up to test.

davet2001 avatar Apr 04 '23 09:04 davet2001

@Yosha89 I have created a new branch with a retry loop in it.

https://github.com/davet2001/home-assistant/tree/aurora_dataupdatecoordinator_serialexception

Please can you try it and see if this resolves the issue?

davet2001 avatar Apr 06 '23 14:04 davet2001

Hi, I had problems after upgrading to Home Assistant 2023.5.1, up to version 2023.4.6 it worked perfectly: image

aurora_abb_powerone: Error on device update! 10:13:20 – (ERROR) Sensor aurora_abb_powerone: Error on device update! 10:13:20 – (ERROR) Sensor Communication with Temperature lost 10:13:20 – (WARNING) Aurora ABB PowerOne Solar PV - message first occurred at 10:10:24 and shows up 4 times Update for sensor.total_energy fails 10:12:54 – (ERROR) components/aurora_abb_powerone/sensor.py - message first occurred at 10:11:24 and shows up 3 times Update for sensor.temperature fails 10:12:54 – (ERROR) components/aurora_abb_powerone/sensor.py - message first occurred at 10:11:24 and shows up 3 times Update for sensor.power_output fails 10:12:54 – (ERROR) components/aurora_abb_powerone/sensor.py - message first occurred at 10:11:54 and shows up 2 times Update for sensor.total_energy fails 10:11:54 – (ERROR) components/aurora_abb_powerone/sensor.py Update for sensor.temperature fails 10:11:54 – (ERROR) components/aurora_abb_powerone/sensor.py Update for sensor.power_output fails 10:11:24 – (ERROR) components/aurora_abb_powerone/sensor.py - message first occurred at 10:10:58 and shows up 2 times

Logger: homeassistant.components.sensor Source: components/aurora_abb_powerone/sensor.py:105 Integration: Sensor (documentation, issues) First occurred: 10:13:20 (1 occurrences) Last logged: 10:13:20

aurora_abb_powerone: Error on device update! Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 521, in _async_add_entity await entity.async_device_update(warning=False) File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 784, in async_device_update await coro File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/usr/src/homeassistant/homeassistant/components/aurora_abb_powerone/sensor.py", line 105, in update energy_wh = self.client.cumulated_energy(5) File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 392, in cumulated_energy response = self.send_and_recv(request) File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 592, in send_and_recv response += self.serline.readline(8 - len(response)) File "/usr/local/lib/python3.10/site-packages/serial/serialposix.py", line 575, in read buf = os.read(self.fd, size - len(read)) TypeError: 'NoneType' object cannot be interpreted as an integer

aurora_abb_powerone: Error on device update! Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 521, in _async_add_entity await entity.async_device_update(warning=False) File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 784, in async_device_update await coro File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/usr/src/homeassistant/homeassistant/components/aurora_abb_powerone/sensor.py", line 102, in update temperature_c = self.client.measure(21) File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 221, in measure response = self.send_and_recv(request) File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 592, in send_and_recv response += self.serline.readline(8 - len(response)) File "/usr/local/lib/python3.10/site-packages/serial/serialposix.py", line 575, in read buf = os.read(self.fd, size - len(read)) TypeError: 'NoneType' object cannot be interpreted as an integer

Update for sensor.total_energy fails Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 592, in send_and_recv response += self.serline.readline(8 - len(response)) File "/usr/local/lib/python3.10/site-packages/serial/serialposix.py", line 595, in read raise SerialException( serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 554, in async_update_ha_state await self.async_device_update() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 784, in async_device_update await coro File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/usr/src/homeassistant/homeassistant/components/aurora_abb_powerone/sensor.py", line 118, in update raise error File "/usr/src/homeassistant/homeassistant/components/aurora_abb_powerone/sensor.py", line 105, in update energy_wh = self.client.cumulated_energy(5) File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 392, in cumulated_energy response = self.send_and_recv(request) File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 595, in send_and_recv raise AuroraError(str(e)) aurorapy.client.AuroraError: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

Update for sensor.total_energy fails Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 554, in async_update_ha_state await self.async_device_update() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 784, in async_device_update await coro File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/usr/src/homeassistant/homeassistant/components/aurora_abb_powerone/sensor.py", line 105, in update energy_wh = self.client.cumulated_energy(5) File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 392, in cumulated_energy response = self.send_and_recv(request) File "/usr/local/lib/python3.10/site-packages/aurorapy/client.py", line 592, in send_and_recv response += self.serline.readline(8 - len(response)) File "/usr/local/lib/python3.10/site-packages/serial/serialposix.py", line 575, in read buf = os.read(self.fd, size - len(read)) TypeError: 'NoneType' object cannot be interpreted as an integer

gpbicego avatar May 05 '23 10:05 gpbicego

@gpbicego @Yosha89 are either of you able to try the branch above?

@gpbicego There haven't been any changes to the integration since the last release so I don't know what would make it worse.

davet2001 avatar May 06 '23 05:05 davet2001

@davet2001 I tried the branch above but it doesn't work. I think some changes in "core 2023.5" broke this integration. I've never had a problem with it before except that after a system reboot the usb port changes because you can't use "Persistent USB names". Now I want to try restore Home Assistant to version "core 2023.4.6".

gpbicego avatar May 06 '23 10:05 gpbicego

Yes, this is a limitation of the pyserial library. I have an open PR there to fix this:

https://github.com/pyserial/pyserial/pull/700

At the moment if you have more than one usb serial device, assignment depends on how the OS handles this during startup.

For now you could insert all devices then do a full power cycle to (perhaps) get a repeatable assignment.

davet2001 avatar May 06 '23 10:05 davet2001

I confirm that Home Assistant version "core 2023.4.6" works

gpbicego avatar May 06 '23 12:05 gpbicego

@Yosha89 short term: delete or rename the custom integration folder. Behaviour will then revert to what it was before the experimental change.

davet2001 avatar May 07 '23 14:05 davet2001

I suggest:

Shut down HA Unplug usb adaptor Replug usb adaptor Start HA

davet2001 avatar May 07 '23 17:05 davet2001

@Yosha89 I have updated the branch and tested it on my local setup - it works ok for me now: https://github.com/davet2001/home-assistant/tree/aurora_dataupdatecoordinator_serialexception

If you find it still shows as unavailable, it could be that the serial port has changed. In that case, please delete and re-add the integration via the UI. Your historic data will not be affected.

If you still get 'unavailable' after running the branch above, paste the log output back here.

davet2001 avatar May 08 '23 09:05 davet2001

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

Keep open, need to merge #72363 first.

Then we can merge a PR based on the above improvement, even if not a confirmed complete fix.

davet2001 avatar Oct 26 '23 20:10 davet2001

@Yosha89 I don't 100% know why this occurs. I have had this happen but it is extremely rare (maybe 3-4 times all year).

One thing that I have seen happen is that on disconnection/reconnection, the USB port gets re-enumerated, this can result in it changing and not matching the config.

I have a theory that something inside the USB adaptor locks up that is only resolved by a power cycle (unplug/replug). But I have not been able to check this.

davet2001 avatar Nov 04 '23 20:11 davet2001

Is there any chance to make this work with any wireless module or esphome?

incarvr6 avatar Nov 04 '23 21:11 incarvr6

@incarvr6 Yes, kind of. The physical layer of the inverter is RS485, so the interfacing device would need to support that.

There is a modbus(?)/TCP version available as a custom component and I would like to get that capability included. That would potentially be a good basis to build an ESP interfacing device around.

PSA: Please be extremely careful if attempting any DIY interfaces. Inverters have higher than mains voltages, big capacitors and live components during daylight.

But we need to get the open PRs merged to enable DataUpdateCoordinator support before changing anything else.

davet2001 avatar Nov 05 '23 05:11 davet2001

Sorry, I've now checked, it's serial over RS485 but not modbus.

davet2001 avatar Nov 05 '23 23:11 davet2001

Hello, sorry to answer in that topic, because it's not possible to create a new issue. I'm using Homeassistant on a VM (Proxmox) and RS485 LAN instead of your mentioned RS485 USB adapter. How can I connect to them? grafik

Additionally I have only the choice to select 2 as the WR address. I'm using 1, which is defined as a standard address, I believe.

Thanks. BR, Klaus

KlausV66 avatar Dec 10 '23 14:12 KlausV66