yahoofinance icon indicating copy to clipboard operation
yahoofinance copied to clipboard

Published 20 hours ago • verified publisher icon iprak

The sensors became unavailable.

Open Buakao opened this issue 2 years ago • 43 comments

Hi,

First of all, Thanks for creating this project. I was able to get it up and running for a month (I think) but as of last week, all the sensor shows unavailable. Any advice is appreciated.

Buakao avatar Jan 28 '22 06:01 Buakao

The last release was on Dec 25. Can you search for entities with yahoofinance?

Anything interesting in log? You can enable debug logging as following, restart will be requrieduired.

logger:
  default: info
  logs:
    custom_components.yahoofinance: debug

iprak avatar Jan 28 '22 10:01 iprak

Thanks for quick reply. I will try debugging and let you know. Btw, it has been unavailable just last 2 days. Jan 26 is the last day that the sensor entities were working.

Buakao avatar Jan 28 '22 11:01 Buakao

Here is the error. This error originated from a custom integration.

Logger: custom_components.yahoofinance.coordinator Source: custom_components/yahoofinance/coordinator.py:108 Integration: Yahoo Finance (documentation, issues) First occurred: 7:01:04 AM (134 occurrences) Last logged: 8:18:19 AM

Error obtaining data, retrying in 20 seconds.

Buakao avatar Jan 29 '22 01:01 Buakao

There should be some error message logged indicating why the updated failed. When an update failed, the retry is attempted after 20 seconds.

iprak avatar Jan 30 '22 21:01 iprak

I'm having the same problem, I found this in the LOG

Logger: custom_components.yahoofinance.coordinator
Source: helpers/update_coordinator.py:193
Integration: Yahoo Finance (documentation, issues)
First occurred: 10:21:04 (1 occurrences)
Last logged: 10:21:04

Timeout fetching YahooSymbolUpdateCoordinator data

luizgois-luizalabs avatar Feb 02 '22 14:02 luizgois-luizalabs

I suspect that the request to Yahoo API end point is timing out. If you enable debug logging then that can tell more.

You can change the logger level without restarting from Developer Tools > Services > Logger: Set level

service: logger.set_level
data:
  custom_components.yahoofinance: debug

Next invoke service Yahoo Finance: refresh_symbols to refresh the data. You should see some log.

iprak avatar Feb 03 '22 02:02 iprak

These were the logs I got

2022-02-03 06:24:44 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Requesting data from 'https://query1.finance.yahoo.com/v7/finance/quote?symbols=MGLU3.SA,LAME4.SA,AMER3.SA,LWSA3.SA,VIIA3.SA,CASH3.SA,MELI34.SA'
2022-02-03 06:24:59 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timeout fetching YahooSymbolUpdateCoordinator data
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Finished fetching YahooSymbolUpdateCoordinator data in 15.001 seconds (success: False)
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance] Coordinator did not report any data, requesting async_refresh
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Requesting data from 'https://query1.finance.yahoo.com/v7/finance/quote?symbols=MGLU3.SA,LAME4.SA,AMER3.SA,LWSA3.SA,VIIA3.SA,CASH3.SA,MELI34.SA'
2022-02-03 06:24:59 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.yahoofinance
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] Created entity for target_currency=BRL
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] Created entity for target_currency=BRL
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] Created entity for target_currency=BRL
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] Created entity for target_currency=BRL
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] Created entity for target_currency=BRL
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] Created entity for target_currency=BRL
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] Created entity for target_currency=BRL
2022-02-03 06:24:59 INFO (MainThread) [custom_components.yahoofinance.sensor] Entities added for ['MGLU3.SA', 'MELI34.SA', 'AMER3.SA', 'VIIA3.SA', 'LAME4.SA', 'LWSA3.SA', 'CASH3.SA']
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MGLU3.SA Coordinator data is None
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MELI34.SA Coordinator data is None
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] AMER3.SA Coordinator data is None
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] VIIA3.SA Coordinator data is None
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LAME4.SA Coordinator data is None
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LWSA3.SA Coordinator data is None
2022-02-03 06:24:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] CASH3.SA Coordinator data is None
2022-02-03 06:25:14 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Finished fetching YahooSymbolUpdateCoordinator data in 15.007 seconds (success: False)
2022-02-03 06:25:14 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MGLU3.SA Coordinator data is None
2022-02-03 06:25:14 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MELI34.SA Coordinator data is None
2022-02-03 06:25:14 DEBUG (MainThread) [custom_components.yahoofinance.sensor] AMER3.SA Coordinator data is None
2022-02-03 06:25:14 DEBUG (MainThread) [custom_components.yahoofinance.sensor] VIIA3.SA Coordinator data is None
2022-02-03 06:25:14 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LAME4.SA Coordinator data is None
2022-02-03 06:25:14 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LWSA3.SA Coordinator data is None
2022-02-03 06:25:14 DEBUG (MainThread) [custom_components.yahoofinance.sensor] CASH3.SA Coordinator data is None
2022-02-03 06:25:34 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Requesting data from 'https://query1.finance.yahoo.com/v7/finance/quote?symbols=MGLU3.SA,LAME4.SA,AMER3.SA,LWSA3.SA,VIIA3.SA,CASH3.SA,MELI34.SA'
2022-02-03 06:25:49 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Finished fetching YahooSymbolUpdateCoordinator data in 15.002 seconds (success: False)
2022-02-03 06:25:49 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MGLU3.SA Coordinator data is None
2022-02-03 06:25:49 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MELI34.SA Coordinator data is None
2022-02-03 06:25:49 DEBUG (MainThread) [custom_components.yahoofinance.sensor] AMER3.SA Coordinator data is None
2022-02-03 06:25:49 DEBUG (MainThread) [custom_components.yahoofinance.sensor] VIIA3.SA Coordinator data is None
2022-02-03 06:25:49 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LAME4.SA Coordinator data is None
2022-02-03 06:25:49 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LWSA3.SA Coordinator data is None
2022-02-03 06:25:49 DEBUG (MainThread) [custom_components.yahoofinance.sensor] CASH3.SA Coordinator data is None
2022-02-03 06:26:09 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Requesting data from 'https://query1.finance.yahoo.com/v7/finance/quote?symbols=MGLU3.SA,LAME4.SA,AMER3.SA,LWSA3.SA,VIIA3.SA,CASH3.SA,MELI34.SA'
2022-02-03 06:26:24 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Finished fetching YahooSymbolUpdateCoordinator data in 15.004 seconds (success: False)
2022-02-03 06:26:24 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MGLU3.SA Coordinator data is None
2022-02-03 06:26:24 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MELI34.SA Coordinator data is None
2022-02-03 06:26:24 DEBUG (MainThread) [custom_components.yahoofinance.sensor] AMER3.SA Coordinator data is None
2022-02-03 06:26:24 DEBUG (MainThread) [custom_components.yahoofinance.sensor] VIIA3.SA Coordinator data is None
2022-02-03 06:26:24 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LAME4.SA Coordinator data is None
2022-02-03 06:26:24 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LWSA3.SA Coordinator data is None
2022-02-03 06:26:24 DEBUG (MainThread) [custom_components.yahoofinance.sensor] CASH3.SA Coordinator data is None
2022-02-03 06:26:44 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Requesting data from 'https://query1.finance.yahoo.com/v7/finance/quote?symbols=MGLU3.SA,LAME4.SA,AMER3.SA,LWSA3.SA,VIIA3.SA,CASH3.SA,MELI34.SA'
2022-02-03 06:26:59 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Finished fetching YahooSymbolUpdateCoordinator data in 15.013 seconds (success: False)
2022-02-03 06:26:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MGLU3.SA Coordinator data is None
2022-02-03 06:26:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MELI34.SA Coordinator data is None
2022-02-03 06:26:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] AMER3.SA Coordinator data is None
2022-02-03 06:26:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] VIIA3.SA Coordinator data is None
2022-02-03 06:26:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LAME4.SA Coordinator data is None
2022-02-03 06:26:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LWSA3.SA Coordinator data is None
2022-02-03 06:26:59 DEBUG (MainThread) [custom_components.yahoofinance.sensor] CASH3.SA Coordinator data is None
2022-02-03 06:27:19 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Requesting data from 'https://query1.finance.yahoo.com/v7/finance/quote?symbols=MGLU3.SA,LAME4.SA,AMER3.SA,LWSA3.SA,VIIA3.SA,CASH3.SA,MELI34.SA'
2022-02-03 06:27:34 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Finished fetching YahooSymbolUpdateCoordinator data in 15.002 seconds (success: False)
2022-02-03 06:27:34 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MGLU3.SA Coordinator data is None
2022-02-03 06:27:34 DEBUG (MainThread) [custom_components.yahoofinance.sensor] MELI34.SA Coordinator data is None
2022-02-03 06:27:34 DEBUG (MainThread) [custom_components.yahoofinance.sensor] AMER3.SA Coordinator data is None
2022-02-03 06:27:34 DEBUG (MainThread) [custom_components.yahoofinance.sensor] VIIA3.SA Coordinator data is None
2022-02-03 06:27:34 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LAME4.SA Coordinator data is None
2022-02-03 06:27:34 DEBUG (MainThread) [custom_components.yahoofinance.sensor] LWSA3.SA Coordinator data is None
2022-02-03 06:27:34 DEBUG (MainThread) [custom_components.yahoofinance.sensor] CASH3.SA Coordinator data is None

luizgois-luizalabs avatar Feb 03 '22 09:02 luizgois-luizalabs

As per the logs, the request https://query1.finance.yahoo.com/v7/finance/quote?symbols=MGLU3.SA,LAME4.SA,AMER3.SA,LWSA3.SA,VIIA3.SA,CASH3.SA,MELI34.SA is not returning any data in 15 seconds. 15 seconds is a long time. I tried the same and got response back in a second.

I can't say why is preventing the access in your case. Can you navigate to that url successfully in browser? Did you define any scan_interval under yahoofinance ?

iprak avatar Feb 03 '22 11:02 iprak

I can easily access the address in the browser. I added:

   scan_interval:
     hours: 1

luizgois-luizalabs avatar Feb 03 '22 11:02 luizgois-luizalabs

Could the url be somehow blocked? The only thing I can think of is to add some more logging to see what is response code.

iprak avatar Feb 03 '22 11:02 iprak

I reinstalled via hacs and it worked again

luizgois-luizalabs avatar Feb 03 '22 14:02 luizgois-luizalabs

Ok. I had not released anything yet so this might be due to some install issue.

@Buakao What version are you at? What kind of HA install was it?

iprak avatar Feb 03 '22 15:02 iprak

It has become unavailable again

luizgois-luizalabs avatar Feb 03 '22 15:02 luizgois-luizalabs

I have pushed out a new release with more logging in the area when request is made.

iprak avatar Feb 04 '22 01:02 iprak

I updated the version with more logs and returned the log below:

2022-02-04 06:31:40 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting data from https://query1.finance.yahoo.com/v7/finance/quote?symbols=LWSA3.SA,MGLU3.SA,VIIA3.SA,MELI34.SA,CASH3.SA,AMER3.SA

luizgois-luizalabs avatar Feb 04 '22 09:02 luizgois-luizalabs

That confirms the request to https://query1.finance.yahoo.com/v7/finance/quote?symbols=LWSA3.SA,MGLU3.SA,VIIA3.SA,MELI34.SA,CASH3.SA,AMER3.SA is not returning any data in 15 seconds but I don't know why.

iprak avatar Feb 04 '22 11:02 iprak

How is your HA installed? I can try to replicate the setup.

iprak avatar Feb 04 '22 18:02 iprak

I'm running with VirtualBox, with the image available on the HomeAssistant website. I'm running with the latest version, but even before the update it wasn't working, but it worked for a long time with this installation.

luizgois-luizalabs avatar Feb 04 '22 18:02 luizgois-luizalabs

It's working now. Could it be some limitation on the amount of API requests?

luizgois-luizalabs avatar Feb 04 '22 21:02 luizgois-luizalabs

That is possible. The yahoo web end point is undocumented which is why I limited the minimum refresh interval. But you were able to successfully navigate to yahoo query url. One thing could be that the response is being limited based on request headers which I will look into more. I have been using 6hr without any issues for a very long time. See if increasing interval to 3 or 4 hr helps.

iprak avatar Feb 04 '22 22:02 iprak

Okay, I left the break at 3 hours, I'll follow up over the weekend and let you know the result.

Is there any debug I can do? I know python, but I didn't get to develop or debug something with HomeAssistant

luizgois-luizalabs avatar Feb 05 '22 00:02 luizgois-luizalabs

Yes that will be awesome. The method in question is get_json in custom_components/yahoofinance/coordinator.py makes a get request on an aiohttp object.

iprak avatar Feb 05 '22 00:02 iprak

So I created a VirtualBox instance with only yahoofinance sensors with 60 second refresh and everything was fine. Could there something in your network adapter settings preventing requests going out?

iprak avatar Feb 05 '22 01:02 iprak

I've been running it since the last week with a scan_interval of 2h, I haven't had any more problems.

luizgois-luizalabs avatar Feb 10 '22 18:02 luizgois-luizalabs

That is sort of a good news. I did not extensively test use of lower refresh intervals but I can see repetitive requests being blocked. I will mentioned this in the readme.

iprak avatar Feb 10 '22 21:02 iprak

I also can't get this to work.

Standard HA Docker install with Home Assistant 2022.2.6 HACS install of the integration

leogaggl avatar Feb 16 '22 05:02 leogaggl

Well, sorry that I haven't checked this thread for a long while. I solved the issue by switching to another HA in another machine in another location. The new machine is rpi 4b 8Gb, runing HASS.io installing as a docker (the one with the issue running HassOS on dedicated pi4b, 4Gb). The current working updates at 10 mins time interval. Since then I have no issue, which explains why I have been vanish, sorry. I hope this info help with others. I haven't tried reinstall yahoo finance on the first machine.

Buakao avatar Feb 16 '22 11:02 Buakao

Running https://hub.docker.com/r/homeassistant/home-assistant I can not get this integration to work

The following integrations and platforms could not be set up:

yahoofinance yahoofinance.sensor

Please check your config and logs

Nothing reported in logs.

leogaggl avatar Feb 16 '22 11:02 leogaggl

I also can't get this to work.

Standard HA Docker install with Home Assistant 2022.2.6 HACS install of the integration

Are you seeing sensors being unavailable? If so then debug logging might help discover what is going on.

iprak avatar Feb 16 '22 11:02 iprak

yahoofinance yahoofinance.sensor

Can you confirm if python files are present in the folder config\custom_components\yahoofinance folder ?

iprak avatar Feb 16 '22 11:02 iprak