core icon indicating copy to clipboard operation
core copied to clipboard

Not all devices are shown when adding Telldus live integration - Too many requests to pa-api.telldus.com

Open large opened this issue 1 year ago • 115 comments

The problem

Telldus Live is finally back, but it does not work as it did before. I am missing 3 devices "Lekestue", "Terrasseplante" and "Vindfang". Others got problems getting all their devices too?

Cannot be added later either, the popup just shows a blank dialog.

Click on tabs to see pictures from apps:

TelldusLive in HA (8 devices)
Telldus APP

Checking the logs and it seems that Home Assistant ignores the new rules set by Telldus on number of calls pr min

Log entries in HA
Failed request: 429 Client Error: Too Many Requests for url: https://pa-api.telldus.com/json/device/info?id=xxxx
Failed request: 429 Client Error: Too Many Requests for url: https://pa-api.telldus.com/json/device/info?id=yyyy
Failed request: 429 Client Error: Too Many Requests for url: https://pa-api.telldus.com/json/device/info?id=zzzz

See their site regarding that info: https://telldus.com/driftinformation/

  • Gör inte mer än 10 förfrågningar till pa-api.telldus.com per minut.
  • Kontrollera och dubbelkolla att dina skript inte gör nya försök att kontakta Telldus API utan att vänta minst 10 sekunder. Försök på nytt utan fördröjning kommer innebära blockering.

Roughly translated to 10 requests pr minute. Ensure scripts waits 10 seconds between each retry / or command (little unclear in the text)

What version of Home Assistant Core has the issue?

core-2023.12.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

tellduslive

Link to integration documentation on our website

https://www.home-assistant.io/integrations/tellduslive/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

large avatar Dec 15 '23 20:12 large

Hey there @fredrike, mind taking a look at this issue as it has been labeled with an integration (tellduslive) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of tellduslive can trigger bot actions by commenting:

  • @home-assistant close Closes the issue.
  • @home-assistant rename Awesome new title Renames the issue.
  • @home-assistant reopen Reopen the issue.
  • @home-assistant unassign tellduslive Removes the current integration label and assignees on the issue, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)


tellduslive documentation tellduslive source (message by IssueLinks)

home-assistant[bot] avatar Dec 15 '23 20:12 home-assistant[bot]

Having the same issue! Core 2023.12.3

Exchlud avatar Dec 16 '23 05:12 Exchlud

Having the same issue! 6 out of 24 devices showed up when the integration finally came back. Have tried to re-add the integration and it seems like 6 random devices are beeing integrated each time.

andhelle avatar Dec 16 '23 09:12 andhelle

Same here. Verified on both core 2023.12.1 and 2023.12.3

alexandersandstrom avatar Dec 16 '23 10:12 alexandersandstrom

Same for me as well

viktorfransson avatar Dec 16 '23 10:12 viktorfransson

14 out of 49 devices show up in HA. And those found are are all in an unavailiable state!

Says " No longer supported by this Telldus live integration"

Exchlud avatar Dec 16 '23 11:12 Exchlud

In the "Logs", check if you have a bunch of Failed request: 429 Client Error: Too Many Requests for url: https://pa-api.telldus.com/json/device/info?id=xxxx

In that case the TelldusLive integration needs to be more "slow" in the future because of the restrictions. This makes the 433mhz devices through Telldus worse than before. Maybe the integration should enforce "local" handling instead? I guess Telldus would be happy with that solution too. They will only get "premium checks" at api.telldus.com and all traffic are in-house with quicker response.

large avatar Dec 16 '23 11:12 large

Yep, lot's of "failed request: 429 Client error:....."

Exchlud avatar Dec 16 '23 12:12 Exchlud

Looks like the end of Telldus for me.

elyobelyob avatar Dec 16 '23 18:12 elyobelyob

https://telldus.com/driftinformation/ They are limiting 10 calls a minute to API and also expect some days between calls. Not sure what is going on.

Would be nice to be able to add some delays on integration. I'm having Telldus devices in 4 different location running 3 Home Assistants and currently I cannot do anything with them.

henripalmroth avatar Dec 17 '23 13:12 henripalmroth

+1, Unable to activate integration with 16 devices present on telldus api. Will make more than 10 req instantly. Since ha is importing tellduslive python package I think fix should be implemented there to throttle requests. https://github.com/molobrakos/tellduslive/blob/master/tellduslive.py

telldus premium: no

ollilainen avatar Dec 17 '23 18:12 ollilainen

@fredrike If you have some free time during Christmas, we would appreciate a late gift of making this plugin work better 😁

large avatar Dec 17 '23 18:12 large

I tried throttling workaround putting time.sleep(12) to _requests function on line 398 but still getting randomly errors from api. Even waiting 60sec after receiving status code 429 and then retrying will produce error.

2023-12-17 20:34:00.408 DEBUG (MainThread) [homeassistant.components.tellduslive] Update interval 60.0 seconds 2023-12-17 20:34:00.412 WARNING (SyncWorker_4) [tellduslive] Trying url https://pa-api.telldus.com/json/clients/list {} 2023-12-17 20:34:15.661 DEBUG (MainThread) [homeassistant.components.tellduslive] Connected hub hub1 2023-12-17 20:34:15.661 DEBUG (MainThread) [homeassistant.components.tellduslive] Connected hub hub2 2023-12-17 20:34:15.662 WARNING (SyncWorker_8) [tellduslive] Trying url https://pa-api.telldus.com/json/devices/list {'supportedMethods': 915, 'includeIgnored': 0} 2023-12-17 20:34:30.773 WARNING (SyncWorker_8) [tellduslive] Status 429: Too many requests, sleep 60s and retry 2023-12-17 20:35:32.878 WARNING (SyncWorker_8) [tellduslive] Failed request: 429 Client Error: Too Many Requests for url: https://pa-api.telldus.com/json/devices/list?supportedMethods=915&includeIgnored=0

Not sure what those limits are now on telldus api side (site says 10sec but 60sec is not enough...) https://twitter.com/telldus_status

ollilainen avatar Dec 17 '23 18:12 ollilainen

Retry workaround

patch file https://github.com/molobrakos/tellduslive/blob/master/tellduslive.py


diff --git a/tellduslive.py b/tellduslive.py
index 508c253..a4076da 100755
--- a/tellduslive.py
+++ b/tellduslive.py
@@ -10,7 +10,7 @@ from requests_oauthlib import OAuth1Session
 
 sys.version_info >= (3, 0) or exit("Python 3 required")
 
-__version__ = "0.10.11"
+__version__ = "0.10.12"
 
 _LOGGER = logging.getLogger(__name__)
 
@@ -27,6 +27,10 @@ TELLDUS_LOCAL_REFRESH_TOKEN_URL = "http://{host}/api/refreshToken"
 
 TIMEOUT = timedelta(seconds=10)
 
+import time
+RETRIES = 10
+RETRY_DELAY = 15
+
 UNNAMED_DEVICE = "NO NAME"
 
 # Tellstick methods
@@ -336,6 +340,24 @@ class Session:
             response = self._session.get(
                 url, params=params, timeout=TIMEOUT.seconds
             )
+            #check if statuscode 429 and retry
+            #delay=RETRY_DELAY
+            delay=RETRY_DELAY
+            retries=RETRIES
+            while (response.status_code == 429):
+                _LOGGER.warning("Received status 429: Too many requests, sleeping for %s seconds,  %s retries left %s %s", delay, retries, url, params)
+                time.sleep(delay)
+                delay=delay*2
+                response = self._session.get(
+                url, params=params, timeout=TIMEOUT.seconds
+                )
+                #if repsponse 200 logger ok
+                if (response.status_code == 200):
+                    _LOGGER.warning("Received status 200 after retry, %s %s", url, params)
+                retries=retries-1
+                if (retries == 0): #retry did not resolve in time
+                    _LOGGER.error("Retrylimit reached, %s %s", url, params)
+                    break
             response.raise_for_status()
             _LOGGER.debug(
                 "Response %s %s %s",
~
~
~
~
~
2023-12-17 22:48:31.995 WARNING (SyncWorker_0) [tellduslive] Received status 429: Too many requests, sleeping for 15 seconds,  10 retries left https://pa-api.telldus.com/json/device/info {'id': '6909574'}
2023-12-17 22:48:47.100 WARNING (SyncWorker_0) [tellduslive] Received status 429: Too many requests, sleeping for 30 seconds,  9 retries left https://pa-api.telldus.com/json/device/info {'id': '6909574'}
2023-12-17 22:49:17.222 WARNING (SyncWorker_0) [tellduslive] Received status 429: Too many requests, sleeping for 60 seconds,  8 retries left https://pa-api.telldus.com/json/device/info {'id': '6909574'}
2023-12-17 22:50:17.332 WARNING (SyncWorker_0) [tellduslive] Received status 429: Too many requests, sleeping for 120 seconds,  7 retries left https://pa-api.telldus.com/json/device/info {'id': '6909574'}
2023-12-17 22:52:17.585 WARNING (SyncWorker_0) [tellduslive] Received status 200 after retry, https://pa-api.telldus.com/json/device/info {'id': '6909574'}
2023-12-17 22:52:17.783 DEBUG (MainThread) [homeassistant.components.tellduslive.entry] Created device 
2023-12-17 22:52:18.981 DEBUG (MainThread) [homeassistant.components.tellduslive.entry] Created device 
2023-12-17 22:52:19.249 DEBUG (MainThread) [homeassistant.components.tellduslive.entry] Created device 
2023-12-17 22:52:19.497 DEBUG (MainThread) [homeassistant.components.tellduslive.entry] Created device 
2023-12-17 22:52:19.622 WARNING (SyncWorker_9) [tellduslive] Received status 429: Too many requests, sleeping for 15 seconds,  10 retries left https://pa-api.telldus.com/json/device/info {'id': '6979354'}

ollilainen avatar Dec 17 '23 20:12 ollilainen

Yep! Same issue here. I reloaded the integration, only the Tellstick Net showed up, no devices. Log full of "429"

BGunnarB avatar Dec 18 '23 14:12 BGunnarB

Same here, tried to setup local api by adding

tellduslive:
  host: 192.168.50.XXX

Had thought that Telldus servers wouldn't be called then but I still get the error: Failed request: 429 Client Error: Too Many Requests for url: https://pa-api.telldus.com/json/device/info?id=xyz

valleywood avatar Dec 18 '23 17:12 valleywood

Tried throtting devices/list sensors/list device/info requests and did not find simple time interval when api would accept all requests. Practically all sensor data will be minutes late from HA when doing retries over free version. I quess only solution is to buy premium as stated on https://www.home-assistant.io/integrations/tellduslive/

Please add to your answers if you have premium or not. Maybe this is problem only on telldus free version.

ollilainen avatar Dec 18 '23 19:12 ollilainen

It's the same problem with to many requests even with premium.

I have tried local api access to my telldus Znet lite V2, but i get The path '/tellduslive/authorize' was not found.

Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 670, in respond File "/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py", line 217, in call File "/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 61, in call File "/usr/lib/python2.7/site-packages/web/base/Server.py", line 230, in call File "/usr/lib/python2.7/site-packages/web/base/Server.py", line 202, in handle NotFound: (404, "The path '/tellduslive/authorize' was not found.")

36mobg avatar Dec 18 '23 20:12 36mobg

I too had the same issue with too many requests, only detecting 15 out 67 devices, premium member. But, I finally got local API working on my Tellstick Net V2! I thought a Znet v2 was required but I saw that API should be supported even on my model.

If you get the error: "The path '/tellduslive/authorize' was not found." It most likely IS a browser problem! I got this error in both Edge and Chrome.... But using ANOTHER browser, Duck Duck Go on my Android, it worked.

Steps I took: Note: I lost all my device mappings!

  1. Remove Telldus integration
  2. Add tellduslive: host to configuration.yaml
  3. Restart HA
  4. Login in to your HA in a new browser (non-chromium?) For example DuckDuckGo NOT Edge or Chrome!
  5. Telldus should be autodetected, so Configure it
  6. Choose the local IP
  7. Authorize
  8. You should see Telldus login page
  9. Login and choose a renewal period and auto renew, save
  10. Go back to HA and save
  11. Devices should now be detected

Let's see how this works, maybe I'll postpone migration from Telldus for now... 😄

Cheers

Blade3077 avatar Dec 18 '23 21:12 Blade3077

I too had the same issue with too many requests, only detecting 15 out 67 devices, premium member. But, I finally got local API working on my Tellstick Net V2! I thought a Znet v2 was required but I saw that API should be supported even on my model.

If you get the error: "The path '/tellduslive/authorize' was not found." It most likely IS a browser problem! I got this error in both Edge and Chrome.... But using ANOTHER browser, Duck Duck Go on my Android, it worked.

Steps I took: Note: I lost all my device mappings!

  1. Remove Telldus integration
  2. Add tellduslive: host to configuration.yaml
  3. Restart HA
  4. Login in to your HA in a new browser (non-chromium?) For example DuckDuckGo NOT Edge or Chrome!
  5. Telldus should be autodetected, so Configure it
  6. Choose the local IP
  7. Authorize
  8. You should see Telldus login page
  9. Login and choose a renewal period and auto renew, save
  10. Go back to HA and save
  11. Devices should now be detected

Let's see how this works, maybe I'll postpone migration from Telldus for now... 😄

Cheers

Thanks a lot, this worked for me!

KristianBj avatar Dec 18 '23 23:12 KristianBj

I too had the same issue with too many requests, only detecting 15 out 67 devices, premium member. But, I finally got local API working on my Tellstick Net V2! I thought a Znet v2 was required but I saw that API should be supported even on my model.

If you get the error: "The path '/tellduslive/authorize' was not found." It most likely IS a browser problem! I got this error in both Edge and Chrome.... But using ANOTHER browser, Duck Duck Go on my Android, it worked.

Steps I took: Note: I lost all my device mappings!

  1. Remove Telldus integration
  2. Add tellduslive: host to configuration.yaml
  3. Restart HA
  4. Login in to your HA in a new browser (non-chromium?) For example DuckDuckGo NOT Edge or Chrome!
  5. Telldus should be autodetected, so Configure it
  6. Choose the local IP
  7. Authorize
  8. You should see Telldus login page
  9. Login and choose a renewal period and auto renew, save
  10. Go back to HA and save
  11. Devices should now be detected

Let's see how this works, maybe I'll postpone migration from Telldus for now... 😄

Cheers

Hmm... For me duckduck still gives no authorization

raccna avatar Dec 19 '23 05:12 raccna

Hmm... For me duckduck still gives no authorization

I've also seen reports where wifi can interfere. Try using a wired connection on the same network as your Tellstick Net? Try unplugging/rebooting your Tellstick Net. I've also verified that using Firefox 120.0.1 on my (wired) PC also authenticates successfully with my Tellstick Net v2! Login to live.telldus.com and check your Location. Is the version on your Tellstick Net v2 reported as 1.3.2 and is it Online?

Using local API is a LOT zippier! My plugs and lights respond much faster. I don't have that many automations in HA, so i reconfigured most Telldus entities in about an hour. Still have some basic automations in TelldusLive (which I can/should move to HA), but at least everything seems to work again. I have so many devices in Telldus, so I'll probably keep it as long as it works OK... (Note, the Telldus Live automations also continues to work when using local API, so there is no apparent disadvantage to local API?) Wish i discovered this method sooner...

Blade3077 avatar Dec 19 '23 12:12 Blade3077

I too had the same issue with too many requests, only detecting 15 out 67 devices, premium member. But, I finally got local API working on my Tellstick Net V2! I thought a Znet v2 was required but I saw that API should be supported even on my model.

If you get the error: "The path '/tellduslive/authorize' was not found." It most likely IS a browser problem! I got this error in both Edge and Chrome.... But using ANOTHER browser, Duck Duck Go on my Android, it worked.

Steps I took: Note: I lost all my device mappings!

1. Remove Telldus integration

2. Add tellduslive: host  to configuration.yaml

3. Restart HA

4. Login in to your HA in a new browser (non-chromium?) For example DuckDuckGo NOT Edge or Chrome!

5. Telldus should be autodetected, so Configure it

6. Choose the local IP

7. Authorize

8. You should see Telldus login page

9. Login and choose a renewal period and auto renew, save

10. Go back to HA and save

11. Devices should now be detected

Let's see how this works, maybe I'll postpone migration from Telldus for now... 😄

Cheers

Many thanks, I have tried this for weeks since the pa-api is acting up. I have tried asking Telldus for help, premium customer as I am but have received snarky replies. Probably a stressful environment over there.

I can't for my life figure out how a missing URL suddenly would appear when changing browser but alas, here we are. I have been trying in Firefox to no avail, byt when opening the authorize-page in Chromium it worked, the missing URL suddenly appeared on my Tellstick Znet v2. Thanks!

haraldhh avatar Dec 19 '23 12:12 haraldhh

This doesn't make any sense. Premium account. Expired authentication (year, it never seems to autorenew). Trying to get it to autorenew I hit the same wall as everyone of you. I am, and have been, using the local integration due to speed things. However, all hell broke loose this time around. I tried multiple things and finally, I really don't know why, those steps mentioned above, done on Firefox with a local LAN connection work. Same steps with Android phone - no matter the browser - doesn't work Same steps with two different laptops, two different browsers (yes Firefox also) over wifi - no, doesn't work Same steps with laptop connected to LAN & pure installation of FF=> works.

I want to repeat, "this doesn't make any sense".

jleivo avatar Dec 19 '23 12:12 jleivo

Premium user with same issue here. Verified on both core 2023.12.1 and 2023.12.3

I have been in contact with Telldus support and alerted them to this thread. Was told their engineers will investigate

And also - could we please stick to the Telldus Live Integration issue and discuss the local API browser issue somewhere else?

wowbagger44 avatar Dec 19 '23 13:12 wowbagger44

Agreed, it's really really weird and also the reason I simply gave up a long time ago, since i assumed this meant my Tellstick didn't have local API... There is no obvious common setup that seems to work? For me, and it is consistent: Android13+Chrome+Wifi -> doesn't work Win11+Edge+LAN -> doesn't work Win11+Chrome+LAN -> doesn't work Android13+DuckDuckGo+Wifi -> works Win11+FF+LAN -> works Win11+Chromium+LAN -> works

But a wired PC and a clean/new browser is probably the first to try! I just tested Chromium portable on my PC. Version 115.0.5790.131 (Official Build, ungoogled-chromium) (64 bit). That also works, at least for me, but keep checking different browsers and PC combos if any of you have the same issue, there might still be hope! :)

Blade3077 avatar Dec 19 '23 13:12 Blade3077

Agree with @jleivo , it doesn't make any sense.

Really hope the guys at Telldus pick up the ball on this, since I think getting local API working seamlessly would save their bacon as well. It might well be that all the problems they are having is a product of them being to popular and people adding their stuff to HA. They have been slashdotted, if You remember that phenomenon?

haraldhh avatar Dec 19 '23 13:12 haraldhh

Now got it to work with wired connection + FF. Do I need premium for this to work in the future or can I stop paying Telldus?

raccna avatar Dec 19 '23 13:12 raccna

When I tried telling Telldus support that I am a paying customer they replied that I don't pay for the API. The Telldus Live integration page says that "you need a Telldus Premium subscription to access the Cloud API (https://telldus.com/en/important-announcement-english/)."

I will probably know in some time when my current Premium subscription ends since I stopped automatic charging of my card.

haraldhh avatar Dec 19 '23 13:12 haraldhh

I'm not sure if all of this is related to the problems I have. My Tellstick.net is sitting on another network far away. I am running HA via Nabu Casa on my browser at home. When adding the Telldus integration, I have the same problem as all of you above. The Telldus Net is identified but no devices connected to it. The log is flooded with 429 messages which I interpret as the Telldus.net is identified but as soon as the devices connected to it are reported, the throttling kicks in and nothing happens. I have paid for premium access.

BGunnarB avatar Dec 19 '23 13:12 BGunnarB