TWCManager
TWCManager copied to clipboard
ngardiner
Tesla API not working
Hi,
Anyone knows if there is any new change on the tesla API?, the token request is not working for me.
Regards.
Unfortunately, Tesla has again modified the login flow to add a captcha challenge which is breaking the auth flow for 3rd party apps.
Some discussion is here: https://github.com/timdorr/tesla-api/discussions/390
For iOS users, the following app can assist with logging in and capturing your API keys, which can then be inserted into the input boxes on the Settings page: https://apps.apple.com/us/app/auth-app-for-tesla/id1552058613#?platform=iphone
Captcha appears to be back. Some users may be unable to log in currently, if so, it's because Tesla have applied a captcha requirement to your account and we cannot yet handle the captcha in our login process.
Commit 3b77656 adds detection of captcha prompt from Tesla Auth. It's backwards-compatible, if we don't see a captcha request we just continue on, if we do see a Captcha request we redirect to a form to type in the captcha code.
Note that this is not a complete captcha flow - we still need to then take that code, resume the login flow and include the code in the login flow. We're closer to it working.
13010ab is the other half of the logic. It takes the captcha code and returns back to the login flow. I've just successfully logged in with an MFA account using this flow. I'll do some more testing, but I think we're set for Captcha handling
I've now tested this multiple times with both MFA and non-MFA login flows, and all attempts were successful.
@ngardiner today, TWC can't login to the Tesla account. I tried with and without MFA. Never had issues in the past. Maybe, is it possible because of today's Tesla software update?
@KirkKirk which version are you on? The only version that will be able to log in successfully today is the latest revisions of the dev release, v1.2.3 because Tesla have introduced captcha codes at login.
If you're running v1.2.2, unfortunately you'll need to wait a couple of days for the release of v1.2.3. I don't know I could make a release today but I am gearing up to have it done this weekend at the latest.
If you're running v1.2.3, you'll need to do a git pull to get the latest updates, you should then be able to log in.
Does anyone have this working on 1.2.3? I went through the login screens on 1.2.3, but the log shows API errors.
09:24:36 TeslaAPI 20 ERROR: Can't access vehicle status for Fake Vehicle Name. Will try again later.
So....
Tesla have changed the API login flow again.
Now they're using Google Recaptcha, and we don't yet have support for it, but I am working on it.
So....
Tesla have changed the API login flow again.
Now they're using Google Recaptcha, and we don't yet have support for it, but I am working on it.
I presume the manual token workaround does not work either?
I'm getting this as errors, and a blank page when trying to load twcmanager webpage:
20:53:13 TWCManager 20 BackgroundError: Traceback (most recent call last):
File "/home/user/TWCManager/lib/TWCManager/TWCManager.py", line 245, in background_tasks_thread
carapi.applyChargeLimit(limit=task["limit"])
File "/home/user/TWCManager/lib/TWCManager/Vehicle/TeslaAPI.py", line 937, in applyChargeLimit
if not self.car_api_available():
File "/home/user/TWCManager/lib/TWCManager/Vehicle/TeslaAPI.py", line 341, in car_api_available
or self.getCarApiTokenExpireTime() - now < 30 * 24 * 60 * 60
TypeError: unsupported operand type(s) for -: 'str' and 'float'
, occurred when processing background task
Unfortunately, it's nigh impossible to handle ReCaptcha headlessly, and Tesla requiring the callback URL to be a particular target forecloses using true OAuth even though TWCManager users are browser-based. It basically has to be a WebView now.
For MMM-Powerwall, I gave up and just take tokens directly; I included links to token-generating apps for iOS and Android, and am writing one for Windows.
However, @wooter, it's not clear that you're having the same problem. I'd guess that the path to set a token manually is storing a (fake) expiry time and it's accidentally in the wrong type.
So the good news is our login flow is not entirely headless, it's initiated from the UI and what we do is to redirect to a captcha prompt if we detect one. The thing is, the ReCaptcha flow is totally different to Tesla's own flow - they provided an image to validate against whereas ReCaptcha is javascript based.
It has been suggested that what is needed is to snarf the site id from the original page and then load the ReCaptcha Javascript with this site ID and ask the user to enter the code. I'm going to try it out this weekend and see.
What does worry me though, is this is about the 6th incarnation of Tesla's API login flow and they do seem to be trying to keep 3rd party apps out - at least that's how I interpret the constant cat and mouse games, and at some point we may just have to abandon the fight as it could go on like this for a very long time.
I'll try to figure out if there's an issue with manual token entry and at the same time try to update our flow. It's particularly frustrating as my development time is really limited at the moment with $dayjob pressures and my plan was to offer interfaces to avoid our reliance on Tesla API as much as possible by reading state from other projects such as TeslaMate, but the constant changing of the login flow by Tesla takes up all of that time and simply compounds the problem.
FWIW, I now have my Token generator app for Windows, so there should now be good options for getting tokens on most platforms. If someone has a MacOS app, I'd be happy to add it. I've placed app links above my token input form in MMM-Powerwall.
- https://www.microsoft.com/store/apps/9nhdtxbjppxn
- https://play.google.com/store/apps/details?id=net.leveugle.teslatokens
- https://apps.apple.com/us/app/authla/id1546597644
However, @wooter, it's not clear that you're having the same problem. I'd guess that the path to set a token manually is storing a (fake) expiry time and it's accidentally in the wrong type.
It's Friday evening and I did some checking. After manually submitting an Access Token and Refresh Token, the carApiTokenExpireTime in settings.json is "", an empty string. By manually replacing it with an epoch time value (expires_in + created_at from the API response) I've managed to avoid the error and get twcmanager working.
Is there a switch to disable TeslaAPI entirely? apart the current issue and the fact that I'm very uncomfortable to handout any sensible car info or digital keys, I'd like to charge other cars as well.
@nean-and-i, you can change the stop mode in Settings to "Stop Responding" or "Stop command." If TWCManager decides not to charge, though, the car will see it as a sudden loss of power and after a cycle or two won't accept offered power until it's unplugged and plugged back in.
@MikeBishop thanks for the hint, however, I put it again into "fakeMaster: 2" mode and switched the TWC to "5" position so anyone can charge right away.
@nean-and-i - Please open another issue for this. The vast majority of users do use the Tesla API and this is an ongoing issue related to login, your request will simply get lost in here.
Okay, so after a long struggle with the new authentication flow I have made some progress. I'll share my findings here.
The challenge with the new recaptcha flow is that it is javascript based rather than image based, and as a result extra checks are performed by Google when you solve the captcha. One of those checks is an origin check, they check that the domain that the recaptcha check is being executed on is .tesla.com and fail if not.
This is not something we can handle natively, but I have a workaround that does work, but it requires a DNS entry to be added to either a hosts file or to your DNS resolver. In addition, I've made some improvements to the token handling on the settings page, if you enter a token now the refresh time will be automatically set to 45 days which is the default refresh token age, however we try to refresh it starting at half of that time.
Finally, I am working on an interface that will sync tokens from TeslaMate, and similar interfaces could be set up for other projects too. This will allow us to outsource token handling to other projects so we don't have to worry so much about the next time that Tesla changes the login flow.
In the latest commit, if you attempt to log in to the Tesla API and get challenged with a recaptcha, a set of options are presented:
- Use an app to authenticate to Tesla and obtain tokens. These can be entered under the Settings menu.
- Configure Token Sync with an external application like TeslaMate
- You could perform a DNS redirection trick to host TWCManager on a tesla.com domain.
Note that this should only impact those authenticating for the first time, or where existing tokens have been invalidated. If you logged in previously, you'll have the bearer and refresh tokens stored and we'll continue to refresh those tokens going forward.
Development of the TeslaMate interface is still underway and it's not yet usable. I intend to have it ready soon and then release 1.2.4 for those suffering from auth issues as a result of the recent changes.
I’m running the docker development version (1.2.4) and I’ve tried every way to deal with authentication including setting up the DNS redirection to get the captcha working and while it offers, it never completes. Using the tokens gets the API working, but renewal with the refresh token seems to not work either so overnight charging fails. My most recent attempt is to get TeslaMate working and that’s fine with TeslaMate having no issues monitoring the car and I’ve got TWC talking happily to it but it won’t stop the car charging. It sends the stop signal but complains the car hasn’t been woken in the last two minutes and repeats constantly. In desperation, I tried the debug tab and sent a wake signal. The car immediately stopped charging so it seems if you send wake first and then the stop it works. I understand the TeslaMate interface is still in development and it’s really close.
After a long time without problems, my TWCManager instance was logged out of the Tesla account today.
Now I cannot login again: after solving the recaptcha challenge it takes quite a while until I see this error in red:
Error encountered during Phase 2 (POST) of the Tesla Authentication process.
Things I did:
- added "twcmanager.tesla.com" to my hosts file to get the recaptcha
- updated to the latest commit of the main branch
Did Tesla change something again or is there a known workaround for this problem?
Did Tesla change something again or is there a known workaround for this problem?
Yes, what you are encountering is the issue that was introduced in September, where Tesla have effectively made it impossible for web apps to implement full end to end authentication flows.
It is possible they've already worked out that the tesla.com trick was possible and further limited DNS domains to specific hosts - it worked back when I implemented it, but things change constantly with Tesla auth.
You will need to use one of the apps that allow you to obtain tokens I am afraid. It's a problem for all development using the Tesla API and I suspect it won't be going back to the way it was. The reason you need to reauthenticate is that someone was able to obtain the API credentials for 20 vehicles, leading Tesla to expire lots of API tokens, so I'd personally expect more heavy handedness towards API credentials, especially if it turns out it was a 3rd party app that exposed them.
https://www.techspot.com/news/92971-teen-hacker-gains-remote-control-over-20-teslas.html
The car immediately stopped charging so it seems if you send wake first and then the stop it works. I understand the TeslaMate interface is still in development and it’s really close.
That's interesting! I don't have the same issue in my environment and technically the vehicle should already be awake if it's charging, but I have no objection to waking the car before sending a stop charging command, especially if it makes it work with greater stability.
I found a separate issue which I'm about to commit a fix for (if the tokens expire after we do sync with TeslaMate, they don't update on the TWCManager side) so I will add this in at the same time.
Thanks for the testing.
It is possible they've already worked out that the tesla.com trick was possible and further limited DNS domains to specific hosts - it worked back when I implemented it, but things change constantly with Tesla auth.
Good point. I just tried to add www.tesla.com (instead of twcmanager.tesla.com) to my /etc/hosts file and use that to connect to twcmanager and solve the recaptcha, but I get the same Phase 2 error during auth. Maybe they limit it to https now so it doesn't work over http?
You will need to use one of the apps that allow you to obtain tokens I am afraid.
I tried that in the past, but I couldn't get TWCManager to refresh the tokens. They alway expired after 7 hours.
I have now tried it again using the access token and refresh token the "Auth app for Tesla" app from the App Store. This works so far, let's see if it is refreshed when the access token expires.
The reason you need to reauthenticate is that someone was able to obtain the API credentials for 20 vehicles, leading Tesla to expire lots of API tokens, so I'd personally expect more heavy handedness towards API credentials, especially if it turns out it was a 3rd party app that exposed them.
https://www.techspot.com/news/92971-teen-hacker-gains-remote-control-over-20-teslas.html
Thanks, I see. I didn't hear about that before.
Thanks a lot for your answer, I really appreciate it.
Do you accept donations or something similar for this project?
I tried that in the past, but I couldn't get TWCManager to refresh the tokens. They alway expired after 7 hours.
I have now tried it again using the access token and refresh token the "Auth app for Tesla" app from the App Store. This works so far, let's see if it is refreshed when the access token expires.
TWCManager cannot refresh tokens generated by "Auth app for Tesla". I forced a refresh and this is logged by TWCManager:
Jan 15 02:50:08 TWCManager python3[16231]: 02:50:08 🚗 TeslaAPI 13 Entering car_api_available - next step is to query Tesla API
Jan 15 02:50:08 TWCManager python3[16231]: 02:50:08 🚗 TeslaAPI 13 Attempting token refresh
Jan 15 02:50:09 TWCManager python3[16231]: 02:50:09 🚗 TeslaAPI 19 Car API request<Response [401]>
Jan 15 02:50:09 TWCManager python3[16231]: 02:50:09 🚗 TeslaAPI 17 Car API auth response{'error': 'invalid_grant', 'error_description': 'The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client.'}
I also added logging to TeslaAPI.py to see why the auth flow fails after the captcha. This is what Tesla's server returns when the request is sent in apiLoginPhaseTwo:
code: 403
headers: {'Server': 'AkamaiGHost', 'Mime-Version': '1.0', 'Content-Type': 'text/html', 'Content-Length': '296', 'Expires': 'Sat, 15 Jan 2022 02:22:54 GMT', 'X-Reference-Error': '18.1d8a1402.1642213374.8c79ce67', 'Date': 'Sat, 15 Jan 2022 02:22:54 GMT', 'Connection': 'close', 'Permissions-Policy': 'interest-cohort=()', 'Set-Cookie': '_abck=6EFE80B6232CB350321DAEC2D20E952F~-1~YAAQHYoUAmiavLV9AQAASnqLWweiTgbrVtZ7VpzsOkgeu0j/Py0BRYVthWbaCmLAvhuy+/9EhCih8yKMhLmPkzU843NuXgznvhlB/dt8JTLhffIXmGzvOqLMwzu55TF5qJdaqJSPFlqPxHY6b1dnwLaHBCBcro8vtACKTNBlBLoD9lylkj54eCNNihoNw8YOsHDR1/nC91+B7is231CE8YNSXupX9IleivTAaTu3WypM05Q9oUCg2LiAKQLTagXORhk2dI3o+vfab8C+B1Q73HvhJlkX8wT+RH/TJYQqOJ1AJ0VgL9+YgpoUb2ap2QoAryK43Ld5lYmnveENUu6nrv9jaEdd83DjoldAJqMvMiIgu7sYlKIOIkG8lVbhBovHIxY+pM3Z~-1~-1~-1; Domain=.tesla.com; Path=/; Expires=Sun, 15 Jan 2023 02:22:54 GMT; Max-Age=31536000; Secure'}
text: <HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
You don't have permission to access "http://auth.tesla.com/oauth2/v3/authorize?" on this server.<P>
Reference #18.1d8a1402.1642213374.8c79ce67
</BODY>
</HTML>
So I'm not sure what to do now.
Success! I could make TWCManager refresh tokens created by "Auth app for Tesla" after I found this comment: https://github.com/timdorr/tesla-api/discussions/390#discussioncomment-1959992
The following changes were needed in TeslaAPI.py:
Change refreshURL from
"https://owner-api.teslamotors.com/oauth/token" to
"https://auth.tesla.com/oauth2/v3/token".
In def apiRefresh(self): change data to:
data = {
"client_id": "ownerapi",
"grant_type": "refresh_token",
"refresh_token": self.getCarApiRefreshToken(),
"scope": "openid email offline_access",
}
Since access tokens expire after 8 hours, I also changed the expiry check from 30 days to 1 hour. Otherwise it would refresh tokens every time.
In line 371, changed from
or self.getCarApiTokenExpireTime() - now < 30 * 24 * 60 * 60 to
or self.getCarApiTokenExpireTime() - now < 60 * 60