Broken API authorization with long-lived access token

[alexander-vitishchenko]

The problem

• I get "(401) Unauthorized" REST API error message after upgrading my Home Assistant Core 2023.3.1 to 2023.3.3
• I'm using the same long-lived access token, and the issue is gone once I fallback to version 2023.3.1. When I upgrade to 2023.3.3 - the issue returns. I tried to create new long-lived access token but it still fails.
• I'm using Home Assistant OS (running on Home Assistant Blue hardware). Trying to access API both locally and via Nabu Casa
• Tried using HTTP POST and GET methods
• Other people seem to experience a similar kind of issue, and one of the users attached the logs with an authorization issue, but the logs do not explain why access tokens stopped working suddenly

Here goes one of the scenarios that I use for the issue reproduction:

• Open Postman application for REST API testing (I use it for the automated regression testing of the Fibaro Home Center 3 and Home Assistant integration, but not able to share the collection itself due to security reasons)
• Make REST API call using URL https://<nabucasa-anonymized>/api/hassio/ingress/session, and pass Authorization Bearer token within the HTTP request header

Expected result:

• HTTP response code 200
• HTTP response body { "result": "ok", "data": { "session": "<...>" } }

Actual result:

• HTTP response code 401 (Unauthorised)

What version of Home Assistant Core has the issue?

core-2023.3.3

What was the last working version of Home Assistant Core?

core-2023.3.1

What type of installation are you running?

Home Assistant OS

Integration causing the issue

api

Link to integration documentation on our website

https://www.home-assistant.io/integrations/api

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (api) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of api can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign api Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

api documentation api source _{^{(message by IssueLinks)}}

[mkagawa]

I have the same situation after upgrade HA core.

[alexander-vitishchenko]

@bdraco, @epenet - can you please look into it?

It seems like you did the API changes recently and it could potentially break authentication with long-lived tokens accidentally.

The problem seems to be reported by different people in parallel, and looks like a high severity/priority?

Thank you, Alexander

[tarmor1]

FWIW, I get my APIs working OK both from internal network (to local IP) and externally to NabuCasa URL. Running Docker version on 2023.3.3.

[luney-sp-76]

I have had the same issue since 17-03-23. Local Host is working but my Nabu Casa REST API is not. home-assistant_2023-03-19T11-11-47.551Z.log

Current version Home Assistant 2023.3.5 Supervisor 2023.03.1 Operating System 9.5 Frontend 20230309.1 - latest

From the logs you will see there have been failed authorization requests. Some belong to me but not the Mac OSx requests. I tested the API with known good and known bad authorization tokens for both the local and NabuCasa Cloud URL to see if it was my token.

The issue is not on my local instance. The token is valid and the previous three months have been without issue.

[frenck]

We've been looking at this issue, but lacking a reproduction. I've not been able to get into an error state using long-lived access token.

Could you provide a step-by-step reproduction (including the software I have to use), to re-create the error?

Thanks! 👍

../Frenck

[alexander-vitishchenko]

Just finished additional investigating of the issue, and adjusted the description:

• I'm using the same long-lived access token for all the testing
• The issue is resolved once I fall back to Home Assistant Core 2023.3.1. And the issue is consistently reproduced when I upgrade to Home Assistant Core to version 2023.3.3+.
• I'm using Home Assistant OS (running on Home Assistant Blue hardware). Trying to access API both locally and via Nabu Casa. Using HTTP POST and GET METHODs
• Other people seem to experience a similar kind of issue, and one of the users attached the logs with an authorization issue, but the logs do not explain why access tokens stopped working suddenly

Please advise what other information I can provide for you to identify the root cause?

Thank you.

[frenck]

Please advise what other information I can provide for you to identify the root cause?

As per my previous comment: A step-by-step reproduction (including the software I have to use), to re-create the error?

../Frenck

[alexander-vitishchenko]

Here goes one of the scenarios that I use for the issue reproduction:

• Open Postman application for REST API testing (I use it for the automated regression testing of the Fibaro Home Center 3 and Home Assistant integration, but not able to share the collection itself due to security reasons)
• Make REST API call using URL https://<nabucasa-anonymized>/api/hassio/ingress/session, and pass Authorization Bearer token within the HTTP request header

Expected result:

• HTTP response code 200
• HTTP response body { "result": "ok", "data": { "session": "<...>" } }

Actual result:

• HTTP response code 401 (Unauthorised)

[balloob]

That endpoint is not available via the HTTP API anymore.

[mkagawa]

if so, 401 is bad and confusing response code, correct? in my case, i interact with HACS addon (ring live stream) status within the same box.

Home Assistant 2023.3.5 Supervisor 2023.03.1 Operating System 9.5 Frontend 20230309.1 - latest

xxxx is long lived access token

$ curl -H'Authorization: Bearer xxxxx'\
 http://homeassistant:8123/api/hassio/addons/local_ringlivestream/info -D -
HTTP/1.1 401 Unauthorized
Content-Length: 0
Content-Type: application/octet-stream
Date: Mon, 20 Mar 2023 13:24:57 GMT
Server: Python/3.10 aiohttp/3.8.4

[pvizeli]

Correct, these endpoints are not publicly available and are only for the front end. They were also never publicly documented. I guess it comes from reverse engineering of the frontend.

The front end changed, and the endpoints are gone. Please make sure to implement your use case officially.

[mkagawa]

got it. thanks for your response.

[alexander-vitishchenko]

Thank you for the clarification on the broader context.

Can you please advise what would be the recommended approach for the given situation:

• I use "Node-RED" add-on to host automations that are hardly possible to implement with the Home Assistant automation natively
• My "Node-RED" instance has custom HTTP endpoints created, and they accept a few parameters as inputs, and produces response describing results via NabuCasa

Using /api/hassio/ingress/ is not ideal, but looked the cleanest so far. The alternatives are:

• Webhooks => no parameterization supported + no response details provided for the caller + it looks like no synchronous calls support
• Create a pair of "virtual sensors" whose values will be accessible to both Node-RED and a webclient (REST API) for exchanging events asynchronously. But it lacks synchronous calls and looks much hackier than /api/hassio/ingress/

It would be great if you could recommend a better solution, because /api/hassio/ingress/ did all the magic previously.

Thank you.

[balloob]

Create an automation in HA with a webhook trigger and forward that data to Node Red via an MQTT message or something. (ideally Node-RED had their own official integration in HA to forward data to them, but they do not have one).

[alexander-vitishchenko]

Thanks @balloob

This is exactly what I was considering as one of the alternatives from my message above, with a few caveats that you can hopefully help to address.

As I understand, there two ways to connect Webbooks to Node-RED:

• using native Home Assistant Webhook Integration to Node-RED (from HACS store). This is what I tried but the integration has limitations to pass input/output parameters and also for making synchronous calls
• manually configure event transfer from Webhooks to Node-RED like MQTT that you suggested. This might resolve the need for passing input parameters, but leaving the open question how to make synchronous calls + return execution results back to the webhook's caller

[alexander-vitishchenko]

Since all my options look like workarounds anyway - I can do a bit of HTTP profiling and see how new version of the Home Assistant ingress session id/cookies are used, and then update my authorization code for Postman accordingly.

Then continue routing ingress traffic from webcaller/Postmant to Node-RED, similarly to how I did in the past.

I did some prove-of-connect this morning and successfully connected one of my Postman tests to the new version of the Home Assistant API, hope it will work for the rest.

However, Home Assistant might and most likely will change in the future - and then I'd need to adjust my code again.

Hope you have better options.

Thank you.

[balloob]

Closing this issue as it's working as intended.

[covert-]

I followed the curl example with long-loved token ("For a quick test curl can be useful to “simulate” a device.") here https://www.home-assistant.io/integrations/http/#binary-sensor

I ran into the same 401 error with "/api/states/binary_sensor.mytestsenor" testing on the local network on another machine to HA

I changed the user to "Administrator" in config/users and all 401 Unauthorized become 200 OK

changing "Can only log in from the local network" for the user had no effect

Home Assistant 2023.3.6 Supervisor 2023.03.3 Operating System 9.5 Frontend 20230309.1 - latest

[luney-sp-76]

Issue with Nabu Casa has gone away after the last Update to Home Assistant OS