Rest sensor - digest issue with the new httpx

[mateuszdraco]

The problem

I've spent a lot of time trying to debug why my integration with Hikvision intercom stopped working some time ago. What I found out is that httpx with its new version introduced a change which altered how digest auth works and eventually led to my integration being useless.

I raised this issue with httpx via their discussions, I hope they can fix it soon. My request is to downgrade to 0.23.0 or upgrade to the new version if it's going to appear soon.

What version of Home Assistant Core has the issue?

2023.1.2

What was the last working version of Home Assistant Core?

2023.1.2

What type of installation are you running?

Home Assistant Container

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[Joao-Sousa-71]

I have exactly the same problem with My Hikvision Intercom devices and version 2023.1.7

[psike01]

Same problem in 2023.2.2

[mateuszdraco]

So according to my discussion with httpx contributors (big thanks for the support) it seems HomeAssistant should change how it uses this library with digest. It should create httpx.DigestAuth for each call, instead of keeping its single instance.

[Xirt]

I believe I have similar issues hence posting my setup to add information:

rest:
  - authentication: digest
    username: user
    password: [secret]
    scan_interval: 1
    resource: http://192.168.X.X/ISAPI/VideoIntercom/callStatus?format=json
    sensor:
      - name: Doorbell Status
        value_template: "{{ value_json.CallStatus.status }}"

This results in the following log entry (every second):

Logger: homeassistant.helpers.template
Source: helpers/template.py:624
First occurred: 08:18:26 (475 occurrences)
Last logged: 08:26:40

Template variable error: 'value_json' is undefined when rendering '{{ value_json.CallStatus.status }}'

Turning on debug level logging shows the following:

2023-02-08 08:51:07.395 DEBUG (MainThread) [homeassistant.components.rest] Finished fetching rest data data in 0.111 seconds (success: True)
2023-02-08 08:51:07.395 DEBUG (MainThread) [homeassistant.components.rest.sensor] Data fetched from resource: 
2023-02-08 08:51:07.396 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{ value_json.CallStatus.status }}'

Prior to the upgrade yesterday everything was working fine. I assume value_json is undefined as the call is no longer authorized, but this does not show in the log. For now, a rollback to my previous setup (2022.12) is the only option I have as my doorbell is no longer functioning :). I hope a fix can be provided in upcoming releases.

[pergolafabio]

You can always use my addon meanwhile to capture events from Hikvision :-)

[Xirt]

I believe that utilizes options for installation I do not have in my container version of HA? If a manual installation option is available, I could give it a try :). Still I believe above issue must be resolved as it might hamper more than just HikVision setups.

[mateuszdraco]

My workaround was to ssh on the machine which hosts HA (in my case it's a vm), uninstall the new httpx and manually install the older version. Everything works, yet I'm not updating HA since then :(

[pergolafabio]

indeed, but there is also a mqtt version of it

[Xirt]

My workaround was to ssh on the machine which hosts HA (in my case it's a vm), uninstall the new httpx and manually install the older version. Everything works, yet I'm not updating HA since then :(

I tried the same using the following commands in the shell:

pip uninstall httpx
pip install httpx==0.23.0

After a restart everything works again. Let's hope this "fix" is only needed temporarily.

[Xirt]

Confirmed still broken in 2023.3.1, but above workaround (downgrade) appears to still work.

[markus2504]

Still not working in version 2023.3.5

[pergolafabio]

A quick workaround is using the command line sensor instead of the rest sensor

[pergolafabio]

Or use my addon, it now supports MQTT and can run as standalone docker :-)

[markus2504]

I have DS-KB8113-IME1 and the MQTT addon is not stable. How to use command line sensor? Unfortunately, I don't know English.

[pergolafabio]

Not stable? Offcourse it's stable, never had any issues with it, all users are very happy.. not had a single complain... What's your issue then , you can open an issue if you want?

Anyway, for shell, look here https://www.home-assistant.io/integrations/sensor.command_line/

[asdfkid]

Or use my addon, it now supports MQTT and can run as standalone docker :-)

would love to, but I run HA Core and as far as I know, that doesn't support add-ons. Are you releasing a standalone version soon?

[asdfkid]

So according to my discussion with httpx contributors (big thanks for the support) it seems HomeAssistant should change how it uses this library with digest. It should create httpx.DigestAuth for each call, instead of keeping its single instance.

so glad I found this discsusion. I was tearing my hair out trying to figure out why my rest sensor was broken. Interestingly, I also found the following:

• when HA rest sensor was enabled, the API was giving 401 errors from manual command line curl entries as well
• if I disabled the rest sensor, after about 20 mins the curl would work (auth lockout timeout?)
• If I logged into the web interface of my DS-KV6113 that would reset the API lockount instantly (didn't have to wait the 20 mins)

[pergolafabio]

Or use my addon, it now supports MQTT and can run as standalone docker :-)

would love to, but I run HA Core and as far as I know, that doesn't support add-ons. Are you releasing a standalone version soon?

Hi , standalone docker is possible, have a look at the dev branch for the documentation how to run it standalone , it's using MQTT now

[mateuszdraco]

@pergolafabio I just tried your addon and it looks just awesome! Massive thanks for that! One question: can I detect if the intercom is unavailable?

[pergolafabio]

You are using the beta, right?

Hmm, when connected off network you mean? Hmm not sure, didn't tested ;-)

[mateuszdraco]

No, I'm using regular version (I thought beta could be unstable). I can try beta, although I have a couple of places where this is plugged so changing it will take a bit of time ;)

And yes, I have my intercom connected via wifi and sometimes it looses the connectivity. I already replaced my access point and since then it's much better, yet, I'm kind of stressed about it ;)

[pergolafabio]

Use beta, will be released soon as official, it's much more stable now already, lots of more functions and used mqtt

[pergolafabio]

It should detect if it's unable to connect

[mateuszdraco]

Can I build an automation on top of this? Previously when I didn't have a connectivity I sent myself a phone notification to kick the system manually.

[Xirt]

I put a smart plug at the socket that my Hikvision uses for power and HA automatically reboots (toggle plug twice with a small delay) in case Hikvision is unavailable for a longer time. Which is - despite the very strong network nearby - quite frequently...

[pergolafabio]

I never tested it, I have a reboot instruction implemented , bit offcourse , it needs network then... Maybe the callsensors will give unavailable, I didn't test it

[mateuszdraco]

@Xirt yeah, you work the system quite well, in my case after AP change it improved a lot.

[issue-triage-workflows[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.