mike-grant
Improving the performance
Haaska lambda takes too long to process requests. I've stripped down all libraries, using only standard Python 3.7 libraries and I managed to reduce response time from 1106 milliseconds to 650 for cold start. For warm requests, I reduced it from 600 ms to less than 100ms.
Also, the implementation is much short and fits in a gist snippet: https://gist.github.com/unaiur/b354f986f874a8ca7ad80153d7b37440
I am using environment to configure the lambda. You need to create at least two environment variables: HOST=myname.duckdns.org BEARER_TOKEN=same_token_than_in_config_file
Also, you can add some optional configurations: POST=xxx (default 443) DEBUG=1 (debug is disabled by default)
After debugging the lambdas with X-Ray, it seems that extra time comes from:
- Initialisation time: <100ms vs 600ms. My lambda only loads 3KB, while yours uses more than 1MB because of the requests library.
- Executing the lambda: first time it takes more or less the same, but my lambda reuses the connection in following invocations, allowing to reduce the execution time of a "discovery" request from 500 to 100ms. Of course, it depends on how far away is the AWS datacenter from you home. In my case, I am in Spain and AWS in Ireland (ping 50ms)
Just on mobile so I can’t do too much yet, but if you’d like to make a PR, we’d be happy to review it.
I’ll admit, we’re using code that was originally a proof of concept with a few tweaks here and there to keep it working over the years.
If your version works well and doesn’t break compatibility with the config.json too much, I’m sure we can adopt it.
When you say it’s re-using the connection, just to be clear, it’s not keeping it open all the time, right?
We do want to keep an eye on what resources we use, as I would imagine a lot of users are on the AWS Free Tier.
Actually, after I wrote that, the computer opened up, so I read over the code quickly. It looks pretty good. If all the config is moved over to Env Var, I’m okay with that. It shouldn’t be too much of a change, and it might even be easier to deal with any code upgrades, as now it doesn’t live beside the code explicitly.
As I said, let me know how you want to move ahead with it. As long as you’re okay with the same licensing (MIT), then I think we’d probably accept this pretty quickly.
I am still analisyng what are the main latency sources. In my setup, it is:
- Lambda startup: ~~500ms~~ <50ms now
- DNS resolution using duckdns: ~200ms
- TLS handshake: 450ms. For some reason HA takes 250ms to answer client hello. I am diving deeper here.
AWS does not kill lambda containers after serving a lambda invocation. The container is frozen and reused if a new request arrives soon enough (< 10 minutes usually). In that case, the connection is kept open and reused, and only closed on error or when the container is destroyed by AWS.
Using AWS Route53 as DNS provider and setting up an NGINX proxy in front of HomeAssistant, I managed to reduce the cold latency to 300ms and the warm latency to 100ms. It seems that a Raspberry PI is too slow to handle TLS in Python.
Most likely, I just haven't had much time to investigate the changes. But as it largely looks like dependencies it should be fine. @Swiftnesses have you been using it
@mike-grant, not yet, but I plan to roll it out this weekend... Thanks for the great add-on BTW :)
@mike-grant: If I test this on the weekend and it works, I’m willing to PR it, merge it, and update the docs for the new env var steps.
Does that sound good?
@unaiur thanks! I also replaced the original haaska and all the libraries it bundled, with this - and it worked fine. A lot simpler and lighter now. I removed all the libraries & stuff that were included in the old haaska.zip lambda 'bundle'. I had to customize line 54 (the URL 'path') since I use an NGINX reverse proxy with some custom path. The difference in responsiveness is very noticeable.
+!, confirming. The gist snippet works just fine with the Python 3.7 libraries. Nice job.
Hello I try to integrate the file but I can not possibly get the zip pack including the improvement? thank you in advance
@unaiur I'm using your improved version for some time now and it is really fast. Thank you!
For a few days now Alexa answers <devicename> does not respond
The invocation logs show the following error:
Traceback (most recent call last):
File "/var/task/haaska.py", line 78, in event_handler
return handle_event(event)
File "/var/task/haaska.py", line 61, in handle_event
headers = HEADERS)
File "/var/lang/lib/python3.7/http/client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/var/lang/lib/python3.7/http/client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/var/lang/lib/python3.7/http/client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/var/lang/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/var/lang/lib/python3.7/http/client.py", line 987, in send
self.sock.sendall(data)
File "/var/lang/lib/python3.7/ssl.py", line 1034, in sendall
v = self.send(byte_view[count:])
File "/var/lang/lib/python3.7/ssl.py", line 1003, in send
return self._sslobj.write(data)
BrokenPipeError: [Errno 32] Broken pipe
Is there something I can do on my end to fix this?
I'm running HA in docker behind an apache2 reverse proxy.
Hi , I cleared all the code and pasted this one but i get error during test:
Response: { "errorMessage": "Handler 'lambda_handler' missing on module 'lambda_function'", "errorType": "Runtime.HandlerNotFound" }
@unaiur @mike-grant was this change ever merged? Seems like a good improvement based on the discussion, but I didn't see the relevant commit.
It hasn't been merged, I haven't really got around to reviewing the change unfortunately. @dfederm do you have time to make a pull request with these changes involved?
Actually, I just found that the HA docs have a simplified set of instructions with an updated gist as well, similar to the one proposed here. I set up haaska a while back and at some point it stopped working (probably when the legacy api stuff was pulled?) so I was planning to dig into it tonight, but I'll just go that other route instead.