pyawair
pyawair copied to clipboard
netmanchris
discussion: Home-Assistant support
@danielsjf Saw your message on another python Awair API project. I've been working with @deanlyoung on this python wrapper for the Awair API. Wondering if you might consider using this for the Home-Assistant project.
Plan is to stay in compliance with the official API so, by necessity, there will be some functionality that will not be exposed until Awair adds it to the API.
This is currently available on Pypi
Thoughts?
I'll have a look. Is it the official API that is only available for professional users? Because unfortunately I don't have access to that one.
@danielsjf I believe we may have corresponded a while ago. We’re actually opening up the API (new, better endpoints) for developers. I can get you access to the private beta if it would be helpful in working on your Home-Assistant integration.
You can register with your Awair account here: https://developer.getawair.com
@danielsjf Looks like you can get access to the API! :)
I'm still working on the scaffolding. Adding in tests and docs.
My primary use-case is to enable this as part of an automated datapipeline to practice some machine learning stuff I've been playing around with, but I'm running HASSiO so I'm more than happy to help out if you can use this as a foundation for your device support. Just let me know what you need out of the python API and we can figure it out.
Intention is to also create some full python classe(s) to help make it easier to work with the devices but this is a part time weekend project and there's still some pieces missing from the public API ( ability to power the Glow on/off is one which I've already reported for instance ). Happy to work through this with you. I have an Awair and a Glow, but no Mint or Omni here. :)
@netmanchris it looks good. I will have a bit more time in August so I will certainly look into it. I don't see anything obvious missing. Documentation in the code looks fine after a first view. Maybe a few more examples on querying the air data in the Readme would be helpful. In the end that's what people want to have.
Unfortunately I only have the awair, not the glow. I can implement it, but I won't be able to test it.
I suppose the 5 and 15 minute data is coming straight from the awair API? Are there other granularities as well?
@deanlyoung I have registered my account for the development access. So does this mean that the API will be available soon for everybody? Also, are there any constraints in using the API? Max calls per day/month?
@danielsjf Appreciate the feedback. Trying to make sure that this is a project that doesn’t suck. :)
Definitely need to flesh out the examples as I get more data on what’s actually exposed in the API and what the values mean. All part of the process. API is still beta,so there are some things missing, but Awair and @deanlyoung have been great about answering questions and taking requests.
The 5 and 15 minutes are directly out of the API. There are some additional filter mechanisms that can be added which i want to implement to allow for further granularity when asking for data from the API. Current and Raw data is available as well. Is there anything else you think we need?
The free API I think is listed as having 1000 calls a day, but I’ll let @deanlyoung comment on that one.
@netmanchris you've done some great work!
After posting here yesterday I've found the tiers here: https://developer.getawair.com/onboard/tiers
Therefore the free version only has the 15 minute data and 100 calls per day. I think for HA this is perfect. You can query the data every 15 minutes which would make 96 requests per day.
From the tier data it is not clear how this relates to using the app. If this is the same API, we might need to lower it a bit. Just to avoid that you cannot use the app anymore.
@danielsjf Hmmm.. Sounds ok for one device, but if there are multiple devices wouldn’t that double the amount of calls and push us past the free tier? I’ll need to look if there’s a “get_all_device_data” call to gather data points for multiple devices during the same call.
@deanlyoung Can you answer the last question above? How does the tier data limits relate to the app? Are these totally separate things or does using the iOS/Android app affect the API query limits?
@deanlyoung If the free tier limit isn’t locked in 100% yet, based on the use-case above, can I also request that Awair consider raising the number of allowed API calls? I’d be happy with 100 API calls per Awair device registered to the account if that’s possible? This would also solve the problem of multiple devices for Home-Assistant shown above. Seems that 100 a day is just a very small number. Definitely understand the requirement for rate limiting and giving preference to the paid model, but seems like the free tier might be too small and the sudden jump in cost to the small dev tier is pretty steep.
@danielsjf we’re still working out the limits and pricing (it’s currently a placeholder - hence the private beta). It will eventually be open to everyone. Just need to polish it up.
@netmanchris using the app will not affect your API usage limits. See my comment above about pricing. Still a lot to figure out :)
@deanlyoung when do you expect the API to be final? I would like to add it directly to HA in the near future. Obviously this might attract a bit of attention for Awair, so we should wait until the API is out of beta. Is there a Roadmap that you are working with? What are the big changes you are still planning?
@danielsjf because I’m not as familiar with HA, it would be really helpful if you could list out a wishlist of sorts that would make the lowest “tier” of the Awair API useful (or really attractive). Then, as we finalize for public launch, we can come up with some tiers/features that are actually useful.
@deanlyoung I think there are two use cases:
- monitoring
- data science
At the moment, I used the current data endpoint for monitoring.
I could see the following endpoints and tier division:
- Current data instantaneous: last measured point (all tiers).
- Current data 5 min: aggregate of all measurements in the last five min (all tiers)
- Current data 15 min: aggregate of all measurements in the last 15 min (all tiers).
The first three are more a matter of personal flavour. Do you want your values to be a averaged out or instantaneous. In the end the limit here will be the calls per day which are different for the different tiers. Therefore in the lowest tier with 100 calls/day, you will never call this more often than once every 15 min.
The next group is for the data science:
- 15 minute data between date 1 and date 2, at most x days in one call (first tier).
- 5 minute data between two days, at most x days in one call (second tier).
- Raw data between two days, at most x days in one call (third tier).
The last three are for data science. Here you want to look for specific events in your data. A bit up to you and the backend what to give. Here you can make clear tiers based upon what granularity of data you can get.
Does this seem reasonable for you? To be clear, the first three only give one moment in time. Only the last three give multiple values.
For further reference, HA is using the values for monitoring. I was using the current data endpoint. Now that this is blocked, I am using the 15 min endpoint instead. This gives me a longer history but I am only using the latest value. Therefore it is a bit a waste of bandwidth. I think a dedicated endpoint for instantaneous values is more optimal.
@danielsjf definitely helpful, thank you! What is the optimal frequency in HA for calling /latest endpoint (or /15min-avg, etc.)? HomeKit, for example, tries to limit updates to no more than once per every 2 minutes.
@deanlyoung I don't think there are lower limits, other than the processor speed. I found some people that complained that without setting a limit, there were multiple checks per minute.
I think it depends what you would like to do. Temperature is typically moving only slowly. For an AC, it is fine if it starts only 15 minutes after a threshold is reached. Usually the temperature will not be much higher by then.
For things like light intensity or noise (V2), I suppose the reaction speeds needs to be quicker (e.g. lowering the volume). For these sensors, a minute can already be long.
@deanlyoung what is the current status on the API? Do you have a timeline or a roadmap defined?
Specifically it would be interesting to know:
- by when it would be possible to add the instantaneous end points (whether or not they are averaged),
- by when the tiers are final
- by when the API is out of beta
As soon as the answers to all three questions are positive, I'll submit the Awair component to HA as a pull request for the HA project.
@danielsjf sorry we haven’t been super transparent with timing and roadmap. We’ve been heads-down working on improvements to the documentation, working out bugs, and some other as yet unreleased things. As we get closer, I will keep you updated, and may send out an email with a survey.
I would say the endpoint structure is there. New endpoints may get added, or specific adjustments to limits, but we hope to make the management of your developer account very flexible. For light usage, if you don’t want to pay for something you won’t use, no problem. If you want to use a lot of features, there will be options for that, too.
@danielsjf @deanlyoung Doing some clean up. Are we ok to close this or should we keep it open for now? Until the is finalized I don't see us making much movement on this issue. Are we ok to close it and reopen later?
What is the current status on the API. With the latest changes in v0.0.12, the HA component works well. I would like to release it officially for HA, but is the Awair API itself ready for that or is it still in beta?
@danielsjf @netmanchris we updated the default Hobbyist tier quotas. Let me know what you think!
@deanlyoung Honestly, I would like to see the 5 min included in the hobbies as wel. I think the difference in the number of API calls a day is a huge value for the small developer. The main difference between the hobbies and the small developer from a persona standpoint is the small dev is probably going to be trying to monetize his app which requires a lot more API calls, so I think you have that right.
I also think you might want to consider some limited access to web books in all tiered, the Enterprise tier is definitely going to be the one who uses that the most, but I believe the hobbies tier is the one who would be tinkering the most to provide custom integrations and such. I think there’s a lot of value for Awair to build a community of passionate hobbies who would be able to come up with unique solutions that Awair alone wouldn’t be able to achieve.
@danielsjf Thoughts?
@netmanchris the the new hobbiest permissions doesn’t have access to 5min? All air-data endpoints should have some quota available.
“Small developer” may or may not exist in name, but it would probably be reserved for an individual that needs to use tons of data.
As for monetizing apps, we’ll leave that to OAuth apps and the developers who make them. We agree that individual developers shouldn’t be nickled and dimed for access to their data. Until we can add per device quotas, we just need to find a decent quota for accounts.
@deanlyoung could it be that this page is not up to date anymore? https://developer.getawair.com/onboard/tiers
At least there it is mentioned that the five min endpoint is not available.
@danielsjf was hoping no one had bookmarked that 😅 We realized it was causing more confusion than providing answers. That page will get entirely revamped to make everything clear. If we did a simple page like that there would be like 9+ tiers or something... tiers among individuals, app developers, enterprises, etc.
@deanlyoung That's the exact document that I went to as well. :) All my comments were based on looking at that specific document. Is there something you can point us to that will break down the new tiers?
@netmanchris it will be at the same location, just gotta work on the UI/UX. Probably a month or more away on that being 100% ready for launch.
@deanlyoung, @netmanchris the current API seems rather stable so I would like to add the Awair component directly to home assistant. The current tier levels seem right for that. Do you see any objections before a release?
Not finalized, so looking for feedback here:
One thing we plan to do next week (hopefully before you do that) is change the way the backend works. On top of that, there is a bit of a mismatch between what the docs describe desc and limit and how they actually behave. We’re going to try to keep the behavior almost exactly the same, but there might be some subtle differences (edge cases).
Currently, desc defaults to false (I believe most people use desc=true to return the latest of raw/5min-avg/15min-avg). The desc parameter was intended to sort AirData points before returning them (as it is written in the description it makes it sound like it sorts the data after querying it). In other words, if no from datetime were supplied and desc=false (default), then it would search from January 1, 1970 00:00. Under the new timeseries database that we will be using, this would normally return a null array because no device was online back then.
To get around this, we will default the from datetime to the maximum time range given whatever to (default: current datetime) is supplied. For raw this would be 5 minutes ago, 5min-avg is 24 hours ago, and 15min-avg is 7 days ago. Theoretically, something should always be returned then, as long as the device was reporting data in the last 5 minutes, 24 hours, or 7 days (respectively). desc will also default to true and be applied as a post-processing of the data instead of which end of the “from/to” datetimes to start pulling from (this is the really the only main difference). limit will also be post-processed on the dataset returned, so if the maximum time range were ~288 points (24 hours) from 5min-avg, and limit were set to 10, then only 10 would be returned (very very similar if not identical to now).
Endpoints will be the same. Parameters will be the same. Documentation will be updated to reflect the changes.
Number of maximum data points per endpoint will change to specific time ranges instead of number of data points. It’s somewhat nicer though because you will know what time range to return vs. some arbitrary length of time determined by a number of arbitrary data points. In addition, it should be much faster and more reliable.
Number of calls per day should be unaffected.
We look forward to your feedback!
@deanlyoung Yes I remember that there was something odd with that desc parameter. In the end I solved it by using limit=1.
I think the changes sound good and are for the better. Especially the default to true for desc.
What do you mean with changes to the backend?
Could you ping us when the changes to the backend go live? In that way we can quickly check if something went wrong with our code by running the tests.
There is no reason to do it quickly for home assistant. I only want to avoid that it keeps on being postponed. Certainly now that I had the feeling that the API started to mature. But I think that the changes that you propose are good. I certainly agree with waiting for those changes.
@danielsjf yes, hopefully for the better all around, and we’re really trying to consider all edge cases. Most people seem to be “live” streaming data by polling at a regular interval, in which case the changes shouldn’t cause any problems.
In order to get the performance improvements, there are mostly invisible changes on our end (how we surface information to the API endpoints). Not really relevant, but in case you were interested 🤷🏻♂️
Apparently someone else also made a package already. He also added it to home assistant so it will be in the next release. https://github.com/ahayworth/python_awair