architecture
architecture copied to clipboard
home-assistant
RPC: Unit systems conversion
As we know, there are different unit systems around. We have the temperature covered but there are more unit of measurement. Ok, my screen 14" but our car drives not faster than 145 km/h. Weather data units, speed, length, pressure, etc. are exposed in different unit of measurements around the world.
In #11166 (was triggered by #11136) the idea was to an additional helper (util/speed.py) to do the conversion of different wind speeds and their unit of measurements.
The question is: Do we want to start writing our own helper to convert various unit of measurements into the needed output format (if one is required) or use something like pint to do it?
Writing it as needed wouldn't require a lot of changes but is a bit of reinventing the wheel and introduce a solution to a problem which was already solved.
Using a third-party module would require changes to the core but in the long run it would add flexibility if we are pushing further in the direction of more localization (not only language) of the user interface.
With c/f it is easy you always want one and not the other,but with other units it is not always clear. I.e. i might prefer metric, but what do I convert inches to - mm, cm or m?
One thing I like about HA is that common functionality lies in external libraries.
Be it pint or any other library, if I ever get to vote on the matter, I would always go for the use of an external library to handle the case.
Just a thought:
What if the hass backend always returned standardized metric units, and we made unit localization a frontend concern?
@armills I mean that's true, but let's not forget about automations/back-end. While I myself always use the metric system, a user might be confused when they suddenly have to use metric units in scripts, numeric triggers, ...
Off the top of my head, it also seems like normalizing to metric carries with it rounding errors that could cause display problems. We don't want an sensor that originally reports a nice round number to end up being a long irrational number because of a conversion to and from a normalized unit? (That may be a solved problem, just spitballing here)
I think that if we talk about speed conversion, we can't do it automatically anymore inside Entity#update_ha_state. Instead, for example the weather component will have to take charge, look at the users preference (miles or kilometers) and look at the context to do the conversion. Context could mean: it's a wind speed, we express those in m/s or miles/h etc.
I think it makes sense to add a speed preference and then people will have to update each component to do contextual conversion.
I think that makes a lot of sense. So many unit preferences depend on the thing that's being measured. I want my cars in mph, my rockets in m/s, and my variometers in ft/min!
It would probably make sense to figure out how the user can configure preferred units in a consistent global way.
Something like?
units:
vehicle_speed: mph
wind_speed: kph
And then it's up to each component to determine which of the measurement types it is reporting. For the actual conversion, should we have a standard library like pint (as faboff mentioned) and encourage components to use that for the actual conversion itself?
One problem with this approach is that it will destroy historical data when the user changes their preference, if this conversion is done before reporting its state to the state machine...
@OverloadUT This doesn't seem like a good idea to me:
- What if I want to measure some slower vehicles in m/s and some in km/h; or more realistically: what if I want to measure my total power usage in kW, but the power usage of a lamp in W?
- The components don't necessarily know what type their sensor is unless we want to have a new config value for ~1/4 of platforms. For example, MQTT devices will never be able to know the measurement type without an explicit config entry.
- If you want to add specific entries for each and every speed type: There are tons and tons of different possible measurement types; What if I want to measure the speed of my cat 🐈 or my dog 🐕, or more realistically: what if I want to measure the atmospheric pressure outside in
atmand the pressure of my vacuum pump inhPa? We would then need to have a way to override the global preferences for specific entities too.
We should only have 1 speed preference. It will be up to the component to use that speed preference to create the right value based on the context. That is not something that we can do.
I don't think I agree with that. In aviation for example, it's very common to use different units for ground speed and vertical speed. I think considering "speed" to be all one category is oversimplifying the reality of how units of measurements work. That's why I suggested that the user can define the unit preference by context. Some contexts would cover all uses of a particular measurement (F and C for temperature for example) both they don't always need to.
@OttoWinter My proposed system actually covers every one of your examples - that's exactly what I was trying to solve with my suggestion. pressure_inside, pressure_outside, velocity_animal, etc. That list could of course become unwieldy, but I think it wouldn't be unreasonable to maintain a list? Then again, this might be a big case of YAGNI.
I don't feel super strongly about this suggestion, it just feels like a pretty clean and flexible approach.
@OverloadUT What I was trying to say was that, in order to fullfill all needs, units: would have to cover every single use case/context, including the speed of cats and dogs. So I think that list would eventually include hundreds of entries - and that's simply impossible from a maintenance point of view. If you have to write velocity_animal somewhere in your code in order to cover all uses cases you're doing something horribly wrong.
I think every single solution that attempts to solve unit systems will have to have some way to manually override the unit on a per-entity basis. For me, a complete unit system would allow overriding the unit of an entity without having to create a template sensor. Whether such a "complete" unit system is possible or not is another question though.
So, I thought even with the simplification of having one entry for each physical quantities, we'd still have to have a ton of entries, but the amount of entries is actually not too bad. Off the top of my head, most/some of these would have to be supported (gets a bit silly towards the end; might contain spelling mistakes):
- temperature
- pressure
- velocity (+ acceleration)
- luminosity
- length (+ area/volume)
- voltage, current, power, energy
- charge (+ capacitance, impedance, inductance)
- time
- angle (+angular velocity/acceleration/momentum, torque)
- mass, force
- frequency
- data_rate (for example MByte/s)
- radiation
I haven't had a look at pint yet, but I think creating config validation for this might get messy (for example: we'd need to make sure that you can't use light years for power).
I think that if a sensor would show vertical speed (can't think of a use case tbh), it would be fine for the sensor to judge which unit to use when the "main" preference is miles or kilometers. Making it super configurable is going to overcomplicate things.
@andrey-git, I know this is 7 months old now but turns out you don't always want everything in °C or °F. I have everything in imperial but then added my 3D printer as a sensor, which always uses °C and trying to convert it was a headache. Managed to hack it together by converting it and using a unit of ' °C' with that space in front so HA did try re-converting it. It would be great to have a flag in yaml to not auto convert for situations like this.
For the weather platforms it could be handy to keep the origin unit of measurements in some cases, too. Now we are converting the values (https://github.com/home-assistant/home-assistant/pull/16823) to a unit of measurement we have chosen. My guess is that it will not take long till a "sailor" wants it in knots and no, beaufort is something for a template sensor :wink: .
I can’t believe HA does not have the ability to override the global metric variable. I live in the US so I have my global metric variable set as Imperial - so I can see weather temps as fahrenheit.
I recently added sensor for my servers cpu temperature but I cannot force it to report the temperature in celsius, despite setting the unit_of_measurement. Not convenient to see cpu temps in fahrenheit.
** I was led to this issue item by searching my issue online. **
Internationally, unit of measure is tied to locale and countries often have a mixture of units irregardless what their official unit system is. Here's just a few of those problems.
- Canada - Metric country
- Temperature: People use both Fahrenheit and Celsius to measure body temperature, so much so that Health Canada often publishes recommendations in both units for fevers.
- USA - Imperial country
- Velocity: Uses miles per hour, except for medicine and science, pulse wave velocity is measured in meters per second.
- UK - Metric country
- Volume: Uses liters, except for blood and beer.
- Mass: Users killograms but people often measure humans in stone.
In writing the Withings component for HA, I've already ran into issues where temperature is incorrectly converted to Celsius when the user cares about Fahrenheit. In the Withings component's case, I've allowed the user to use either by creating a different entity to hold a different state for mass, temperature, etc. While this works well, it fails with temperature and just looks buggy. This is the sort of thing the developer can switch off on a case-by-case basis.
Continuing to implement automatic unit conversions would be VERY hard to do correctly. A more sensible approach would be use the system config provided by the user but then allow the user to override the value on the entity level.
In the meantime, we should allow developers to skip this automatic conversion. I've create a PR to do exactly that.
https://github.com/home-assistant/home-assistant/pull/23499
I think that adding a property to not do something is silly. I think that instead, and this is a bit more impactful, we should extract the entity conversion logic out of the base entity class and move it to a new sensor entity (as it's the only component that cares about it). We should definitely not add more features to the base entity class to satisfy edge cases inside an integration.
I also think that the provided use cases could be solved by expanding our unit system, instead of trying to hack yourself a way around it, or even worse, force users to configure themselves around it, while it should be solved at a core or integration level.
Moving the conversion logic into a dedicated sensor class provides a sensible workaround for my immediate use case. I can start that pr if you like.
Longer term however, I'm not confident implementing a sensor class solves the issue with conversion to undesired units. It seems this can be solved by allowing users to force the units of measurement with entity customization.
On Sun, Apr 28, 2019, 12:02 PM Paulus Schoutsen [email protected] wrote:
I think that adding a property to not do something is silly. I think that instead, and this is a bit more impactful, we should extract the entity conversion logic out of the base entity class and move it to a new sensor entity (as it's the only component that cares about it). We should definitely not add more features to the base entity class to satisfy edge cases inside an integration.
I also think that the provided use cases could be solved by expanding our unit system, instead of trying to hack yourself a way around it, or even worse, force users to configure themselves around it, while it should be solved at a core or integration level.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/home-assistant/architecture/issues/10#issuecomment-487406481, or mute the thread https://github.com/notifications/unsubscribe-auth/AANL2MSBGRUAYR7P3OOZLBTPSXYDPANCNFSM4ERAX2DA .
On Sun, Apr 28, 2019, 12:02 PM Paulus Schoutsen [email protected] wrote:
I think that adding a property to not do something is silly. I think that instead, and this is a bit more impactful, we should extract the entity conversion logic out of the base entity class and move it to a new sensor entity (as it's the only component that cares about it). We should definitely not add more features to the base entity class to satisfy edge cases inside an integration.
I also think that the provided use cases could be solved by expanding our unit system, instead of trying to hack yourself a way around it, or even worse, force users to configure themselves around it, while it should be solved at a core or integration level.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/home-assistant/architecture/issues/10#issuecomment-487406481, or mute the thread https://github.com/notifications/unsubscribe-auth/AANL2MSBGRUAYR7P3OOZLBTPSXYDPANCNFSM4ERAX2DA .
On Sun, Apr 28, 2019, 12:02 PM Paulus Schoutsen [email protected] wrote:
I think that adding a property to not do something is silly. I think that instead, and this is a bit more impactful, we should extract the entity conversion logic out of the base entity class and move it to a new sensor entity (as it's the only component that cares about it). We should definitely not add more features to the base entity class to satisfy edge cases inside an integration.
I also think that the provided use cases could be solved by expanding our unit system, instead of trying to hack yourself a way around it, or even worse, force users to configure themselves around it, while it should be solved at a core or integration level.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/home-assistant/architecture/issues/10#issuecomment-487406481, or mute the thread https://github.com/notifications/unsubscribe-auth/AANL2MSBGRUAYR7P3OOZLBTPSXYDPANCNFSM4ERAX2DA .
To add to the use cases for mixed °C/°F units: I monitor my garden and environmental conditions in °C because I prefer the metric system, but live in a country that uses imperial units. My partner's recipe collection is in °F, for the most part, so all cooking sensors (BBQ probes, sous vide) report in °F. I am doing the same space-prefix hack as @domgregori (' °F') for those Fahrenheit sensors so conversion does not occur automatically.
I would much prefer to be able to get Home Assistant's units to match up with how we do things in my home, rather than use one system of measurement across the board. I'd prefer the global setting to be treated as a default. Then another config flag like unit_of_measurement_conversion could be set to null to disable conversion from the input unit_of_measurement value, or set to any other unit of measure than the default to allow the user to override the default and get something fun like Kelvin for their science experiment when their sensors report in F and their global system of measure is set to metric.
Sorry for the back-seat design -- this is just the experience I would have liked to have setting up new cooking sensors today. Hope that's useful input!
My full use case today was configuring a Weber iGrill to talk to HA via a raspberry pi to bridge from Bluetooth -> MQTT. AFAIK that sensor only reports in °F and I can't change that. I want it to display in °F anyway. That's just what we use for cooking! But I don't want to change all my other sensors to °F as well.
Another use case: system monitoring. I use imperial units for mostly everything, but metric units for measuring CPU temps (as does most people). Currently the Glances sensor (and I suspect any other component that measures CPU temperature) displays CPU temperatures in Fahrenheit, with no way to override this despite the Glances API endpoint displaying the temperature in Celsius. In order to get the temperature in Celsius I have to create another template sensor that reverts the conversion of the temperature:
Glances API (°C) -> HomeAssistant (units = imperial, °F) -> Template Sensor (°C)
Having to create an extra sensor to undo the automatic conversion of HomeAssistant just feels wrong to me. Instead I override the Glances component and force the unit_of_measurement to display what is returned via the Glances API (as it should)
Sensible defaults with the option to override is always my go to, regardless of how it's implemented.
This architecture issue is old, stale, and possibly obsolete. Things changed a lot over the years. Additionally, we have been moving to discussions for these architectural discussions.
For that reason, I'm going to close this issue.
../Frenck
