eigendude
[REQUEST] OASIS integration for Home Assistant
Please make an OASIS integration (custom component) for Home Assistant to enable computer vision interaction + input streaming.
-
https://developers.home-assistant.io/
-
https://developers.home-assistant.io/docs/development_index/
-
https://developers.home-assistant.io/docs/creating_component_index/
-
FYI, also posted this for discussion on Home Assistant community forums:
- https://community.home-assistant.io/t/oasis-smart-home-operating-system-based-on-ros-2-robot-operating-system/405110
I believe that most other somewhat similar integrations for Home Assistant use their WebSocket API or REST API (and MQTT) to communicate.
PS: Alternatively, consider extending device types in existing Kodi integration to take input from this and other inputs via it instead:
- https://www.home-assistant.io/integrations/kodi
I started experimenting with Home Assistant integration so that I can talk to my Philips Hue bulbs, but now I'm waiting for Matter to launch and I'll add Matter support directly.
All I need is another use case that HA would provide. If one pops up then you can expect integration.
I started experimenting with Home Assistant integration so that I can talk to my Philips Hue bulbs
Tip for directly connecting Zigbee based Philips Hue lights to Home Assistant is to buy a Silicon Labs EFR32MG21 (or EFR32MG13) based USB adapter (like Elelabs/Popp Zigbee USB stick) or a Texas Instruments CC2652P/CC2652R based Zigbee USB adapter (like "Itead Sonoff Zigbee 3.0 USB Dongle Plus") and connect that via a long external USB extension cable (to get it away from any electonics so they do not interfer with signals) and then use Home Assistant's native ZHA integration component -> https://www.home-assistant.io/integrations/zha/
I' m waiting for Matter to launch and I'll add Matter support directly.
I believe that a more popular option for OpenThread Board Router is to get a Silicon Labs EFR32MG21 (alternatively older EFR32MG13 or newer EFR32MG24) based dongle but those are more expensive and harder to find today due to the current chip shortage (which Silabs has suffered most), see example:
Regardless, recommend that you consider buying several of those "Itead Sonoff Zigbee 3.0 USB Dongle Plus" adapters since they are very inexpensive and not only can you use it as a Zigbee Coordinator adapter as well as flash them as Zigbee Router device firmware (to act as a repeater for extending Zigbee range and coverage) but you should also be able to flash them with OpenThread Board Router (OTBR) firmware (to be used as a bridge for Matter over Thread devices).
- https://github.com/NabuCasa/universal-silabs-flasher
Home Assistant does however not yet have a Matter integration for Home Assistant the founders who now employ some deevelopers under Nabu Casa mentioned in the latest newsletter that they have employed the lead developer from the Z-Wave JS project might help get it started -> https://building.open-home.io/a-new-home-for-our-newsletter/
Another Home Assistant developer also wrote that he started working on both an OpenThread Board Router addon and a Silicon Labs multi-protocol addon for Thread and Zigbee which can be used on the same dongle at same time if using an "RPC" firmware:
https://github.com/zigpy/zigpy/discussions/894
https://github.com/home-assistant/addons-development/tree/master/silabs-multiprotocol
https://github.com/home-assistant/addons-development/tree/master/openthread_border_router
Previously I believe those using OpenThread Board Router (OTBR) firmware on Silicon Labs EFR32 (EFR32MG21 or EFR32MG13) or CC2652P (and CC2652R or CC2652RB) are only communicate with OpenThread devices simply by using their IPv6 addresses, controlling and publish a local OpenThread network Thread/OpenThread togther with the wpantund Userspace WPAN Network Daemon:
https://github.com/openthread/wpantund
All I need is another use case that HA would provide. If one pops up then you can expect integration.
FYI, I know that many in the Home Assistant userbase may not actually use Home Assistant itself as an automation platform, (as that part of Home Assistant still is neither very user-friendly or extensive out-of-the-box if used via the UI), therefor many in the Home Assistant userbase might instead use Node-RED or third-party scripting for automations and only utilize Home Assistant as an integration platform and protocol bridge/proxy to consolidate all your devices because Home Assistant already has a very extensive collection of integrations for most things.
Home Assistant Analytics statistics even show that the average Home Assistant instance has 76 integrations installed (that of course include several default system integrations as well) -> https://analytics.home-assistant.io/#stats
By the way, be sure to check out HACS (Home Assistant Community Store) for additional integrations -> https://hacs.xyz
Hey I need to connect my moorebot scout (ros1melodic) to my smart home to get images (Preferably in color) and send commands to and get data from to use in automatons. Like alert triggered, send scout #1 to door. Or if motion in kitchen use path to kitchen. I really also want to just straight up control it from a home assistant dashboard with the buttons for forward, backwards, and turning. Just in case I want to give selective access to someone. Also because theirs no other way to do it in the app. Also to use it as a motion sensor. This seems like a fantastic idea that would make my smart home far more reactive and likely to drive away intruders with sheer confusion from multiple robots converging on their position.
Hey I need to connect my moorebot scout (ros1melodic) to my smart home to get images (Preferably in color) and send commands to and get data from to use in automatons.
Sorry, I don't support EOL versions of ROS.
Tip for directly connecting Zigbee based Philips Hue lights to Home Assistant is to buy a Silicon Labs EFR32MG21 (or EFR32MG13) based USB adapter (like Elelabs/Popp Zigbee USB stick) or a Texas Instruments CC2652P/CC2652R based Zigbee USB adapter (like "Itead Sonoff Zigbee 3.0 USB Dongle Plus") and connect that via a long external USB extension cable (to get it away from any electonics so they do not interfer with signals) and then use Home Assistant's native ZHA integration component -> https://www.home-assistant.io/integrations/zha/
I have renewed interest in connecting Hue to OASIS. I'll start looking for solutions again.
Custom component integration for Home Assistant’s core (backend) would be very cool to see some day, and made available via publishing it on HACS (Home Assistant’s Community Store) to make it so that more people discover it:
-
https://developers.home-assistant.io/docs/creating_component_index/
- https://developers.home-assistant.io/docs/architecture_components/
-
https://www.hacs.xyz
-
https://www.hacs.xyz/docs/contribute/
- https://www.hacs.xyz/docs/publish/integration/
-
A good template to use as a reference is blueprint. You can generate a template similar to blueprint and customize it to your context by using cookiecutter-homeassistant-custom-component.
@eigendude Again, regardless of you creating an custom OASIS integration component or not, I highly recommend you look into migrating away from your Philips Hue Bridge and move to the native ZHA (Zigbee Home Automation) integration in Home Assistant:
- https://community.home-assistant.io/t/migrating-from-a-philips-hue-bridge-to-skyconnect/620862
While many Home Assistant users do have Philips Hue Bridge there are a lot more users who are using that native ZHA integration:
- https://analytics.home-assistant.io/integrations/
All need for ZHA (Zigbee Home Automation) is a Zigbee Coordinator USB adapter (Zigbee radio) + a long USB extension cable:
- https://community.home-assistant.io/t/zigbee-buyers-guide/654695
Suggest just buying the official Home Assistant Connect ZBT-1 (a.k.a. SkyConnect) because it is easiest to update and maintain:
- https://www.home-assistant.io/connectzbt1/
Best alternative option today is otherwise Sonoff ZBDongle-E and/or Sonoff ZBDongle-P.
-
https://sonoff.tech/product/gateway-and-sensors/sonoff-zigbee-3-0-usb-dongle-plus-e/
-
https://sonoff.tech/product/gateway-and-sensors/sonoff-zigbee-3-0-usb-dongle-plus-p/
I'd like to ditch the Hue Bridge for a more open (and far cheaper!) solution. It might be a good summertime project to switch my lighting infrastructure over to a new system.
My current goal is apartment-wide person detection to control the lights for when I do get them automated. I've scattered a combination of 10-year-old laptops with webcams, gaming cameras, and raspberry pi cameras around the place. I'm piping the video feeds to a central server doing pose detection on all images in parallel using MediaPipe. I have two detection tasks left before I start on lighting:
- Modify the Kinect2 driver to downscale the video
- Add infrared lighting so the NO-IR cameras work in the dark.
Here's a NO-IR raspberry pi camera I embedded in some LEGO infrastructure doing pose detection in realtime:
After detection is ready I'll work on lighting.
I'd like to ditch the Hue Bridge for a more open (and far cheaper!) solution. It might be a good summertime project to switch my lighting infrastructure over to a new system.
If you want an inexpensive dedicated home automation as an entry-level appliance entry-level that is super easy to get started with and can fully replace your Philips Hue Bridge then I strongly recommend buying the official Home Assistant Green in combination with their official Home Assistant Connect ZBT-1 (formerly "SkyConnect"). The main benefits with this is that it is very simple to maintain since it is a dedicated appliance, and with tha anyone can be up and running with their official built-in native ZHA (Zigbee Home Automation) integration within 30-minutes. That allows you to integrate all Zigbee compatible devices, so not just Philips Hue products, (as well as have the possibility of adding additional USB radio adapters connecting Thread-based Matter devices and Z-Wave devices too if you want later).
However, if what you are looking for a more flexible solution then suggest to run Home Assistant Operating System (Home Assistant OS) in a Virtual Machine (VM) on a more powerful computer/server (which is turn is running Proxmox Virtual Environment (PVE) virtualization hypervisor). If you go with that instead then you can pass-though USB radio adapters to the VM with Home Assistant OS. The main benefits with that is you can run a fully sandboxed operating-system in each virtual machine, (which can also make quick backup and retro easier if you use VM snapshots in combination with traditional backups).
Again, if you want a cheaper Zigbee USB radio adapter that is still great then buy the Sonoff ZBDongle-E and/or Sonoff ZBDongle-P:
- https://itead.cc/product/zigbee-3-0-usb-dongle/
- https://itead.cc/product/sonoff-zigbee-3-0-usb-dongle-plus/
PS: Another tip is that ITead that makes Sonoff devices have loads of cheap Zigbee devices and many other smart home products:
- https://itead.cc/smart-home/
OK, finished Kinect 2 support enough for pose detection in MediaPipe: https://github.com/eigendude/oasis_kinect2
I integrated HA into OASIS: https://github.com/eigendude/OASIS/tree/main/oasis_ha. I installed it on a spare RPi 4. I set up all the services provided out-of-the-box. It added my Hue devices and rooms.
Again, highly recommend you migrate from a Philips Hue Bridge to the native ZHA (Zigbee Home Automation) integration in Home Assistant
I looked into direct zigbee comms, and went with a Hue Hub, because while expensive it's high quality infrastructure. Being able to store state safely on the network is valuable. ZHA would need to keep state on-device, with all the difficulties that brings.
Home Assistant does however not yet have a Matter integration
I was disappointed when Matter came out. It sounds like it's far more dependent on the internet that I thought it would be.
I strongly recommend buying the official Home Assistant Green in combination with their official Home Assistant Connect ZBT-1 (formerly "SkyConnect").
If I knew about Home Assistant Green before I bought the Hue Bridge I'd definitely have gone with that. But now that I have the bridge, and it's a valuable device, I feel like I should continue using it.
Custom component integration for Home Assistant’s core (backend) would be very cool to see some day, and made available via publishing it on HACS (Home Assistant’s Community Store) to make it so that more people discover it
Now that I have HA setup I need to integrate it with the pose detector. The problem with a HA component is the ROS dependency, whereas a ROS component simply has a HA dependency.
I'll figure out the best way to integrate.
What integration ideas do you have?
It looks like MQTT is the protocol most suited for IoT. I found a ROS 2 MQTT bridge. I'll see if I can get it to work.
Believe that WebSocket API connection is usually prefer for real-time bi-direction connections to native Home Assistant integrations:
-
https://www.home-assistant.io/integrations/websocket_api/
- https://developers.home-assistant.io/docs/api/websocket/
Found these video tutorials on WebSocket API in Home Assistant via a quick search:
-
https://www.youtube.com/watch?v=Qk9A0QbG5-I
- https://quickstarts.postman.com/guide/home-assistant/index.html?index=..%2F..index#0
-
https://www.youtube.com/watch?v=Q5k2lj2iskw
-
https://www.youtube.com/watch?v=34yD15jmeEA
Note! They have an official "thick" JavaScript WebSocket client for Home Assistant that can maybe also be used as a reference(?):
https://github.com/home-assistant/home-assistant-js-websocket
REST (RESTful) is an alternative but understand that API is for more quick and dirty connections instead of a proper integration(?):
-
https://www.home-assistant.io/integrations/rest/
- https://developers.home-assistant.io/docs/api/rest/
Maybe could also check out ESPHome's Native API Component and ESPHome native API? It too has many advantages over MQTT:
-
https://esphome.io/components/api.html
- https://esphome.io/components/api.html#advantages-over-mqtt
Also, while very old and probably outdated today, but FYI, also had this old Home Assistant client for MicroPython as an example:
https://github.com/home-assistant/micropython-home-assistant
So while MQTT works too but it is more complex and requires depencey of having an MQTT Broker so it is less user-friendly to end-users. There are howver a few integrations for Home Assistant that support both MQTT and WebSockets in order to be more flexible, I think that the main benefit with MQTT is that you could make a general implementation on the client side so not Home Assistant specific, in case you want to consider the use case of having integrations to other smart home ecosystems and platforms as well.
PS: Also note that "Local Push" is preferred over "Local Polling" to enable real-time state changes, see their IoT classifications here:
- https://www.home-assistant.io/blog/2016/02/12/classifying-the-internet-of-things/#classifiers
Hey I need to connect my moorebot scout (ros1melodic) to my smart home to get images (Preferably in color) and send commands to and get data from to use in automatons.
Sorry, I don't support EOL versions of ROS.
Tip for directly connecting Zigbee based Philips Hue lights to Home Assistant is to buy a Silicon Labs EFR32MG21 (or EFR32MG13) based USB adapter (like Elelabs/Popp Zigbee USB stick) or a Texas Instruments CC2652P/CC2652R based Zigbee USB adapter (like "Itead Sonoff Zigbee 3.0 USB Dongle Plus") and connect that via a long external USB extension cable (to get it away from any electonics so they do not interfer with signals) and then use Home Assistant's native ZHA integration component -> https://www.home-assistant.io/integrations/zha/
I have renewed interest in connecting Hue to OASIS. I'll start looking for solutions again.
Oh shoot that stinks. Idk why it's so hard to find a little robot with a charging dock that I can integrate into the home assistant. All the robots I find are usually closed source and require some app. The appbot riley, the robo buddy, the moorebot scout has some stuff available but it's operating system is apparently ancient and I can't even install packages because the repositories don't exist, so I can't figure out how to add the sources list to allow the download of an mqtt client to connect to my home assistant mqtt broker. And I don't know if I'm knowledgeable enough to install it from the source and ensure I have the dependencies and won't accidentally brick my device or something stupid. It's unfortunate that the ones with ros2 pre installed have no built in charging contacts and charging dock for completely autonomous use. U know drive it, hit return, have it automatically return (like the scout) and integrate into the other smart technology. I really need to learn to code in c.
I wrote my own MQTT <-> ROS 2 bridge: https://github.com/eigendude/OASIS/blob/main/oasis_hass/oasis_hass/nodes/hass_bridge_node.py
The file automations.yaml file is entirely AI generated.
It works for my smart bulbs, but I think I can make it more generic using https://github.com/ika-rwth-aachen/mqtt_client. Home Assistant yaml is easy to generate with AI, so I can move more of the complexity into generated yaml.
I've given up on mqtt_client. I could use it to translate from MQTT to ROS messages to avoid an MQTT bridge, but I'd still need a ROS bridge to house the logic of processing HA MQTT Statestreams, and probably automation events in the other direction. As a result, I'd have two bridges, one for mqtt_client (MQTT to ROS) and one for home automation logic (ROS to ROS). If both nodes were in Python, it'd add a lot of latency. So if I had to start over, I'd do a C++ nodelet, with the mqtt_client bridge loaded in, and a ROS-to-ROS c++ nodelet doing the home automation logic. But I already have good Python code so I'll stick with my single bridge.
EDIT: I couldn't get mqtt_client to work as a component but I'm including it as a node, currently idle, but with the ability to add bridge comms later without the steep code requirement of the custom bridge.
Believe that WebSocket API connection is usually prefer for real-time bi-direction connections to native Home Assistant integrations:
https://www.home-assistant.io/integrations/websocket_api/
- https://developers.home-assistant.io/docs/api/websocket/
Found these video tutorials on WebSocket API in Home Assistant via a quick search:
https://www.youtube.com/watch?v=Qk9A0QbG5-I
- https://quickstarts.postman.com/guide/home-assistant/index.html?index=..%2F..index#0
https://www.youtube.com/watch?v=Q5k2lj2iskw
https://www.youtube.com/watch?v=34yD15jmeEA
Note! They have an official "thick" JavaScript WebSocket client for Home Assistant that can maybe also be used as a reference(?):
https://github.com/home-assistant/home-assistant-js-websocket
REST (RESTful) is an alternative but understand that API is for more quick and dirty connections instead of a proper integration(?):
https://www.home-assistant.io/integrations/rest/
- https://developers.home-assistant.io/docs/api/rest/
Did you also look at their native WebSocket and/or REST APIs?
REST is out, no streaming. I didn't think of checking WebSocket, wish I had. But I did MQTT with the "MQTT Statestream" integration for state streaming and "MQTT" integration for commands. It's working perfectly. I'm building control logic on top of it now.
I've also added more drivers for controlling stuff around the apartment: Wake-on-Lan, UPS monitoring/control, and VESA MCCS brightness of monitors
I'm really impressed with Home Assistant. I bought my friend a Home Assistant Green to support the project. He's got a bunch of expensive SRAM bike parts using the AXS protocol. We'll see how well it works with them.
REST is out, no streaming. I didn't think of checking WebSocket, wish I had. But I did MQTT with the "MQTT Statestream" integration for state streaming and "MQTT" integration for commands. It's working perfectly. I'm building control logic on top of it now.
I've also added more drivers for controlling stuff around the apartment: Wake-on-Lan, UPS monitoring/control, and VESA MCCS brightness of monitors
Nice, later you might still want to take another look at their WebSocket API as an alternative, because it is their preferred API and such is more developed, for this reason (and that it is more user friendly to end-users so need to MQTT Broker) there are a few integratioons that support both MQTT and WebSocket as options.
@eigendude Off-topic but if you like Home Assistant then you should also check out the ESPHome (microcontroller firmware) project which share many developers as is also part of the same non-profit Open Home Foundation:
-
https://esphome.io
- https://github.com/esphome
ESPHome is the go-to MCU firmware for users that also use Home Assistant since those projects can be very tightly integrated out-of-the-box so to speak.
I finished my HA integration.
First I made a Hue plug control power to one display, then used the MQTT bridge and a ROS node to also control power to all the other displays using DPMS and CEC. I still have two displays that I need smart plugs for. I was thinking of saving money and going with a non-hue plug. Ideas for plugs?
Second I made a lighting controller that cycles through the rainbow colors on a 25s timer. This was tricky, because Hue events are arriving late, out-of-order, with lots of jitter, noisy data, and sometimes not arriving at all. This lighting controller made me regret using Hue hardware everywhere. I definitely want to try other solutions before investing any more in Hue. Ideas for RGB lights?
My goal for the lighting controller was to still allow for the vanilla Hue app to control everything. I noticed that as I added stuff to HA, the UI got more complex. But no matter what I added to HA, the Hue app didn't get more complex (obviously). This was the goal. But I think the risk of the app getting shitty later is too high. Like, it's already warning me I need an email. Instead, I'll put effort into making a simple HA dashboard and ditch the Hue app. It'll be a maintenance burden to maintain the config, but I think it's the safer solution.
I'm now working on connecting the machine vision system to the lights. I'm currently writing the on-device image rectification for accurate pose positioning. Next I'll look into markered calibration of the camera system. And if that goes well then I'll do the occupancy registration, and connect that to light control.
My goal for the lighting controller was to still allow for the vanilla Hue app to control everything
If so then you must specifically buy products that are compatible with the Philips Hue Bridge as their app can not control any devices that are not directly connected to an official Philips Hue Bridge. Not a flexible solution at all as you can not add any other devices at all. I think you will regret going down that route.
I noticed that as I added stuff to HA, the UI got more complex.
That is only happens if you use their auto-generated dashboard as that still uses the old legacy style (which their developers working on replacing before the end of this year). Anyway, that is not what I would personally recommend which instead would be the official Home Assistant Companion app with your own custom dashboards as the you can use the new ”Sections” style instead (because the auto-generated one is still horrible as it uses the old style but it is now very easy to one yourself these days by selecting ”Sections” which is default when creating your own dashboard). Leason one is to not use the auto-generated dashboard and instead only add specific entities manually yourself, maybe start using only tile card(s) as it is very clean.
https://community.home-assistant.io/t/migrating-from-a-philips-hue-bridge-to-the-zha-zigbee-home-automation-integration/620862
If still want to keep exposing those lights as Hue devices to third-party ecosystems then you should be aware that you can use Home Assistant as a ”emulated Hue” bridge to expose any non-Hue lighting device as a Hue compatible product to Amazon Alexa and Google Home:
https://www.home-assistant.io/integrations/emulated_hue/
Ideas for RGB lights?
While it is true that are a lot of good Zigbee RGB controllers for lights available (see link below) but if you want to have the-best-of-the-best RGB light controller that is available today then you should actually check put the open-sourve WLED project instead and use Home Assistant as a Hue bridge if that is what you want, as FYI, WLED firmware on ESP32 chips make the best RGB controllers currently and you can make them yourself DIY or even buy finished products with WLED firmware that are sold by several sites online. Check out the WLED project:
https://kno.wled.ge
- https://github.com/wled/WLED
I still have two displays that I need smart plugs for. I was thinking of saving money and going with a non-hue plug. Ideas for plugs?
Check out this database of Zigbee devices (though note that it isncommunity maintained so is not fully up to date):
https://zigbee.blakadder.com
IKEA’s ”INSPELNING” (Zigbee smart plug with power meter) or IKEA’s ”TRETAKT” (Zigbee smart plug without power metering) is deffinitely best value and quality compromise.
The ”SMART+” series from Ledvance are also good, (which their older models that still had the OSRAM brand was not), and they also have
Another great value are the newer products/models ITead in the Sonoff series (just be careful to not buy their older models that shipped with old firmware as some of those had firmware issues and did not support OTA updates so firmware could not be updated to fix the firmware.
If you need something smaller in size then the Zigbee smart plug from Neo Coolcam is very slimmed down but is lower quality and read that it has had issues with accurate power measurements, (it is actually a rebranded white label product made by Tuya, who are kind of infamous for making poorly developed Zigbee firmware, so the same product is also available under many different brands which can have other firmware variants/versions).
Intro -> https://www.youtube.com/watch?v=Wo_xQ5m8rbU
WLED semi-DIY -> https://www.cnx-software.com/2025/05/01/adafruit-sparkle-motion-an-esp32-based-addressable-led-controller-with-four-outputs-100w-usb-c-power-and-wled-xlights-support/
WLED finished -> https://www.aliexpress.com/item/1005006625076701.html
Checking in with progress. I connected the machine vision system to the lights. I'm using OpenCV to calibrate and rectify all cameras. Poses are positioned in the X-Y camera space for now (e.g. turn on kitchen lights when pose is on the left 1/3 of the hallway camera feed).
I still have some performance improvements to make in the vision system. Then I'll make my next hardware purchase.
In the meantime, I'm working on connecting my EV to home assistant through DIMO, and bagging some sweet crypto in the process.
In the meantime, I'm working on connecting my EV to home assistant through DIMO, and bagging some sweet crypto in the process.
Suggest that you also check out the WiCAN project if interested in an open-source OBD-II canbus adapter for cars and trucks that is compatible with Home Assistant:
-
https://www.crowdsupply.com/meatpi-electronics/wican-pro
-
https://github.com/meatpiHQ/wican-fw
- https://github.com/jay-oswald/ha-wican
-
I still have two displays that I need smart plugs for. I was thinking of saving money and going with a non-hue plug. Ideas for plugs?
Another tip is that ITead has also released new Sonoff branded Zigbee smart plugs for some regions:
https://itead.cc/product/sonoff-iplug-zigbee-smart-plug-s60-series/
You can usually find coupons and/or get free shipping if buy more of their things:
https://itead.cc/?s=Zigbee&post_type=product&type_aws=true
I got distracted. I compiled mediapipe into a C++ node. Unfortunately its Abseil dependency can't work in ROS components. It fails to load a simple graph, so I'll leave this project for another time.
Only fixes, no new features to report, but I decided to add DeepSeek integration with an NVidia Spark