cync2mqtt icon indicating copy to clipboard operation
cync2mqtt copied to clipboard

KeyError: 'bulbs'

Open CaptainN opened this issue 1 year ago • 21 comments

Hi! I followed the guides for both venv and docker, and both lead me to the same error. Any idea what's up?

kevin@ha:~$ ~/venv/cync2mqtt/bin/cync2mqtt  ~/cync_mesh.yaml
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/kevin/venv/cync2mqtt/bin/cync2mqtt", line 378, in run_in_subprocess
    asyncio.run(cm.run_mqtt(),debug=(args.log_level.upper()=='DEBUG'))
  File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/kevin/venv/cync2mqtt/bin/cync2mqtt", line 288, in run_mqtt
    self.meshnetworks.populate_from_configdict(self.configdict)
  File "/home/kevin/venv/cync2mqtt/lib/python3.9/site-packages/acync/__init__.py", line 140, in populate_from_configdict
    for bulb in mesh['bulbs'].values():
KeyError: 'bulbs'

I appreciate any help!

CaptainN avatar Sep 27 '24 00:09 CaptainN

Can you send the content of cync_mesh.yaml (with the MAC addresses/and keys XX out)?

I'll take a look.

juanboro avatar Sep 27 '24 22:09 juanboro

Here is is:

meshconfig:
  1226357509:
    access_key: 777
    mac: XXXXXXX
    name: XXXXXXX
  1286718809:
    access_key: 777
    mac: XXXXXXX
    name: XXXXXXX
  1333061485:
    access_key: 777
    mac: XXXXXXX
    name: XXXXXXX
  1409876883:
    access_key: 777
    mac: XXXXXXX
    name: XXXXXXX
  1601122146:
    access_key: 777
    mac: XXXXXXX
    name: XXXXXXX
  1613912743:
    access_key: XXXXXXX
    bulbs:
      1:
        mac: XXXXXXX
        name: Bathroom Light North East
        supports_rgb: true
        supports_temperature: true
      2:
        mac: XXXXXXX
        name: Basement Light Stairs
        supports_rgb: true
        supports_temperature: true
      3:
        mac: XXXXXXX
        name: Basement Light 3
        supports_rgb: true
        supports_temperature: true
      4:
        mac: XXXXXXX
        name: Basement Light Sout East
        supports_rgb: true
        supports_temperature: true
      5:
        mac: XXXXXXX
        name: Kitchen Floodlight 2
        supports_temperature: true
      6:
        mac: XXXXXXX
        name: Kitchen Floodlight 1
        supports_temperature: true
      7:
        mac:XXXXXXX
        name: Basement Light 5
        supports_rgb: true
        supports_temperature: true
    mac: XXXXXXX
    name: Home
  1808127672:
    access_key: 777
    mac: XXXXXXX
    name: XXXXXXX
  1912083427:
    access_key: 777
    mac: XXXXXXX
    name: XXXXXXX
mqtt_url: mqtt://kevin:[email protected]:1883/

Thanks again!

CaptainN avatar Sep 28 '24 07:09 CaptainN

I edited out the stuff that doesn't look like anything your example, and then got a different message - simply, unable to connect to mesh.

acync.mesh - INFO - Unable to connect to mesh mac:

Bummer.

The updated config:

meshconfig:
  1613912743:
    access_key: XXXXXXX
    bulbs:
      1:
        mac: XXXXXXX
        name: Bathroom Light North East
        supports_rgb: true
        supports_temperature: true
      2:
        mac: XXXXXXX
        name: Basement Light Stairs
        supports_rgb: true
        supports_temperature: true
      3:
        mac: XXXXXXX
        name: Basement Light 3
        supports_rgb: true
        supports_temperature: true
      4:
        mac: XXXXXXX
        name: Basement Light Sout East
        supports_rgb: true
        supports_temperature: true
      5:
        mac: XXXXXXX
        name: Kitchen Floodlight 2
        supports_temperature: true
      6:
        mac: XXXXXXX
        name: Kitchen Floodlight 1
        supports_temperature: true
      7:
        mac:XXXXXXX
        name: Basement Light 5
        supports_rgb: true
        supports_temperature: true
    mac: XXXXXXX
    name: Home
mqtt_url: mqtt://kevin:[email protected]:1883/

CaptainN avatar Sep 28 '24 07:09 CaptainN

Yeah - that seems like it should have worked. I plan to do a bunch of updates hopefully soon to add more robust connection options. One thing to try is see if adding usebtlib: bluepy to the yaml makes a difference.

juanboro avatar Oct 03 '24 01:10 juanboro

I did try bluepy, it doesn't seem to make a difference. I ordered an outdated G by CE bulb from ebay - hoping that'll make connecting to the mesh a bit easier. Will let you know how it goes!

CaptainN avatar Oct 03 '24 15:10 CaptainN

I got the old bulb, still can't connect. I wonder if my bluetooth dongle is the issue.

CaptainN avatar Oct 06 '24 01:10 CaptainN

I'm getting the same error, here is my yaml, seems pretty identical. I'm using Intel BT if that helps at all!

meshconfig:
  316356668:
    access_key: XXXXX
    mac: XXXXX
    name: XXXXX
  441483036:
    access_key: XXXXX
    mac: XXXXX
    name: XXXXX
  1364391871:
    access_key: XXXXX
    bulbs:
      1:
        mac: XXXXX
        name: Ceiling Fan 1
        supports_rgb: true
        supports_temperature: true
      2:
        mac: XXXXX
        name: Ceiling Fan 2
        supports_rgb: true
        supports_temperature: true
      3:
        mac: XXXXX
        name: Ceiling Fan 3
        supports_rgb: true
        supports_temperature: true
    mac: XXXXX
    name: Home
  1882501669:
    access_key: XXXXX
    mac: XXXXX
    name: XXXXX
mqtt_url: mqtt://127.0.0.1:1883/

Adding usebtlib: bluepy didn't help also, definitely a strange error.

wearrrrr avatar Oct 26 '24 04:10 wearrrrr

If it helps, I'm using a "Full Color Direct Connect Smart Bulb (A19)" on firmware version 1.0.372

wearrrrr avatar Oct 26 '24 04:10 wearrrrr

I'm going to try to explore some alternatives this week for these issues. Only one other thing I can think of for the above issues is to try is to set the --log-level switch to DEBUG and see more what is going on (or you could try the cloud/lan based alternatives: https://community.home-assistant.io/t/cync-integration/428815).

For cync-BLE... I'm going to try to aim for implementing an alternative hardware solution with really cheap hardware and a more reliable BLE stack. As it is - my current set it and forget it setup is running on pretty cheap hardware reliably --- just a raspberry pi-zero 2w connected to my router in the basement via usb (actually - it isn't quite as cheap anymore :-( --- but it was when I bought it at microcenter a few years ago). Unfortunately, I don't have many Cync devices to work with --- I build my new home mainly around tuya lights (most of which hopefully i can get around to custom flash the firmware for all of them at some point).

juanboro avatar Dec 16 '24 13:12 juanboro

Turns out you just have to increase the connection timeout for the Bluetooth mesh... After reformatting the YAML file to match what the code was expecting, increasing the connection timeout to 30 seconds lets it actually connect to the mesh. https://github.com/juanboro/cync2mqtt/commit/fb589d3eb514ea5488c029bfae760c5875bc15b1

Image

Though, this still sometimes fails... just seems to increase the chances of it actually connecting to something. It took a few devices to connect here:

Image

Lioncat6 avatar Feb 27 '25 03:02 Lioncat6

  340198845:
    access_key: XXX
    mac: 786DEBB842D8
    name: 786DEBB842D8
  391722853:
    access_key: XXX
    mac: 786DEBBB0C72
    name: 786DEBBB0C72
  523955445:
    access_key: XXX
    mac: 786DEBBD22AA
    name: 786DEBBD22AA
  1299658474:
    access_key: XXXXXX
    bulbs:
      1:
        is_plug: true
        mac: 78:6D:EB:07:61:19
        name: IR Blaster
      2:
        mac: 88:D0:39:5C:81:39
        name: Remote
      3:
        mac: F4:BC:DA:73:64:12
        name: Roof Light 2
        supports_rgb: true
        supports_temperature: true
      4:
        mac: 78:6D:EB:B8:42:EC
        name: Nightstand Lamp
        supports_rgb: true
        supports_temperature: true
      5:
        mac: F4:BC:DA:74:63:AB
        name: Dresser Lamp
        supports_rgb: true
        supports_temperature: true
      6:
        mac: F4:BC:DA:74:72:CF
        name: Roof Light 1
        supports_rgb: true
        supports_temperature: true
      7:
        mac: F4:BC:DA:70:1B:27
        name: Light strip
        supports_rgb: true
        supports_temperature: true
      8:
        mac: 30:C0:1B:28:EA:C0
        name: Dimmer
      9:
        mac: 78:6D:EB:BB:0C:8A
        name: Bathroom Light 1
        supports_rgb: true
        supports_temperature: true
      10:
        mac: 78:6D:EB:BA:9B:39
        name: Bathroom Light 2
        supports_rgb: true
        supports_temperature: true
      11:
        mac: 78:6D:EB:BD:22:BE
        name: Bathroom Light 3
        supports_rgb: true
        supports_temperature: true
      12:
        mac: F4:BC:DA:30:1B:CF
        name: Bathroom Switch
    mac: CDFC0CCB42DF
    name: Home
  1322836220:
    access_key: XXX
    mac: 786DEBBA9B2F
    name: 786DEBBA9B2F
mqtt_url: mqtt://cynclights:[email protected]:1883/

Was reformatted to

  1299658474:
    usebtlib: bluepy
    access_key: XXXXXX
    bulbs:
      1:
        is_plug: true
        mac: 78:6D:EB:07:61:19
        name: IR Blaster
      2:
        mac: 88:D0:39:5C:81:39
        name: Remote
      3:
        mac: F4:BC:DA:73:64:12
        name: Roof Light 2
        supports_rgb: true
        supports_temperature: true
      4:
        mac: 78:6D:EB:B8:42:EC
        name: Nightstand Lamp
        supports_rgb: true
        supports_temperature: true
      5:
        mac: F4:BC:DA:74:63:AB
        name: Dresser Lamp
        supports_rgb: true
        supports_temperature: true
      6:
        mac: F4:BC:DA:74:72:CF
        name: Roof Light 1
        supports_rgb: true
        supports_temperature: true
      7:
        mac: F4:BC:DA:70:1B:27
        name: Light strip
        supports_rgb: true
        supports_temperature: true
      8:
        mac: 30:C0:1B:28:EA:C0
        name: Dimmer
      9:
        mac: 78:6D:EB:BB:0C:8A
        name: Bathroom Light 1
        supports_rgb: true
        supports_temperature: true
      10:
        mac: 78:6D:EB:BA:9B:39
        name: Bathroom Light 2
        supports_rgb: true
        supports_temperature: true
      11:
        mac: 78:6D:EB:BD:22:BE
        name: Bathroom Light 3
        supports_rgb: true
        supports_temperature: true
      12:
        mac: F4:BC:DA:30:1B:CF
        name: Bathroom Switch
    mac: CDFC0CCB42DF
    name: Home
mqtt_url: mqtt://cynclights:[email protected]:1883/

I might make a pull so the code ignores entities without the 'bulbs' field

Lioncat6 avatar Feb 27 '25 03:02 Lioncat6

Thought I'd try this out again, see what I get. I'm running this on Bazzite, from within a podman container. I think I got the settings right - but it's an additional potential complication.

04:44:30 bazzite@iktomi-bazzite cync2mqtt ±|main|→ podman start -ai cync2mqtt
2025-05-31 20:44:41,743 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:77:46:B8
2025-05-31 20:45:12,117 - acync.mesh - INFO - Unable to connect to mesh mac: 78:6D:EB:77:46:B8, Error: 
2025-05-31 20:45:12,218 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:C5:67:DB
2025-05-31 20:45:12,458 - acync.mesh - INFO - Unable to connect to mesh mac: 78:6D:EB:C5:67:DB, Error: device 'dev_78_6D_EB_C5_67_DB' not found
2025-05-31 20:45:12,558 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:6B:D6:6D
2025-05-31 20:45:12,629 - acync.mesh - INFO - Unable to connect to mesh mac: 78:6D:EB:6B:D6:6D, Error: device 'dev_78_6D_EB_6B_D6_6D' not found
2025-05-31 20:45:12,730 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:C4:74:CA
2025-05-31 20:45:15,999 - acync.mesh - INFO - Unable to connect to mesh mac for notify: 78:6D:EB:C4:74:CA - [org.bluez.Error.NotPermitted] Write not permitted
2025-05-31 20:45:17,775 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:12:0B:F2
2025-05-31 20:45:51,496 - acync.mesh - INFO - Unable to connect to mesh mac for notify: 78:6D:EB:12:0B:F2 - [org.bluez.Error.Failed] Operation failed with ATT error: 0x0e (Unlikely Error)
2025-05-31 20:45:53,667 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:13:A0:B7
2025-05-31 20:46:29,074 - acync.mesh - INFO - Unable to connect to mesh mac for notify: 78:6D:EB:13:A0:B7 - [org.bluez.Error.Failed] Operation failed with ATT error: 0x0e (Unlikely Error)
2025-05-31 20:46:31,167 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:71:AB:4B
2025-05-31 20:47:01,676 - acync.mesh - INFO - Unable to connect to mesh mac: 78:6D:EB:71:AB:4B, Error: 
2025-05-31 20:47:01,777 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:29:2F:7C
2025-05-31 20:47:31,818 - acync.mesh - INFO - Unable to connect to mesh mac: 78:6D:EB:29:2F:7C, Error: Device with address 78:6D:EB:29:2F:7C was not found.
2025-05-31 20:47:31,919 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:C4:74:CA
2025-05-31 20:47:32,034 - acync.mesh - INFO - Unable to connect to mesh mac: 78:6D:EB:C4:74:CA, Error: device 'dev_78_6D_EB_C4_74_CA' not found
2025-05-31 20:47:32,135 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:12:0B:F2

CaptainN avatar May 31 '25 20:05 CaptainN

Only thing I can think of right now is to maybe reduce it just down to the closest bulbs to the bluetooth adapter by setting the others to priority: -1 in the mesh config. Those write not permitted errors seem like I remember my direct connect bulbs doing without using bluepy. Bluez/bluetooth on linux just seems flaky in general with this stuff - alot of interdependencies. The best luck I've had with the limited cync bulbs I have left right now has been with my Pi Zero 2W connected directly to me home router (and pretty much the only thing it does now is run this).

juanboro avatar Jun 13 '25 00:06 juanboro

I finally figured out how to get this running in a local HA add-on (easy to do, hard to figure out lol). Sadly, I get the same error:

2025-09-30 17:04:55,225 - acync.mesh - INFO - Attempting to connect to mesh mac: 78:6D:EB:6B:D6:6D
2025-09-30 17:05:02,981 - acync.mesh - INFO - Unable to connect to mesh mac for notify: 78:6D:EB:6B:D6:6D - [org.bluez.Error.NotPermitted] Write not permitted

I'm happy to share how I did get that working, if anyone is interested. I basically had ChatGPT tell me how to do it. It also suggested a few possible reasons for it, including that the mesh is already bound to something else. I'm going to try again later though - I have one of those old pre-direct-connect bulbs in my mesh, I just have to remember which mac it is, and see if I can directly pair to that with bluetoothctrl. Will keep you posted!

CaptainN avatar Sep 30 '25 17:09 CaptainN

@CaptainN Would you mind sharing? Getting it running entirely local on one device would be amazing.

Lioncat6 avatar Oct 19 '25 21:10 Lioncat6

I actually just used ChatGPT to guide me through it - but it's essentially a local add-on, with the files I need (the cync configs) transfered over SAMBA, and CLI access through proxmox's root access. (You can alternatively use one of the SSH add-ons, if you are using HAOS).

Here is a summary that I had ChatGPT create:

Here’s a recap of the whole flow that got your Cync2MQTT integration running inside Home Assistant OS (HAOS) as a local add-on, even though HAOS normally hides Docker:


🧩 1 | Set up a local add-on skeleton

Inside your HA Samba share you created the directory \\<HA_IP>\addons\local\cync2mqtt\ (which maps to /root/addons/local/cync2mqtt in HAOS).

That folder ended up containing:

addons/
└── local/
    └── cync2mqtt/
        ├── Dockerfile
        ├── config.yaml
        ├── run.sh
        ├── pyproject.toml
        └── uv.lock        (optional)

⚙️ 2 | Dockerfile — a Python venv image built with uv

We built our own container on top of the HA base image:

ARG BUILD_FROM=ghcr.io/home-assistant/amd64-base:latest
FROM ${BUILD_FROM}

RUN apk add --no-cache \
    python3 python3-dev musl-dev git curl ca-certificates jq \
    bluez bluez-dev glib glib-dev build-base linux-headers pkgconf

RUN curl -LsSf https://astral.sh/uv/install.sh | sh
ENV PATH="/root/.local/bin:${PATH}"

WORKDIR /app
COPY . /app/

RUN uv venv
RUN test -f uv.lock || uv lock \
      --index https://pypi.org/simple \
      --extra-index-url https://wheels.home-assistant.io/musllinux-index/ \
      --index-strategy unsafe-best-match
RUN uv sync --frozen --no-cache

ENV PATH="/app/.venv/bin:${PATH}"
CMD ["/run.sh"]

This installs BlueZ + GLib dev libs (for bluepy), creates a uv virtual env, and installs the upstream juanboro/cync2mqtt package.


🧾 3 | Add-on metadata and permissions

config.yaml:

name: "Cync2MQTT"
slug: "local_cync2mqtt"
description: "Connects Cync BLE mesh lights to MQTT"
version: "0.1.0"
startup: services
host_dbus: true
udev: true
arch:
  - amd64
options: {}
schema: {}

Those flags (host_dbus: true, udev: true) let the container use HAOS’s Bluetooth adapter through BlueZ D-Bus.


📦 4 | Deploy and build through the Add-on Store

In HA → Settings → Add-ons → Add-on StoreLocal add-ons, you saw Cync2MQTT appear and clicked Rebuild / Install.


📁 5 | Provide your mesh configuration

You used the official Cync2MQTT container on another computer to generate cync_mesh.yaml, then copied it via Samba to:

\\<HA_IP>\config\cync2mqtt\cync_mesh.yaml

(i.e. /config/cync2mqtt/cync_mesh.yaml inside HA.)

Your final YAML only kept the real mesh section:

mqtt_url: mqtt://USER:PASS@core-mosquitto:1883
bluetooth_adapter: hci0
meshconfig:
  1613912743:
    access_key: XXXXXX
    mac: 4E9512C41499A689
    name: Home
    bulbs:
      - mac: 78:6D:EB:77:46:B8
        name: Bathroom Light NE
        supports_rgb: true
        supports_temperature: true
      # …more bulbs…

▶️ 6 | Restart and monitor logs

You controlled the add-on through the HA CLI (or Proxmox console):

ha addons restart local_cync2mqtt
ha addons logs local_cync2mqtt --follow

The container launched, connected to BlueZ via D-Bus, and attempted to connect to each bulb.


🔧 7 | Bluetooth debugging and pairing

Because BlueZ sometimes denied writes (org.bluez.Error.NotPermitted), you:

  1. Logged into the HAOS host (ha > login)

  2. Used bluetoothctl to

    power on
    scan on
    connect <bulb-mac>
    pair <bulb-mac>
    trust <bulb-mac>
    disconnect <bulb-mac>
    
  3. Restarted the add-on, confirming encrypted writes worked.


✅ 8 | Result

You now have:

  • A fully self-built Home Assistant add-on running cync2mqtt in Docker.
  • Your Bluetooth dongle shared from HAOS into the container via D-Bus.
  • Your mesh configuration file (cync_mesh.yaml) stored under /config/.
  • Logs and control through normal HA CLI (ha addons …).

This setup mirrors an external docker-compose deployment but stays 100 % inside HAOS, no extra VM or host Docker needed.


There's remnants of a bluetoothctl bit of sleuthing I did to try and force it to connect to my mesh. It says, "Because BlueZ sometimes denied writes (org.bluez.Error.NotPermitted)" - but actually, that is ALWAYS the case, not sometimes. I'm able to get it to connect to single bluetooth devices (pair) but I'm not yet able to access the proprietary cync mesh. Based on that - I actually wonder if it's the BTLE switch I have in the mesh that is keeping it from working.

CaptainN avatar Oct 20 '25 17:10 CaptainN

Nice! I used to run cync2mqtt on the same machine as HA Supervised (until they decided to deprecate Supervised). Now, I can try a 100% local installation again. Thanks!

Here's some issues I ran into with a Supervised installation that might be useful for those attempting a local add-on:

  • If Bluetooth isn't required for anything else besides Cync lights, disable HA's Bluetooth integration so that Cync has exclusive access to the bluetooth device. If nothing else, it will save quite a few CPU cycles because the Bluetooth integration is a pig.
  • Use 'Priority' in cync_mesh.yaml to disable Wire-Free switches/remotes (-1) because they are offline 99.99% of the time and to prioritize scanning (0-99):
     3:
        # Device Type: 0x70
        # Wire-Free Dimmer Switch
        # fw Version: 2.3.054
        mac: 88:D0:39:xx:xx:xx
        name: Desk Lamps Dimmer
        priority: -1
      4:
        # Device Type: 0x09
        # Soft White Smart Bulb (A19)
        # fw Version: 3.8
        mac: A4:C1:38:xx:xx:xx
        name: Night Light
        priority: 0
      5:
        # Device Type: 0x80
        # Soft White Direct Connect Smart Bulb (A19)
        # fw Version: 1.0.276
        # wifi mac: 78:6D:EB:xx:xx:xx
        mac: 78:6D:EB:xx:xx:xx
        name: Closet Light
        priority: 1

johnrosshunt avatar Oct 20 '25 18:10 johnrosshunt

Oh neat, I'll try that priority thing when I get a change.

CaptainN avatar Oct 20 '25 20:10 CaptainN

2025-09-30 17:05:02,981 - acync.mesh - INFO - Unable to connect to mesh mac for notify: 78:6D:EB:6B:D6:6D - [org.bluez.Error.NotPermitted] Write not permitted

I'm still looking into if there's anything I can do for this... bluepy completely broke in my current setup and won't connect to anything... and bluez still is pretty picky with what it will connect with.

juanboro avatar Oct 20 '25 23:10 juanboro

@CaptainN Sorry for the trouble, but do you mind making a quick repo with all the files in it, as the GPT summary seems to be missing a few...

Looks really promising, though

Lioncat6 avatar Oct 21 '25 00:10 Lioncat6

Yeah, it didn't get a few things right - I had to figure them out. I'll try to get something more repeatable when I get a chance. (First I have to remember how it works, heh.)

CaptainN avatar Oct 21 '25 15:10 CaptainN