assistant-relay

assistant-relay copied to clipboard

I installed this on Home Assistant with this addon https://github.com/Apipa169/Assistant-Relay-for-Hassio. And it works just fine. But now I want to use another Google account. When I set it up, the Google Home's says it was successfully setup. And I can broadcast a message to all the Google Home's. But after one or 2 broadcasts it stops working. After a couple of minutes (like 10 or 15) I can send one or 2 broadcasts again. And then it stops working again. It seems pretty random to me. I already made 2 new Google accounts. But both the new Google accounts have this problem. With my own personal Google account is just works. But I want to use another account so I can use another name which is broadcasting.

See https://github.com/Apipa169/Assistant-Relay-for-Hassio/issues/22

See also: https://www.reddit.com/r/googlehome/comments/f1caz6/broadcast_limit_or_bug/?utm_source=share&utm_medium=ios_app&utm_name=iossmf

I having and intermittent problem with broadcasts being dropped recently with my Google Nest Hub. I have not been able to tie it down to what might be causing the problem. But I was able to confirm that it isn't a problem with GAR but with the display itself. I was able to prove this by trying to do a broadcast from my phone just after the one from GAR failed. it didn't play that broadcast either. So, if it's not accepting broadcasts from my phone, that would have nothing to do with the GAR. So, I would suggest that you try the same with your setup when you have a failed broadcast. It might be a problem with the speakers internet connection and not GAR.

Same problem for me

same issue here but only have one account and it's this one that have intermittent issue

If it fails via AR, can you immediately try it via your phone to see if that works please

I thought it was something with AR for a while till I tried this right after one was dropped. Turned out it failed from my phone too. So, either the speaker is losing internet connection or something is futz up in the Google Cloud somewhere. Not the fault of AR (at least in my case).

I have the same experience. I'm pretty sure it's not speaker related. Because if I use my personal account it works every single time. When I change it to my Home Assistant Google account the problem occurs. I triple checked all the settings. And they are definitely the same. So I think there is something wrong with the broadcast service from Google.

Do you have that account linked to your speakers as well? You have to invite that account to join your home and then you have to make sure the account is linked to all of your speakers. Otherwise, you won't get broadcasts.

Yes, I did. And I get the broadcast messages sometimes. So it is working on that speaker. The only thing I didn't do yet is to link al 5 Google Home's to the "Home Assistant Google account" I only linked one speaker to test. I'll try to link all 5 speakers to that Google account. Maybe that will help. I let you know when I did this.

I'm experiencing this issue on cast-web-api as well. Some things to note...

• The first time I broadcast a message (if some time has passed since a broadcast message) all devices broadcast the message
• The second time (if close to the first time) much fewer devices broadcast, sometimes only one
• The third or fourth time (if close to the first time) and every time after only one device broadcasts the message
• If I turn off the one device that is broadcasting the message a new single device takes over the subsequent broadcasts. If I turn the device back on when it finalizes boot it (the old device) and the new device broadcast once or twice together. The next broadcasts the new device stops and the old device resumes broadcasts as the only device that broadcasts
• It is not affected by whether or not the device is playing music either by itself or in a group.
• Broadcasts from a hardware device (like a physical Google Home) still work to all devices
• Broadcasts from my phone still work to all devices

Did one of the mDNS libraries upstream change? For me, I can continue to broadcast and the one device broadcasts. All other devices drop off. Maybe it has to do with the way that the library handles groups and/or large amounts of speakers. If I have time I will try and capture some of the mDNS/bonjour messages as I broadcast.

Also, probably worth noting, I have what some would consider a very large amount (29) of cast "devices" including 4 Android TV devices, 3 hardware displays (Lenovo and Google Home Hub), 4 Chromecast Audios, 2 JBL Link 20s, and a slew of Google Homes and Google Home Minis as well as 6 speaker groups. This issue may be related to groups or a large amount of devices. It may also be related to the way that non-Google devices (which aren't updated ever or as quickly) behave. The single device that continues to respond and broadcast on its own tends to be the Lenovo Display.

I don't think this has anything to do with an mDNS module is AR, since AR just calls the Google Assistant Service and issues a broadcast command.

Cast web API on the other hand uses mDNS as it's casting locally - a different solution and implementation.

If it is upstream, then it's inside the Google system and completely out of view. I wonder if this is something to do with the delay between devices.

How long are you leaving it before issuing another broadcast. Maybe they're rate limiting... 🤔

Fair enough. Then do they share any components? I can see this behavior in both. I can see worse behavior in cast-web-api by the way. In cast-web-api the thread hangs and no web response is given. In GAR at least a response is given over the node server.

Maybe they are rate limiting. 5 or so minutes is usually long enough time for all devices to broadcast again. How would I prove that is what is happening? How do the other speaker devices get the broadcast? Can I see that traffic somehow?

Also, if they are rate limiting, why? Don't the projects that are setup have limits in the 10k range of requests per hour or day or something like that? Wouldn't it show in the Google project space that it was being limited? And if it is being limited, why does the one device continue to work indefinitely? Maybe that makes sense if I understood how the protocol/communication worked.

I'm not overly familiar with castweb, but I didn't think it connected to the Google Assistant Service?

Should also mention there is no mDNS in AR

As for monitoring the traffic, it's all encrypted now. As soon as AR calls the assistant service, it's off into the world of Google, and then it comes back from them and broadcasts to your Google homes.

It doesn't go from AR to your device's directly.

cast-web has connected to Google Assistant for the last year or so.

Okay, so a request goes from AR to one of the devices on the network? And then from that device to Googoo's cloud? Or does it go from AR straight to cloud? If it went to a device locally I could understand why one device for me kept working locally. And then I'm assuming some type of subscription pattern lets all the other hardware devices know that a message is there. They pull it from the cloud and play it? So, I should be able to see at least encrypted traffic from each of the hardware devices when a broadcast happens up to a Google cloud address?

Would a library or could a library in AR be capable of making a device unsubscribe from broadcast messages? If not, then the problem is on Google's side and it probably means they are rate limiting or something along those lines.

Got a link to their setup process for this? I've had a look but can't see any mention of setting up Assistant.

It's likely that Google utilise some form of socket system, so the devices sit there listening for a message.

I can't see anyway that you could unsubscribe a device from broadcasts. There's no way to control DnD, Digital Wellbeing or the device directly.

He put all of the information in the SmartThings app code. This is from Hubitat but it's because I ported it over. I don't have ST running anymore.

That link opens this popup:

From there, you can follow the linked guide which is this guide: https://developers.google.com/assistant/sdk/guides/service/python/embed/config-dev-project-and-account

You'd have to look in his repo to see which library he is using though for the assistant communication.

The reason I wondered if we are forcing an unsubscribe or maybe breaking the socket connection is because of the way that cast groups work. With cast groups one of the devices in the groups acts as a virtual device and chooses a random port in a range to emulate another hardware device. This way apps and things that are looking locally for groups find them.

Cast-web, if you cast to groups too frequently, causes so many mDNS updates (not from the node server itself but from all of my 29 devices) that some drop out and form a new virtual device for the speaker group. When I look from Spotify at this point I can see two of the speaker group present. A short time later, everything settles down and they all come back under one or the other.

So, I didn't know if maybe some action overloads a device or devices in the same way, they have to stop to reset, etc. and while they are settling back into place they are missing the frequent broadcasts.

All guess work though.

If AR is talking directly to Google, at the AR level is there a way to see the response of what is coming back from AR? Or is that libraries below? Maybe we would see in the response that rate limiting was happening? Like, in the first response in JSON we saw the result was "{"result": "okee dokee"}", the second request the response was "hmm..." and the third request is like... "whoa, whoa, whoa". Not exactly those responses obviously but you know what I mean I hope.

I don't think there's a response from Google on a broadcast. You basically start a conversation with the Assistant Service, and it responds accordingly. So a broadcast is no different to asking it a question, it just changes the text parameter that's passed to it.

Let me have a look at the code, see if there's anything that is ignored on a broadcast we could use.

Edit: nope, they all use and populate the same response data as you can see here: https://github.com/greghesp/assistant-relay/blob/master/relay/routes/index.js#L81

So, I'm nearly 100% convinced it's on the Google side at this point. I was dorking around just now and I discovered that if I add a second OAuth user to AR (my primary Google account in my case) and I use it to broadcast instead of my previous junk email "home" Google account it works every time.

Google must be doing some rate limiting based on account. I know that they have already classified my junk email account as a duplicate account in some regards. It's not listed as a recovery account for anything and in some services, like Google Voice, I remember Google actually giving me an error message that it couldn't be used or something to that effect because they knew it wasn't my primary. Maybe that's based on which accounts are signed into a browser session.

At any rate, when using my junk Google account I was limited. When using my primary Google account I never hit such a limitation. Maybe I've just used my junk Google account too frequently for broadcast and my primary will get limited as well eventually.

Either way, I think we can say this is upstream and the workaround is to switch Google accounts. Can anybody else experiencing the issue confirm this also works for them?

Now I'm starting to wonder if I'm going to have to write user pooling into the driver so that with every broadcast it randomly or sequentially chooses from a pool so we don't exhaust and hit limiting on a given user. I'm really glad, at this point, that you added the multi-user ability to AR.

Wow, crickets. Anybody? Can anybody else experiencing the issue described here confirm that switching to a different Google account resolves the issue?

I think Google are rate limiting. Someone on the SmartThings forum tested it with their phone and had exactly the same issue.

Can you upvote and comment here please: https://support.google.com/assistant/thread/31597536?hl=en

In true Google fashion they have locked the thread for comment and there is no recourse for remediation. I hate Google. They just mow over you and there is nothing you can do to fix their issues or change their policy or talk to anybody who can.

Interestingly enough I hadn't experienced this in many months. It came back today or yesterday. Google is up to some changes again I guess.