How to properly handle speaker groups in multi-room audio systems

[klada]

Context

Home Assistant supports a variety of multi-room audio systems, such as Bluesound, HEOS and Sonos. Bluesound and Sonos already have support for speaker groups, for HEOS I started working on a PR (https://github.com/home-assistant/core/pull/32568). Speaker groups are mainly used for perfectly synced audio playback across multiple speakers.

Bluesound and Sonos have solved the grouping of speakers this way:

• Every speaker is represented by a media player entity so it can be controlled individually
• There is a service call bluesound.join/sonos.join for creating a new group or joining an existing speaker group. The media player entities for the desired speakers need to be passed in.
• There is a service call bluesound.unjoin/sonos.unjoin for removing a speaker (media player entity) from a group.
• Every media player entity has an attribute bluesound_group/sonos_group, which is a list of media player entities which are grouped together. The first entity is the master speaker (group coordinator).
• A speaker group can be controlled through any of it's media player entities.

There already is a very nice custom Lovelace card (https://github.com/kalkih/mini-media-player) which uses these grouping capabilities.

What I did with the HEOS component is implementing the very same interface that Bluesound and Sonos expose, so the custom Lovelace card mentioned above is also working with HEOS groups.

However the integration maintainer @andrewsayre raised concerns whether this is the right approach. He suggests that speaker groups should be exposed as separate entities. It looks like there actually would be some work in the HASS core required for this, that's why I am raising this as an arch issue right here.

Pros of speaker groups as separate entities:

• Volume control of groups would be more natural (if group volume is supported by hardware, otherwise individual speaker volumes would need to be adjusted)
• Explicit playback control on group (group playback control is also possible on individual player entities ATM)

Cons of speaker groups as separate entities:

• Speaker groups are very dynamic and added/destroyed on the fly depending on which speakers the user wants to group for a specific scenario. There is not just one single speaker group entity. There are times when there is only one speaker group with all speakers and sometimes there might be many smaller speaker groups.
• ~~Possibly a breaking change for Bluesound and Sonos, if we want to maintain the same interface across all multi-room audio integrations (which would be desirable IMHO)~~ [not relevant for arch issue]
• ~~Breaking change for mini-media-player~~ [not relevant for ach issue]
• Even if group volume control is missing, the speaker volumes can still be adjusted individually through their media player entities

Proposal

There are two main questions:

1. Do we want to try to maintain a matching API (service calls, attributes) for all multi-room audio integrations?
2. Do we need separate entities for representing grouped speakers or is the current approch of Bluesound/Sonos with the bluesound_group/sonos_group attribute on each media player entity sufficient?

Consequences

See Pros/Cons above

[rytilahti]

I think it makes at least sense to discuss whether having a standardized interface for grouping media players is a good idea. Besides sonos, heos, and bluesound, there are also plenty of other solutions that could use grouping, for example some AVR devices (e.g., https://helpguide.sony.net/ha/strdn108/v1/en/contents/TP0001224849.html) have "zones".

A couple of points:

• If the discussion with the integration maintainer was done in public, maybe add link that for some more background information for others to read?
• The second and third point (breaking current API/impls) is not that relevant for this (conceptual) discussion, IMO, as those are already non-standard interfaces which can break any day anyway.
• I don't understand what's the last con, care to elaborate on that?

A couple of pros:

• Volume adjustments. I assume that changing the group volume will cause relative changes on speakers, while adjusting the speaker-specific volume will affect only the volume of that speaker. I'm not sure how this currently work in sonos as I'm personally synchronizing the volumes, but I can see that could be a useful feature for some.
• Source changes. If I change a source for my living room speaker, it doesn't necessarily mean that I want to change it for rest of the speakers. Having a possibility to control a group of speakers would make the choice explicit.

[andrewsayre]

Adding @amelchio (sonos maintainer) to the discussion. We had talked (probably a year ago) about how groups were "hacked" into the sonos component because there wasn't a better way at the time to handle it. Curious to get your input on this arch issue.

An additional pro:

• It's not just about volume control. heos and possibly others, already represent groups as first-class discrete items in their API and do exactly as @rytilahti mentioned. Features include relative volume adjustment, mute/unmute, source selection, and queue management. This ensures that control of the group is not conflated with control of the "lead" speaker.

Some other thoughts:

• Why is the dynamic nature of a group a con? I would also point out that the volatility is based on usage -- some people keep speakers grouped in a more static nature. Additionally, the speakers themselves are dynamic, for example if you turn off or connect a new heos device, the representing entities are removed/created in HA in real-time.
• I don't understand this con: "Even if group volume control is missing, the speaker volumes can still be adjusted individually through their media player entities".
• Agree with @rytilahti that preserving unofficial cards and non-standard interfaces is not a con. It would actually be a consequence (and resulting pro): they would need to be updated and aligned with a standard architecture which improves support, reduces future breaking changes, and improves maintainability.

[tmjo]

Great discussion! The missing grouping feature for HEOS is deeply missed, and I've worked together with @klada on the working-but-not-so-perfect group feature for HEOS as a custom component. As far as I can see it is working quite well - also based on review from others - but as initially disclaimed, we were hoping that more experienced developers would contribute to make it perfect.

I believe the PR proposed by @klada would do the trick in a similar way as it does for Sonos and Bluesound, but nothing would make me happier than some pro's doing an even better job on the overall architecture behind it. However, stopping HEOS from getting a working function while waiting for this clarification seems like a mistake to me. The HEOS grouping is working, merely utilizing @andrewsayre 's excellent work on the pyheos, so why don't adopt it while awaiting a clarification of the principals on grouping? Don't get me wrong, I'm applauding such an improvement, but I would urge not to postpone a working feature for an integration based on principle discussions on how it should work while other integrations seems to be very happy with what they already have.

PS! I'm not a pro, but I'll be happy to contribute on any future work on these issues.

PPS! @andrewsayre since you're referring to the grouping features of sonos being "hacked" into the integration, I hope you're not taking any offence by us trying to "hack" it into the HEOS-integration. Speaking for myself, I'm just an amateur, but I was putting on a great smile (and still smiling) when the hack was finally working (although only as a custom component yet). Thanks for your great work on the pyheos!

[klada]

If the discussion with the integration maintainer was done in public, maybe add link that for some more background information for others to read?

The discussion is in the HEOS PR linked in the issue.

I don't understand what's the last con, care to elaborate on that?

Volume control is a pro fact for group entities. However speaker group entites are not a hard requirement for volume control, as volume can also be set on the individial player entities (or maybe even through a service call). That's why I listed the fact that group entities are not required for volume control as a con for group entities.

• It's not just about volume control. heos and possibly others, already represent groups as first-class discrete items in their API and do exactly as @rytilahti mentioned. Features include relative volume adjustment, mute/unmute, source selection, and queue management. This ensures that control of the group is not conflated with control of the "lead" speaker.

This is correct. However source selection, playback control and queue management can be controlled through any group member, not just the lead speaker (at least in HEOS).

Why is the dynamic nature of a group a con? I would also point out that the volatility is based on usage -- some people keep speakers grouped in a more static nature.

First of all there are a few issues which would make a group implementation very hard:

• A speaker can only be a member of one single group. This is different from all other integrations which have grouping concepts (such as ZigBee lights).
• As soon as the lead speaker is removed from a group (or moved to a different group) the entire group is destroyed. In HEOS it means it's group ID is lost.
• At least in HEOS there cannot be groups with only one member

And yes, these might not be big issues for environments with one static group. But I doubt that people do not change group membership on the fly once in a while. Sometimes you might even want a normally grouped speaker to play in standalone mode (which might result in group membership being changed/groups being destroyed if it was the lead speaker).

To give you an example, this is my setup:

• In the morning: bathroom and kitchen speakers are one group. Group setup is semi-automatic through mini-media-player.
• Throughout the day: Speakers are grouped dynamically based on presence detection in rooms. Group members are populated automatically.
• In the evening: living room and kitchen are one group, bathroom, child's room and bedroom are another group. Group setup is semi-automatic through mini-media-player.

So you can see that groups change over the day. I do not know how this could be represented by a speaker group entity. Don't get me wrong, I really like the idea of making grouping more intuitive than it is in Sonos/Bluesound right now. It's just that I don't see how from a user-perspective I would be interacting with such dynamic groups.

• Agree with @rytilahti that preserving unofficial cards and non-standard interfaces is not a con. It would actually be a consequence (and resulting pro): they would need to be updated and aligned with a standard architecture which improves support, reduces future breaking changes, and improves maintainability.

I agree.

[jjlawren]

As soon as the lead speaker is removed from a group (or moved to a different group) the entire group is destroyed. In HEOS it means it's group ID is lost.

This is not the case with Sonos. When removing the coordinator another is chosen automatically and playback continues uninterrupted on the remaining group members.

[emontnemery]

As soon as the lead speaker is removed from a group (or moved to a different group) the entire group is destroyed. In HEOS it means it's group ID is lost.

This is not the case with Sonos. When removing the coordinator another is chosen automatically and playback continues uninterrupted on the remaining group members.

This is the case also for Google Cast speaker groups.

Also, Google Cast speaker groups are typically pre-defined by the user and can be discovered in the same manner as a physical Google Cast device.

There's no lower limitation to the number of devices in a Google Cast speaker group.

The support currently implemented in the Google Cast integration for speaker groups is very limited:

• Speaker Groups are discovered and shown as Google Cast media player entities
• If a group, of which a physical device is a member, is playing the physical devices will:
  • Copy the status (name of track, time, cover art, etc.) from the group
  • Forward media actions such as play, pause, etc to the group with the exception of play_media

[balloob]

I think that we should add support for this. Only then can we add proper support in the frontend.

So support would be something like:

• new feature flag SUPPORT_GROUPS
• two new services join and unjoin mapped to entity methods async_join and async_unjoin. Will be called on the coordinator. It's up to the entity to manage the other entities.
• new attribute entity_id that list group members.

[klada]

@balloob That sounds a little like the current approach in Sonos and Bluesound (and my proposed HEOS PR). What I did not really get from your point is whether you think we should create new entities for each speaker group or if we should reuse existing media player entities with some special attributes.

I personally think reusing existing media player entities with a well-defined set of attributes is the better approach here, as new entities for speaker groups only make sense when these groups are static and never changed. In my usage scenario the same speaker is a member of different groups throughout the day and I don't see how this could work with speaker group entities.

[jjlawren]

To throw another wrench at this, Sonos will be releasing a new platform (S2) in June that seems to allow you to pre-define groups instead of only grouping/ungrouping speakers individually. It won't be supported on some legacy equipment. And it's also assuming that the update doesn't just straight-up break the API we use today.

[balloob]

Good question about entities. I don't know right now, would need more research.

Current Sonos groups "consume" the speakers, so you just see the group. In Cast you see both the group and the device as a play target. That's something that will need to be captured.

[nickrout]

I have wanted to see this (speaker grouping) included for squeezebox for some time, but didn't know if there was some "correct" way. Very glad to see this thread.

PS does anyone know who the codeowner is for squeezebox? It is not in the manifest file and I am not clever enough to go back that far in the github history!

PS @persandstrom wrote the original squeezebox code.

[persandstrom]

I have not used Squeezebox for a long time now @nickrout, and do not know if it has been maintained by anyone else.

[klada]

Is there a place like a wiki for Home Assistant development, where multiple people can add/comment pros and cons of various approches? Right now I feel like a lot have things have been said, but it's difficult to spot them between the lines in a GitHub issue, especially when it gets longer.

I think the ultimate goal here should be a list of different approches which people can comment and add ideas to. When the list is complete the core developers can decide how things should be handled.

[JeffLIrion]

So support would be something like:

• new feature flag SUPPORT_GROUPS
• two new services join and unjoin mapped to entity methods async_join and async_unjoin. Will be called on the coordinator. It's up to the entity to manage the other entities.
• new attribute entity_id that list group members.

@balloob and @klada

Regarding the treatment of static vs. dynamic groups:

• Perhaps the SUPPORT_GROUPS flag should only be used for integrations that have dynamic groups, and these integrations would support the join and unjoin services. Maybe call it SUPPORT_GROUPING instead?
• For integrations with static groups, there could be a property is_group which is true for the group entities and false for the individual entities. And as quoted above, these entities could have an entity_id attribute that lists their members. I think a member_volumes attribute that lists the volumes for the media players in entity_id would also be useful.

[JeffLIrion]

Volume control for groups can be tricky. For example, the table below illustrates a scenario where you have 4 speakers (A, B, C, and D) in a group and you want to set them all to the same volume level (0.4). Each time you change a member's volume level, the volume for the group will change.

Step \ Speaker	Group	A	B	C	D
Initial	0.4	0.1	0.7	0.3	0.5
1	0.475	0.4	0.7	0.3	0.5
2	0.4	0.4	0.4	0.3	0.5
3	0.425	0.4	0.4	0.4	0.5
4	0.4	0.4	0.4	0.4	0.4

Note that:

• The group volume was initially 0.4, so there may need to be special logic in the service call and/or in the handling of updates to ensure that the desired volume change is made and that HA recognizes the state change.
• Currently, I think the cast integration would recognize 4 state changes for the group media player and 1 state change for each of the individual media players. It might be better if state changes for the group media player could be blocked until the service call to set the volume has finished.

[klada]

@JeffLIrion Regarding group volume a few things come to my mind:

• I think the vast majority of systems actually have native support for group volume. We should use that. Are there even multi-room audio integrations where the hardware is not aware of "group volume"?
• Group volume needs to be relative to the individual speaker's volume. That's the whole point of the group volume. But I still don't think that we should do such calculations in Home Assistant.
• Changing an individual speaker's volume should not change the group volume at all. So if we are deciding to not not use group entities we'd need two different volume properties for media players (volume_level and group_volume_level) and of course proper support for two volume sliders in Lovelace UI.

[pvizeli]

What I did not really get from your point is whether you think we should create new entities for each speaker group or if we should reuse existing media player entities with some special attributes.

Yeah, that would create a random entity which is hard to use it on the UI or use it for automation because of names and reservation. That means, we can make a media_player.group like light.group or switch.group to coordinate media player inside HA over multiple media player but that solve not this architecture issue.

However, to use the group function we should just implement the approach of i.e. Sonos and let handle that with a nice UI card. That should be fine for the first MVP as a start.

[klada]

Thanks @pvizeli for your feedback.

However, to use the group function we should just implement the approach of i.e. Sonos and let handle that with a nice UI card. That should be fine for the first MVP as a start.

That's what I am thinking as well.

So here's my proposal:

Step 1 (unifying the status quo)

1. Add a media player flag SUPPORT_GROUPING

2. Add service for creating/joining a group (integration.join) with the following signature:

      leader:
        description: The group leader. [Required]
      entity_id:
        description: A single or multiple media player entities to join the group. [Required]
   

   Note: If a media player is already a member of the same group as leader it should not be removed from that group if join is called a second time without that player in the entity_id list. This will allow people to add players to existing groups through automations without having to worry about other group members.

   Note 2: For integrations which do not have a concept of group leaders leader can be any speaker which should be grouped with entity_id.

3. Add service for leaving a group: integration.unjoin.

   If the leader speaker if removed from a group the group might be destroyed. This will depend on the integrations, as some might have a "takeover" feature.

4. Add an attribute to each player group_members which will be the list of players in the group. For integrations which have a concept of a "leader speaker" the first element in this list will be the entity_id of the group leader

5. Add an attribute group_candidates to each player which will be a list of entities which can be grouped together.

This is more or less the current state of Sonos and Bluesound with the following changes:

• The feature flag from 1. is new
• Breaking change: Rename attribute sonos_group/bluesound_group to group_members. The idea is to remove the integration name from the attribute to make it easier to access it in a generic way.
• Breaking change: Rename master to leader in the join service call

Each media player entity which is a member of a group should control the group playback with it's actions (pause/play/skip/...). However volume control should be player specific. This should mimic the behavior of most speakers' hardware buttons (if they have any, such as the HEOS speakers).

Step 2 (Add support for group volume control)

This should be the next stage. Group volume was discussed earlier and can be set through a simple service call.

1. Add a media player flag SUPPORT_GROUP_VOLUME
2. Define a service integration.group_volume_set with the same signature as media_player.volume_set. The entity_id needs to be a grouped media player.
3. Add a group_volume attribute to each player.

Step 3 (UI polish)

For people who want to create/destroy groups through the UI we need to update the media player card. This can be postponed until some integrations are done with at least step 1.

1. Allow Lovelace media player card to have another volume slider for group volume, if the player is grouped and supports group volume
2. Add dropdown in Lovelace media player card for grouping/ungrouping. This could be done through a input select multiple.

The grouping/ungrouping through Lovelace might be a bit tricky. Example:

1. Let there be 3 ungrouped players media_player.one, media_player.two and media_player.three
2. Click media_player.one and select media_player.two as a group member. The two speakers should now be grouped together with media_player.one being the group master.
3. Click media_player.two. When media_player.three is added to the group it should be added to the group where media_player.one is the group master.
4. Click media_player.three. The group member input select should now display all 3 media players. When unchecking media_player.one the group will be destroyed on platforms with a "leader speaker" concept.

Comments

I think step 1 should be pretty straightforward. Especially for Sonos/Bluesound there should not be much work required.

What do you think of this proposal? Feel free to comment individual things and I'll update them in this proposal after a short discussion. If you dislike attribute names or service call signatures just say so. I am not fully aware of the HASS conventions so I just chose what I found meaningful. :smile:

I am also perfectly fine if somebody else comes up with a better idea. :+1:

[tmjo]

Hi! Any decisions on this arch issue? Would be great to have the grouping feature included in the official HEOS-component instead of relying on custom components to do the same job. I don't think it is fair to stop the group implementation on HEOS due to a principal disagreement on how to do grouping. Unless the arch issue can be decided soon, I suggest the proposed HEOS solution to be approved just as it has been for Sonos and similar, and in case an arch decision is made later it may be changed (just like it has to be done for other integrations).

[klada]

What is the workflow with this now? Any comments on my suggestion in https://github.com/home-assistant/architecture/issues/364#issuecomment-625405413?

I'd like to finish my PR for grouping in the HEOS component. I have already put a fair amount of time in it and I do not want to put any more effort into this before I can be sure that it'll be merged.

[sharebear]

I stumbled in here while currently deciding whether to invest in bluesound/heos/sonos that I'm likely to automate with HA. I'm not sure I have any input on the primary issue here but unless I'm missing something the suggested spec for "grouping/ungrouping through Lovelace" might need a little more precision in the list of which players should be available for selection.

For example, today I have a single sonos device, if I decide to go forward with HEOS and get myself a super link, I shouldn't be able to choose the sonos device when creating a group from one of the HEOS zones. It might not even be useful to show grouping ui for the sonos, even though it supports it, if there isn't another sonos device present to group with.

Back onto the primary topic, if groups were modelled as a separate entity would this allow creation of "pre-defined groups" similar to what is mentioned as coming to sonos, but for all systems? Some kind of declarative helper for creating the different groups you use often.

[amelchio]

@andrewsayre Thanks for looping me in. Even though I see where you are coming from architecturally, I am afraid that I also think dynamic entities for dynamic groups will be a user interface (GUI and automations) mess.

@klada There is unfortunately no process for moving architecture issues forward but it seems fair to say that your proposal has not met much resistance. I think you should go ahead with your proposal.

Rename attribute sonos_group/bluesound_group to group_members. The idea is to remove the integration name from the attribute to make it easier to access it in a generic way.

This makes sense but how will the UI then know that Bluesound and Sonos cannot be joined with each other?

[klada]

I'm not sure I have any input on the primary issue here but unless I'm missing something the suggested spec for "grouping/ungrouping through Lovelace" might need a little more precision in the list of which players should be available for selection.

To be honest I am not much of a GUI type. I do not have any idea yet how this will look like, but if you take a look at https://github.com/kalkih/mini-media-player I think this will make a good UI. I am hoping that after doing the backend work that frontend developers are hopping on to the train.

Back onto the primary topic, if groups were modelled as a separate entity would this allow creation of "pre-defined groups" similar to what is mentioned as coming to sonos, but for all systems? Some kind of declarative helper for creating the different groups you use often.

That's very easy to achieve with my proposal. Just set up automations as presets which do the grouping you like for you.

@klada There is unfortunately no process for moving architecture issues forward but it seems fair to say that your proposal has not met much resistance. I think you should go ahead with your proposal.

Thanks for the input. I'll fire up my dev environment as soon as I get some more time.

This makes sense but how will the UI then know that Bluesound and Sonos cannot be joined with each other?

This is a very good thought. :+1: I think we need another attribute such as group_candidates which will be a list of entities which can be grouped together. I'll look into that when updating my HEOS PR.

[Ramblurr]

@klada I see you've started moving on this recently. I am curious what you decided with regards to the "how does HA know which player entities can be grouped together" question?

[klada]

@Ramblurr The basic idea was that each player entity provides an attribute called group_candidates (see https://github.com/home-assistant/architecture/issues/364#issuecomment-625405413). Paulus suggested to leave this out in the beginning (see https://github.com/home-assistant/core/pull/41193#discussion_r556654149).

So once proper support for player groups is being worked on in Lovelace UI, an attribute like that will be required. But right now the integrations will need to be updated first.

[frenck]

This architecture issue is old, stale, and possibly obsolete. Things changed a lot over the years. Additionally, we have been moving to discussions for these architectural discussions.

For that reason, I'm going to close this issue.

../Frenck