[Swatch|Swatch Group] First Swatch selected interacts poorly with `display: inline-flex`

[Westbrook]

Description

display: inline-flex sometimes adds the trailing "space", but selecting the first Swatch in the Swatch Group causes the "space" to collapse.

Steps to reproduce

1. Go to https://opensource.adobe.com/spectrum-css/swatchgroup.html#density-compact
2. Add a <div> with content AFTER the .spectrum-SwatchGroup element
3. Remove the width of the .spectrum-SwatchGroup element, so the Swatches all display on a single row.
4. Add .is-selected to the FIRST Swatch element.
5. See the "space" between the Group and the <div> change.

Expected behavior

Default styles are not effected by the selection of Swatches in the group.

Screenshots

With selection:

Without selection:

synced to jira: https://jira.corp.adobe.com/browse/CSS-202