ESP32 S3 Box only turn on screen when Assist is in progress

[TheZoker]

I think the title says it all: Allow the screen to only be turned on, when a Assist is in progress.

I have one box in the bedroom and the turned on screen is not optimal. Would be nice if it only turns on when needed.

[ChristophCaina]

hm... yes, I think having a "switch" or something to decide if you want the "sleeping" screen (display = on) or if the display should be off by default would be a nice option to have.

This would also allow to decide for each device how you want it to behave.

[kevireilly]

I've got this working for the most part, but since I just got a Box Lite tonight and have also barely done any ESPHome yaml programming stuff, I could use some help testing this in general and on the Box [3]. Could y'all help me with some testing before I propose a change set via pull request?

Assuming that you've got the device adopted or otherwise are capable of modifying the yaml to install, should be easy enough for us to do some beta testing with a repository that I'll set up tomorrow.

• Basically, whenever it would normally be the smiley home image, it should instead be backlight off
• There is a new "Idle display" toggle switch -- does that name make sense to you?
• Idle display is on by default and turning it off should be retained across reboots
• It should still display the screen while booting, if no wifi, no HA, or other init or errant type scenarios

Oops. I just noticed that turning off the LCD Backlight option may not be adhered to (e.g. keeping it off when voice activated) and turning idle display back on does not immediately turn the display back on, but I'll work on those two too.

https://github.com/esphome/firmware/assets/431471/49c62751-bbbd-47f2-b354-14233394bddf

[kevireilly]

Alrightie! I think it is in a stable enough place for others to give it a whirl and I ported the changes over to the S3 Box / 3 which I have not been able to test since I have a S3 Box Lite, but reads like it should just work. This assumes that you've Adopted the device in ESPHome or previously Installed a firmware so that you can Edit the device yaml configuration which should be similar to the following:

substitutions:
  name: esp32-s3-box-lite-abc123
  friendly_name: ESP32 S3 Box Lite abc123
packages:
  # Previous line that included the yaml configuration
  # esphome.voice-assistant: github://esphome/firmware/voice-assistant/esp32-s3-box-lite.yaml@main

  # New line that includes the yaml configuration from my repository
  # IMPORTANT: Make sure that you use the correct yaml file per your device (detailed below this code)
  esphome.voice-assistant: github://kevireilly/firmware/voice-assistant/esp32-s3-box-lite.yaml@s3-box-idle-display
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
api:
  encryption:
    key: [...]
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

Make sure to use the file specific to your device:

S3 Box 3

esphome.voice-assistant: github://kevireilly/firmware/voice-assistant/esp32-s3-box-3.yaml@s3-box-idle-display

---

S3 Box

esphome.voice-assistant: github://kevireilly/firmware/voice-assistant/esp32-s3-box.yaml@s3-box-idle-display

---

S3 Box Lite

esphome.voice-assistant: github://kevireilly/firmware/voice-assistant/esp32-s3-box-lite.yaml@s3-box-idle-display

To have a look at the diff between the main branch that you're using now and the s3-box-idle-display branch from my fork, take a look at: https://github.com/esphome/firmware/compare/main...kevireilly:firmware:s3-box-idle-display

As this is a device switch, you can of course automate turning off/on Idle Display as part of a goodnight routine, at 9pm, or whichever other events or conditions you deem fit.

Open questions:

• When you have Idle display off and LCD Backlight off, should turning LCD Backlight on potentially turn the display back on so that you get feedback that it should display in the future? It would remain on until the device changes state
• After attempting to give a voice command, should there be a delay before the screen turns off so that you can see what the outcome was? Right now it immediately turns off, but presumably that is because it went back to the idle page
• Is there a way to turn the green LED off? In a dark bedroom, presumably this still produces a significant amount of light

[kevireilly]

Based on interactions with the original post, tagging individuals potentially interested in no S3 Box display when idling: @TheZoker @ChristophCaina @jpenschuck @the-mentor @pixeye33

[kevireilly]

Another totally viable but apparently not apparent option is simply using automations. My good friend @whoinow not only gave me this device, but also made this simple and totally logical suggestion. Great idea fine sir.

Automation: Box: turn on display on assist

- Triggers
  ESP32 S3 Box Assist in progress turned on
- Actions
  Turn on ESP32 S3 Box LCD Backlight

Automation: Box: turn off display after assist

- Triggers
  ESP32 S3 Box Assist in progress turned off
- Actions
  Turn off ESP32 S3 Box LCD Backlight

This accomplishes virtually the same outcome as the code changes with some very minor exclusions. For example, the display will not be off by default unless you voice activate/deactivate. But! No code changes required.

[ChristophCaina]

Hi, thanks :) So basically, it is just required to replace the package linking to your repository instead of using the link to the esphome/firmware...

I think, that should be OK for some tests, but I would prefer to use the original source, if something might change here... ;)

So my thoughts: After testing and validating, consider to create a PR to this repository

[kevireilly]

Yup, temporary change as part of a request for testing. I can only verify behavior on the Lite, so while it reads like the changes set should be fine on all others, confirmation would be appreciated.

What do you think about the automations approach versus code changes? Since a number of us weren't aware of this option, I do wonder if the more prudent change might be proposing a documentation update instead.

[whoinow]

@kevireilly You could also do it as a single automation in HA, which triggers being progress on and off, then a conditional action (not using conditions before the action section, but the if/then action).

Mine:

alias: Office VA Backlight Control
description: ""
trigger:
  - type: turned_on
    platform: device
    device_id: progress_device_id
    entity_id: progress_entity_id
    domain: binary_sensor
  - type: turned_off
    platform: device
    device_id: progress_device_id
    entity_id: progress_entity_id
    domain: binary_sensor
condition: []
action:
  - if:
      - type: is_on
        condition: device
        device_id: progress_device_id
        entity_id: progress_entity_id
        domain: binary_sensor
    then:
      - type: turn_on
        device_id: device_backlight_id
        entity_id: entity_backlight_id
        domain: light
    else:
      - type: turn_off
        device_id: device_backlight_id
        entity_id: entity_backlight_id
        domain: light
mode: single

[igorsantos07]

Nowadays, it seems the progress sensor will be sunset in favor of the assist entity.

But anyway, I also had the same idea and built my own automation before I stumbled upon this post, so I'm sharing what I did with the current state of affairs. This works even after booting up - the screen goes off as soon as it changes to idle.

Nonetheless, it has a tiny bit of lag and, given this could be easily processed by the ESP32 itself, it would be awesome to have this feature baked into it.

alias: Ligar/desligar tela do assistente
description: ""
triggers:
  - trigger: state
    entity_id:
      - assist_satellite.escritorio_assist_state
conditions: []
actions:
  - choose:
      - conditions:
          - condition: state
            entity_id: assist_satellite.your_assist_entity
            state: idle
        sequence:
          - action: light.turn_off
            target:
              entity_id: light.escritorio_assist_screen
            data: {}
      - conditions:
          - condition: not
            conditions:
              - condition: or
                conditions:
                  - condition: state
                    entity_id: assist_satellite.your_assist_entity
                    state: idle
                  - condition: state
                    entity_id: assist_satellite.your_assist_entity
                    state: unavailable
                  - condition: state
                    entity_id: assist_satellite.your_assist_entity
                    state: unknown
        sequence:
          - action: light.turn_on
            target:
              entity_id:
                - light.escritorio_assist_screen
            data: {}
mode: single