Bug : supervisor does not detect `Image` tag in config.* when switching addons from local build to image build

[alexbelgium]

Describe the issue you are experiencing

If an addon is already installed with a local-build strategy, it doesn't update correctly if the new version has an image-based strategy.

Example from the logs below : an addon named addon_updater is installed in v3.9, with a config.json that does not contain an image field. It is therefore locally built. When updating to the v5.0, with a config.json that contains an image field, it still builds locally the v5.0 and ignores the image tag. Starting the addon then fails, as the addon reads the image tag and does not find the downloaded image.

What is the used version of the Supervisor?

supervisor-2021.12.2

What type of installation are you running?

Home Assistant OS

Which operating system are you running on?

Home Assistant Operating System

What is the version of your installed operating system?

7.1

What version of Home Assistant Core is installed?

core-2021.12.10

Steps to reproduce the issue

1. Install an addon without an image field (local build)
2. Update the config.json in github to add the image field, and build images
3. Update repo, install the addon
4. Observe in the Supervisor log that the image is built and not downloaded
5. Start the addon and observe in supervisor log the absence of downloaded image

Positive control :

1. Uninstall the above installed app
2. Reinstall from fresh the v5.0 : it correctly downloads the image and starts normally

Anything in the Supervisor logs that might be useful for us?

# Refresh repository 
22-01-21 20:24:27 INFO (MainThread) [supervisor.store] Loading add-ons from store: 144 all - 0 new - 0 remove

# Start addon update
22-01-21 20:24:52 INFO (SyncWorker_7) [supervisor.docker.interface] Updating image db21ed7f/aarch64-addon-updater:3.9 to ghcr.io/alexbelgium/addons_updater-aarch64:5.0
22-01-21 20:24:52 INFO (SyncWorker_7) [supervisor.docker.addon] Starting build for db21ed7f/aarch64-addon-updater:5.0
22-01-21 20:27:02 INFO (SyncWorker_7) [supervisor.docker.interface] Cleaning addon_db21ed7f_updater application
22-01-21 20:27:02 INFO (MainThread) [supervisor.addons] Add-on 'db21ed7f_updater' successfully updated
22-01-21 20:27:02 WARNING (SyncWorker_5) [supervisor.docker.interface] Can't find ghcr.io/alexbelgium/addons_updater-aarch64 for cleanup
22-01-21 20:27:02 INFO (MainThread) [supervisor.host.apparmor] Adding/updating AppArmor profile: db21ed7f_updater

#Start of the addon
22-01-21 20:27:34 ERROR (SyncWorker_6) [supervisor.docker] Image ghcr.io/alexbelgium/addons_updater-aarch64:5.0 does not exist for addon_db21ed7f_updater

Edit 22-Feb: still the same on latest supervisor (I've added the "image" tag in the config.json of the addon before the repo refresh - which then shows the addon trying to local build instead of downloading the image, then trying to run out of the image it hasn't downloaded)

22-02-20 12:03:03 INFO (MainThread) [supervisor.host.apparmor] Adding/updating AppArmor profile: db21ed7f_webtop-kde
22-02-20 12:03:03 INFO (SyncWorker_4) [supervisor.docker.addon] Starting build for db21ed7f/aarch64-addon-webtop-kde:4.16-r0-ls43
22-02-20 12:08:49 INFO (SyncWorker_0) [supervisor.docker.addon] Build db21ed7f/aarch64-addon-webtop-kde:4.16-r0-ls43 done
22-02-20 12:08:49 INFO (MainThread) [supervisor.addons] Add-on 'db21ed7f_webtop-kde' successfully installed
22-02-20 12:08:49 INFO (SyncWorker_4) [supervisor.docker.addon] Build db21ed7f/aarch64-addon-webtop-kde:4.16-r0-ls43 done
22-02-20 12:08:49 INFO (MainThread) [supervisor.addons] Add-on 'db21ed7f_webtop-kde' successfully installed
22-02-20 12:14:06 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-02-20 12:40:42 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/alexbelgium/hassio-addons repository
22-02-20 12:40:42 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/home-assistant/addons repository
22-02-20 12:40:43 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/hassio-addons/repository repository
22-02-20 12:40:43 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/sabeechen/hassio-google-drive-backup repository
22-02-20 12:40:46 INFO (MainThread) [supervisor.store] Loading add-ons from store: 142 all - 0 new - 0 remove
22-02-20 12:40:54 INFO (SyncWorker_2) [supervisor.docker.interface] Removing image db21ed7f/aarch64-addon-webtop-kde with latest and 4.16-r0-ls43
22-02-20 12:40:54 INFO (SyncWorker_5) [supervisor.docker.addon] Starting build for db21ed7f/aarch64-addon-webtop-kde:4.16-r0-ls43
22-02-20 12:40:56 INFO (SyncWorker_5) [supervisor.docker.addon] Build db21ed7f/aarch64-addon-webtop-kde:4.16-r0-ls43 done
22-02-20 12:40:56 INFO (MainThread) [supervisor.addons] Add-on 'db21ed7f_webtop-kde' successfully rebuilt
22-02-20 12:41:04 ERROR (SyncWorker_5) [supervisor.docker] Image ghcr.io/alexbelgium/webtop-kde-aarch64:4.16-r0-ls43 does not exist for addon_db21ed7f_webtop-kde
22-02-20 12:41:04 INFO (SyncWorker_5) [supervisor.resolution.module] Create new issue IssueType.MISSING_IMAGE - ContextType.ADDON / db21ed7f_webtop-kde
22-02-20 12:41:04 INFO (SyncWorker_5) [supervisor.resolution.module] Create new suggestion SuggestionType.EXECUTE_REPAIR - ContextType.ADDON / db21ed7f_webtop-kde

Additional information

No response

[alexbelgium]

Hello, I know you are very busy with tons of things but having this issue solved would allow me to move all my addons to autobuilds. Thanks!

[alexbelgium]

Hi @ludeeus : just wanted to bring your attention to that one.

Some months ago, you had mentioned to me that it would be a good practice to migrate my repo with locally build addons to autobuilds (https://github.com/home-assistant/supervisor/issues/3223#issuecomment-988703018). I really appreciated the time you took to provide me the links to the proper documentation, and implemented the relevant code to my repo.

However, this bug prevents me from using it on my addons : once an addon is installed with a local build, it can't be updated to a new version based on a precompiled image (see all infos including relevant logs on the first post - I've tested the bug is still there in the latest supervisor version). I know you guys are super busy, but I really believe that this bug could really help in the implementation of your vision of pushing more for precompiled addons. Thanks very much!

[alexbelgium]

Edit 22-Feb: still the same on latest supervisor, I didn't add logs as it is the same behavior

Issue troubleshooting :

• ATTR_IMAGE is not found in self.data - is this not refreshed after repo update? : https://github.com/home-assistant/supervisor/blob/a82b4aa6c82b72e9c70e848ff599e3f3910e804e/supervisor/addons/model.py#L522

• This causes https://github.com/home-assistant/supervisor/blob/8c528f7ec54148d9847016a6c71737373e6742a5/supervisor/docker/addon.py#L529 to execute https://github.com/home-assistant/supervisor/blob/main/supervisor/docker/addon.py#L544 instead of https://github.com/home-assistant/supervisor/blob/main/supervisor/docker/interface.py#L187

Anyone has an idea how to solve this ?

[alexbelgium]

Verified : this bug is still present on latest versions

Système d'exploitation hôte | Home Assistant OS 7.6 Mise à jour | stable Version du Supervisor | supervisor-2022.05.0 Version | core-2022.5.3 Type d'installation | Home Assistant OS

However, the ATTR_IMAGE is only not read when it is a local build. If I change the path between two versions, it still updates

FIRST LOCATION : 22-05-09 09:12:52 INFO (SyncWorker_1) [supervisor.docker.interface] Downloading docker image ghcr.io/alexbelgium/test-aarch64 with tag release-4.4.1-ls173-10.
22-05-09 09:13:06 INFO (MainThread) [supervisor.addons] Add-on '6c45555e_qbittorrent' successfully installed
22-05-09 09:20:35 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/home-assistant/addons repository
22-05-09 09:20:35 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/dianlight/hassio-addons repository
22-05-09 09:20:35 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/sabeechen/hassio-google-drive-backup repository
22-05-09 09:20:35 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/alexbelgium/hassio-addons-test repository
22-05-09 09:20:35 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/alexbelgium/hassio-addons repository
22-05-09 09:20:35 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/hassio-addons/repository repository
22-05-09 09:20:38 INFO (MainThread) [supervisor.store] Loading add-ons from store: 133 all - 0 new - 0 remove
22-05-09 09:20:38 INFO (MainThread) [supervisor.store] Loading add-ons from store: 133 all - 0 new - 0 remove
22-05-09 09:20:39 INFO (SyncWorker_0) [supervisor.docker.interface] Updating image ghcr.io/alexbelgium/test-aarch64:release-4.4.1-ls173-10 to ghcr.io/alexbelgium/test2-aarch64:release-4.4.1-ls173-11
UPDATED LOCATION : 22-05-09 09:20:39 INFO (SyncWorker_0) [supervisor.docker.interface] Downloading docker image ghcr.io/alexbelgium/test2-aarch64 with tag release-4.4.1-ls173-11.

[alexbelgium]

Still present with supervisor 2022.05.3

[alexbelgium]

Hi @frenck , really sorry to pull you like this. I know this is bad github etiquette but I don't know how else to bring attention on this issue. I believe it is a legitimate supervisor bug, that I have been able to replicate from supervisor-2021.12.2 up to the latest version, however, I have never been able to get the attention or a response from a supervisor dev, either here or on the discord.

I have tried to solve it myself but python is not my best expertise and I couldn't find what was causing the bug.

Thanks very much for any answer, and apologies again if this is not the proper way of communication! This will only increase the gratitude that I already feel for you for all the great addons you have provided to the community, your work on HA and the help you gave me in the past on topics such as the image builder! ;-)

[alexbelgium]

Still relevant... Honestly I don't understand how you can push dev towards image based addons while keeping this bug in the supervisor which decreases user experience a lot when switching from local build to image based!

[jordanauge]

I confirm i could reproduce this bug with the latest code (I am the author of the aforementioned bug report)

[mouthpiec]

issue still present with latest release .... it would be ideal if this is fixed

[github-actions[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[alexbelgium]

This is not stale, thanks. This is quite a big issue impacting custom addons providers

[alexbelgium]

Edit : I've tested once again with the same protocol on the latest supervisor/core/os

• Install an addon with local build (no image tag in build.json)
• Update the build.json on github to have an image, wait for the image to build
• Update the local addon

The addon then doesn't start, and the log clearly shows that the image tag is ignored when updating (so the addon builds locally), but taken into account when starting the addon (therefore the addon doesn't start as the image wasn't downloaded)

22-10-11 13:15:33 INFO (SyncWorker_1) [supervisor.docker.interface] Updating image db21ed7f/aarch64-addon-joal:2.1.31 to ghcr.io/alexbelgium/joal-aarch64:2.1.31-2
22-10-11 13:15:33 INFO (SyncWorker_1) [supervisor.docker.addon] Starting build for db21ed7f/aarch64-addon-joal:2.1.31-2
22-10-11 13:18:22 INFO (SyncWorker_1) [supervisor.docker.addon] Build db21ed7f/aarch64-addon-joal:2.1.31-2 done
22-10-11 13:18:22 INFO (SyncWorker_1) [supervisor.docker.interface] Cleaning addon_db21ed7f_joal application
22-10-11 13:18:22 INFO (MainThread) [supervisor.addons] Add-on 'db21ed7f_joal' successfully updated
22-10-11 13:18:22 WARNING (SyncWorker_4) [supervisor.docker.interface] Can't find ghcr.io/alexbelgium/joal-aarch64 for cleanup
22-10-11 13:18:22 INFO (MainThread) [supervisor.host.apparmor] Adding/updating AppArmor profile: db21ed7f_joal
22-10-11 13:21:52 ERROR (SyncWorker_1) [supervisor.docker.manager] Image ghcr.io/alexbelgium/joal-aarch64:2.1.31-2 does not exist for addon_db21ed7f_joal
22-10-11 13:21:52 INFO (SyncWorker_1) [supervisor.resolution.module] Create new suggestion SuggestionType.EXECUTE_REPAIR - ContextType.ADDON / db21ed7f_joal
22-10-11 13:21:52 INFO (SyncWorker_1) [supervisor.resolution.module] Create new issue IssueType.MISSING_IMAGE - ContextType.ADDON / db21ed7f_joal

[mdegat01]

Hey sorry about the delay, just put in a PR for it.

Once it is merged you can test the behavior on the dev channel. If you want to test it now before merge and report back then you can if you have host shell access with the following sequence of commands

> docker exec -it hassio_supervisor bash
> cd /usr/src
> rm -rf supervisor
> git clone --depth 1 -b fix-addon-image-change-on-update https://github.com/home-assistant/supervisor/
> python3 -m compileall supervisor/supervisor
> exit
> ha su restart

Supervisor will say your system is unsupported because you're running on untrusted code but that's what you want, means your on the branch. Afterwards you can revert back like this:

docker stop hassio_supervisor && docker rm hassio_supervisor

Observer polls and restarts supervisor automatically when it detects it is down. So you don't need to restart supervisor again, just need to clear out the container before observer starts it up again so it gets rebuilt from the image.

[alexbelgium]

I've tested your procedure and it works great, thanks!