RFC: yabridgectl update

[robbert-vdh]

Recently I got the suggestion to add an updater to yabridgectl. Since I only provide compiled binaries and AUR packages, updating yabridge on non Arch based distros involves downloading new copies of yabridge and yabridgectl, overwriting your old installation, and then remembering to rerun yabridgectl sync. Having an updater built into yabridgectl sounded like a good idea to make yabridge even easier to use (and that way you can't forget to run yabridgectl sync), but it also got me thinking since having a good updater does bring forth quite a few considerations. These were the things I could think of from the top of my head:

• Running yabridgectl update should fetch the releases page on GitHub through either the API or the Atom feed and compare the version of the latest available release to the current version of yabridgectl (since right now there's no convenient way to get the version of yabridge without loading a plugin, and these should be in sync anyways).

• If there is a newer version available, then yabridgectl should show the changelog difference between the currently installed version and the latest version before asking whether the user wants to update or not. This changelog can be parsed from CHANGELOG.md since it's in a standard format.

• If the user chooses to proceed with the update, then yabridgectl should download the new files from the release on GitHub. I can think of two challenges here:

  • yabridgectl will need some way to verify whether the uploaded file is genuine and not corrupted. My idea here was to sign both yabridge-X.Y.Z.tar.gz and yabridge-X.Y.Z-ubuntu-18.04.tar.gz with gpg and to then include yabridge-X.Y.Z.tar.gz.sig and yabridge-X.Y.Z-ubuntu-18.04.tar.gz.sig in the release so the downloaded archives can be verified. This would mean either adding a dependency on a pgp implementation, or requiring that the user has gpg installed (which would often be the case, but not always).
  • Another slight challenge would be figuring out which version of yabridge to download. The *-ubuntu-18.04.tar.gz archives are built on Ubuntu 18.04 to support older versions of glibc and are statically linked to libstdc++. One way to figure out which one would be the correct download would be to check the currently installed version of glibc and it that to the one available on Ubuntu 20.04.

• Updating should replace both the current yabridge installation as well as the currently used yabridgectl binary since these need to stay in sync.

• After yabridge and yabridgectl have been replaced with updated versions, yabridgectl should run yabridgectl sync using the newly downloaded yabridgectl version to finish the update since the currently installed version of yabridgectl may be incompatible with any future changes to yabridge.

• It should also be possible to downgrade to an earlier version and to reintall the current version if needed. This would go through the same process as above but without the changelog.

• If yabridge was installed through a distro package (or from the AUR), then the update functionality should be disabled. This could be done either through a compile time feature flag, or by simply checking whether the yabridgectl binary can be written to by the user. A flag would make more sense but the updater should still be disabled for the yabridge-bin AUR package, and since that package downloads the files directly from GitHub that makes things a bit more complicated.

• yabridgectl should never update or connect to the internet without asking. One possible way to remind users to check for updates every once in a while would be to show a reminder when running yabridgectl sync if it has been a few weeks since running yabridgectl update. I'm not sure if this is a good idea or not though.

So all in all, it seems like it will be a bit more complicated then I initially anticipated. Any opinions? Things that I missed or should be done differently?

[siborg]

What I do on my arch system is use a pacman hook for yabridge install.

/etc/pacman.d/hooks/yabridge.hook

[Trigger] Operation=Install Operation=Upgrade Operation=Remove Type=Package Target=yabridge

[Action] Depends=yabridgectl-git When=PostTransaction Exec=/bin/sudo -u '/bin/yabridgectl' sync

This essentially runs sync every time the yabridge is being upgraded. I must say I haven't investigated if pacman/yay provide any variables for the username the update command was run as. This solution works well for me.

Perhaps there are similar mechanisms for other distros?

S.

On Sun, 15 Nov 2020 at 12:46, Robbert van der Helm [email protected] wrote:

Recently I got the suggestion to add an updater to yabridgectl. Since I only provide compiled binaries and AUR packages, updating yabridge on non Arch based distros involves downloading new copies of yabridge and yabridgectl, overwriting your old installation, and then remembering to rerun yabridgectl sync. Having an updater built into yabridgectl sounded like a good idea to make yabridge even easier to use (and that way you can't forget to run yabridgectl sync), but it also got me thinking since having a good updater does bring forth quite a few considerations. These were the things I could think of from the top of my head:

Running yabridgectl update should fetch the releases page on GitHub through either the API or the Atom feed and compare the version of the latest available release to the current version of yabridgectl (since right now there's no convenient way to get the version of yabridge without loading a plugin, and these should be in sync anyways).

If there is a newer version available, then yabridgectl should show the changelog difference between the currently installed version and the latest version before asking whether the user wants to update or not. This changelog can be parsed from CHANGELOG.md https://github.com/robbert-vdh/yabridge/blob/master/CHANGELOG.md since it's in a standard format.

If the user chooses to proceed with the update, then yabridgectl should download the new files from the release on GitHub. I can think of two challenges here:

• yabridgectl will need some way to verify whether the uploaded file is genuine and not corrupted. My idea here was to sign both yabridge-X.Y.Z.tar.gz and yabridge-X.Y.Z-ubuntu-18.04.tar.gz with gpg and to then include yabridge-X.Y.Z.tar.gz.sig and yabridge-X.Y.Z-ubuntu-18.04.tar.gz.sig in the release so the downloaded archives can be verified. This would mean either adding a dependency on a pgp implementation https://crates.io/crates/pgp/, or requiring that the user has gpg installed (which would often be the case, but not always).
  • Another slight challenge would be figuring out which version of yabridge to download. The *-ubuntu-18.04.tar.gz archives are built on Ubuntu 18.04 to support older versions of glibc and are statically linked to libstdc++. One way to figure out which one would be the correct download would be to check the currently installed version of glibc and it that to the one available on Ubuntu 20.04.

Updating should replace both the current yabridge installation as well as the currently used yabridgectl binary since these need to stay in sync.

After yabridge and yabridgectl have been replaced with updated versions, yabridgectl should run yabridgectl sync using the newly downloaded yabridgectl version to finish the update since the currently installed version of yabridgectl may be incompatible with any future changes to yabridge.

It should also be possible to downgrade to an earlier version and to reintall the current version if needed. This would go through the same process as above but without the changelog.

If yabridge was installed through a distro package (or from the AUR), then the update functionality should be disabled. This could be done either through a compile time feature flag, or by simply checking whether the yabridgectl binary can be written to by the user. A flag would make more sense but the updater should still be disabled for the yabridge-bin AUR package, and since that package downloads the files directly from GitHub that makes things a bit more complicated.

yabridgectl should never update or connect to the internet without asking. One possible way to remind users to check for updates every once in a while would be to show a reminder when running yabridgectl sync if it has been a few weeks since running yabridgectl update. I'm not sure if this is a good idea or not though.

So all in all, it seems like it will be a bit more complicated then I initially anticipated. Any opinions? Things that I missed or should be done differently?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/robbert-vdh/yabridge/issues/51, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ6NUQLRHJ3PAGAEJCZDALSP7ERJANCNFSM4TWFS4EA .

[cameronleger]

I hate the idea (would never use it since I also don't use yabridgectl) and would consider it a massive waste of your potential and a much larger surface for bugs. Of 2.4k packages installed on my machine, I think only Steam has this "feature." I understand that you're trying to make it easier for users, but I'd suggest that we focus on what the real and perceived problems are, and try to find better solutions (this is a solution without a problem, imo).

[robbert-vdh]

I agree that this could be a pretty large surface for bugs (and it would of course make the compile time for yabridgectl a bit longer since this would add dependencies on reqwest and possibly also pgp), although I don't think the implementation would cost a lot of time or effort. It's just that the most common source of issues for yabridge I've seen has been when when people forget one part of the update process, which is 1) replace old yabridge files, 2) replace yabridgectl, 3) rerun yabridgectl sync to update all your copies of libyabridge.so. Although I did also kind of forgot to put a reminder to do all three of those things in the release notes on the releases page. I've now added it to the bottom of the latest release and I'll also add it to new releases in the future, so maybe that's enough.

[noisecode3]

Maybe there could just be a script in the source tree for people how compile it from source, It could update the source, compile and upgrade the home installation and run yabridgectl sync. and the packages for the distros and AUR could run yabridgectl sync for users how have it installed in the home folder when updateing or something like that. So it updates fully from start to end everywhere.

[robbert-vdh]

When building from source you actually don't have to copy things to your home directory! I've just added the build directory in the repo to my PATH, and with yabridgectl set --path /path/to/the/repo/build you can tell yabridgectl to look for the files there. And then I just use ninja -C build && yabridgectl sync -nf to compile and update. With the AUR packages I've also included a post update script with a reminder to rerun yabridgectl sync, but it doesn't do it automatically.

[noisecode3]

I meant in you're post_install you could ask the user if he/she wants to run yabridgectl sync for some user in the home directory. Or do it automatically if its not a security problem to do so. I'm not sure

[robbert-vdh]

I don't think you're allowed to prompt in an .install script (but please correct me if I'm wrong!). Those scripts are also run as root so if you wanted to run yabridgectl sync automatically then it would involve some nastiness to figure out which user is calling pacman. I think a reminder is the best we can do here there without being potentially disruptive. Even the nvidia-all package reminds you to run sudo mkinitcpio -P after installing instead of doing it for you, and if you forget to run that then you'll end up with a system without any graphics.

But yeah, I'll put this on low priority for now. VST3 support for yabridge is next although that's still far off.

[siborg]

I suppose you could write a hook which will list all home directories in /home and run yabridgectl for each user with sudo. it's far from an ideal solution though.

On Sun, 15 Nov 2020 at 19:55, Robbert van der Helm [email protected] wrote:

I don't think you're allowed to prompt in an .install script (but please correct me if I'm wrong!). Those scripts are also run as root so if you wanted to run yabridgectl sync automatically then it would involve some nastiness to figure out which user is calling pacman. I think a reminder is the best we can do here there without being potentially disruptive. Even the nvidia-all https://github.com/Frogging-Family/nvidia-all package reminds you to run sudo mkinitcpio -P after installing instead of doing it for you, and if you forget to run that then you'll end up with a system without any graphics.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/robbert-vdh/yabridge/issues/51#issuecomment-727626721, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ6NUT3I23Q7W2VY2UH4STSQAW3JANCNFSM4TWFS4EA .

[robbert-vdh]

I also have all faith in Arch users that they remember to run yabridgectl sync :grin: (or set up a hook themselves like you did, great idea btw). The only times this has really come up as an issue has been when using the CI builds on Debian based distros, so now I just always remind people to do run sync after replacing the files.

[noisecode3]

what if yabridgectl could run the package manager in that particular distro its installed on. to upgrade itself and restart itself with yabridgectl sync ? lol

[noisecode3]

You could have a yabridgectl sync program that only runs as root when the package is upgraded. But it will will have to be from root I think. that should be allowed. like a yabrideroot-sync program.

[geekositalia]

I am in favor of an updater, I would use an additional binary formula, yabridge-update, which downloads, unpacks, in the same directory, and prompts you to update.

This does not create problems for those who want to use it from the distribution, instead of binary (I prefer binary, and I think it is a useful function for those who use lts distributions)

[robbert-vdh]

If I ever add this, it will just be a subcommand of yabridgectl behind a compile time flag (so you can leave it and all of the dependencies that come with it out when packaging yabridgectl).