MSYS2-packages
MSYS2-packages copied to clipboard
msys2
Updating MSYS2 unattended is broken (MSYS2 locks up on full system upgrade)
Hi all,
the latest update seems to have made it impossible to do a full system upgrade of MSYS2 unattendedly (i.e. via CI on AppVeyor).
- The removal of catgets makes it hard to even have pacman run an update (MSYS2's own CI is affected) for which I found a workaround, though (
yes | pacman). - Even with the workaround it's still not possible to complete the update as MSYS2 seems to lock up when warning about
warning: terminate MSYS2 without returning to shell and check for updates againwarning: for example close your terminal window instead of calling exit
Does anybody know a way how to work around this issue?
Also I seem to recall that it worked previously (MSYS2 did not lock up and CI could continue). This seems not to be true any longer. :-/
@Ede123 there are no solution for it. This warning is always when upgrade core packages. You must close MSYS2 console, reopen it and continue upgrade packages. There are no other way
Yes, I know the message, but previously the console could be closed and there was no locked process reported by mintty. Nowadays a process remains that needs to be killed.
Also note that for this specific reason I chose to run pacman once directly on Windows console before starting a bash and continuing the update. This still allowed to workaround the issue previously, allowing CI to succeed despite upgrades of msys2-runtime.
@Alexpux I found the issue:
- if pacman is run with
--noconfirmit does not wait but return immediately after full system upgrade, see here - however if pacman is run with
--noconfirmit assumes "no" for the question$1 and $2 are in conflict. Remove $2? [y/N] - in my last change I removed to
--noconfirmto account for #1139 which in turn caused the blocking behavior
I think we simply need to either a) change the default answer to the "remove?" question b) make pacman return in this case
Code introduced in https://github.com/Alexpux/MSYS2-pacman/commit/6e0ff8324dca152b312a2d39ab025f9888896b9c
Even found a workaround for it that does not require a code change:
Pacman has an undocumented option --ask that takes a number, which is then converted into a bitmask and is used to override the default answer to questions (so we can use --noconfirm again).
The bit mask can be derived from the question type as defined in the enum _alpm_question_type_t, the default answer is overridden here.
It's certainly ugly and the option was even removed at one point for being "non-user friendly" but added back later (mainly for internal usage, though) but it allows CI to work again!
Maybe removing the blocking behavior is still the easiest solution (probably better than using an undocumented option?). Otherwise we could still attempt to change the defaults, but there does not seem to be a straightforward way to do it (at least I did not find the defaults collected in a central place, it seems they're defined at multiple locations in the code...)
@Ede123 nice find! As far as I can see it just reverses the default answer, and only two commands are defaulting to NO using noyes(), ALPM_QUESTION_CONFLICT_PKG and ALPM_QUESTION_REMOVE_PKGS, so "--ask 20" should make it default to yes for everything.
Ah, yes, I was looking in the wrong place yesterday and assumed "0" was the default answer to all questions, but it is overwritten in cb_question, so --ask=20 should be the proper value.
What about patching MSYS2 pacman to use yesno() everywhere and therefore default to yes? Or do we find that too "risy"? It would certainly be easier than to require everybody who wants to do unattended updates to use an undocumented option...
Sounds good to me, although in this case it will probably not help appveyor users as the current error will prevent pacman updates.
I created a PR for the change, unfortunately pacman does not build for me right now :-/
@Ede123, will --ask 20 still work with your changes? Imagine that i have an old msys2 installation, to update from the old pacman i can use the workaround but after the update to the new version what will happen? Can i add --noconfirm --ask 20 and forget forever?
Since i can never be sure what version of pacman i have on remote host your change will actually complicate things for me :confused: . It means i can't write a script to unattendedly update from unknown version to the last one.
Yeah, to be honest, diverging behaviour on the same flag as arch pacman sounds bad. The patch would be better if the new behaviour was another option, or better yet, get this change (in some form) upstreamed. Or just stick with --ask 20...
I too think the better option is to somehow "document" undocumented --ask and return to the original behavior. I doubt that people expect such change in the pacman habits and will be really surprised. Not everyone follows msys2 commits.
it's up to you of course.
Unfortunate that you comment now that the change was merged...
I still believe this is the best approach (otherwise I would not have proposed it):
-
--askis not documented, i.e. we can assume people don't know about it and don't use it (especially when they do not follows msys2 commits). - Without this change there is (except the undocumented internal flag) no possibility to run pacman really unattended.
If upstream follows this would obviously be the ideal case, otherwise I can not see how this can be unexpected or surprising to people:
- When running unattended it's the behavior they most likely want
- When running interactive it's obvious what the defaults are and I believe people are intelligent enough to make the proper choice (I doubt muscle memory is a factor for these rare queries)
Personally I don't care what solution we choose for allowing unattended full system upgrades as long as it's reasonable - --ask is not that solution for me (remember it was specifically removed by pacman developers for not being user friendly and I agree - it should not be an option used be end users).
I agree that --ask 20 is not ideal, but given that the use case is CI, you would only have to set it once and forget about it, making it more or less a non-issue.
I also agree that having some way to confirm with yes (and without using an undocumented flag) is a needed feature - but I disagree with changing the default behaviour of the flag (and which I infer now also breaks the --ask flag). I disagree because it's a breaking change in the subtlest of ways - sure, it may not be what you would want the flag to do as it stands, but there may already be things depending on this behaviour of not overwriting/upgrading what exists. It's also another behavioural difference to vanilla pacman, which may surprise anyone used to that instead of this msys2 flavour.
As mentioned before, given that there is a workaround in vanilla pacman (--ask 20), this should have just been proposed upstream, or if you really must, patched as a separate flag (like --noconfirm-yes).
The main thing that really bothers me is that now it's impossible to update from an unknown version in easy way. Maybe you can just add hardcode --ask 20 to always yes (actually just ignore the switch if parameter == 20)?
In KDE Craft we have a blueprint that installs msys, its not perfect but works so far. Today I added the --ask flag to make the blueprint work for now. But as far as I understand this change, --ask will make it work for the first few pacman updates but as soon as pacman is updated to the version containing this patch it will break? I think the fastest approach here is to not patch pacman but to provide a new base package which has no initial conflict. I agree with @jtanx that the usecase here are CI and onetime scripted installs.
https://github.com/KDE/craft-blueprints-kde/blob/master/dev-util/msys/msys.py
The idea was that nobody added --ask to start with (I'm almost sorry now that I mentioned the option before) or if they did they were aware it was a temporary workaround.
If it really causes issues for people (although - how many might it affect but the few who read this issue?) then we might consider reverting.
However I still feel deeply that down the road requiring every potential CI / script user to a) find and b) understand the --ask option and to apply it properly to their scripts is an unnecessary burden.
For me personally removing the flag as soon as the pacman update has landed seems more acceptable than to require everybody else to use the arcane flag down the road. Basically the sheer fact that this flag makes it possible to cause troubles like this should be a large indication for everybody that it's usage is to be avoided at all cost.
Also consider that the same would happen if upstream decided to apply this change (and they'd be well within their rights to do so as nobody could blame them for changing the meaning of an internal option that was not supposed to be used in production to start with).
We could ask pacman upstream to add a "--yes" option which is equal to "--ask 20" now.
Or make the "warning: terminate MSYS2 without returning" warning non-blocking (does it really have to be?) so one can pipe yes.
@Ede123 as long as the base is old we still would get issue for the initial upgrade, independent of the patch?
@lazka Adding --yes seems kind of redundant to me but would be a solution.
However it adds ambiguity: How should --yes --ask=20 work? Should it set all default answers to "yes" by default and still invert, basically what the PR does now? Or should it always override --ask? Maybe a --force-yes might be more suitable then?
It also won't solve the concerns raised in this thread as this (newly introduced) command line option can not be safely added to scripts which potentially run old versions of pacman.
Removing the blocking behavior was my first hunch on how to solve this - it certainly would work and make MSYS2 pacman work like upstream pacman. The downside I saw was that having to pipe "yes" seemed hackish to start with (I even consider it a flaw in pacman itself). It's counter-intuitive as one needs to remove the --noconfirm for it to work (which seems like the "proper" way to script pacman). Right now it seems to be the only option which would achieve both, solving this issue while keeping backwards compatibility (which I still don't think should be a hard requirement, though).
@TheOneRing Yes, I think with the old pacman --ask=20 is the only viable workaround (besides an interactive update). As I think this is a fundamental flaw in pacman I aimed to resolve it at its base with my PR.
FWIW, while pacman itself is not really targeted at fully noninteractive scripting use, the pacutils collection of libalpm frontends (created by one of the lead pacman developers) has a pacinstall/pacremove/pactrans program which contains a series of options under the heading Prompt Disposition Options.
There is a catchall option too:
--yolo
Set all prompt disposition options to their all or yes value and set --no-confirm.
So for completely unattended use, pacsync && pacinstall --sysupgrade --yolo will automatically install all upgrades without user interaction, while automatically answering yes to all questions or otherwise optimistically attempting to resolve any prompt option.
Maybe you'd be interested in packaging pacutils?
Could pacman kill MSYS2 by itself by sending a signal to the parent? (based on a command line flag?)
