open-build-service
open-build-service copied to clipboard
openSUSE
Staging list --supersede -> results in weird issues
I somewhat think what I observe could be a regression from https://github.com/openSUSE/open-build-service/pull/8946 probably together with a racing condition between when an SR is incoming and the execution order / runtime of the staging bot
'd observed it at least twice now - latest case this morning.
gambas3 (sr#766204) was staged in adi:11; a newer submission was incoming for the same package (sr#766549 at 10:20).
at 10:23, the package old package was decline by sbot (list --supersede). Which should then replace the package in adi:11 with the new one.
> osc rq show 766204
Request: #766204
submit: Education/gambas3@97 -> openSUSE:Factory
Message:
<no message>
State: declined 2020-01-23T10:23:51 staging-bot
Comment: sr#766549 has newer source and is from the same project
Looks all right, but at the same time tbe bot crashed with:
Server returned an error: HTTP Error 403: Forbidden
No permission to decline request 766204
[script-executor] Script completed with exit code: 1.
The result now is that adi:11 still has 766204/gambas 3 tracked (state declined), but the package container in adi:11 no longer exists (package was unstaged according to the history)
Since 766204 is still tracked though, without a package being present, it is not possible to unstage it anymore - and a future run of staging list --supersede results in
Server returned an error: HTTP Error 404: Not Found
Error getting meta for project 'openSUSE:Factory:Staging:adi:11' package 'gambas3'
The easiest workaround to recover is to delete adi:11 completely and have a new adi be re-setup. I did not do that yet, in case somebody wants to look into more metadata of the proj which would vanish when I delete it
sbot killed the by_project review before trying to unstaging it:
- "195.135.221.2" staging-bot [23/Jan/2020:10:23:51 +0000] "POST /request/766204?cmd=changereviewstate&newstate=declined&by_project=openSUSE%3AFactory%3AStaging%3Aadi%3A11 HTTP/1.1" 200 53 "-" "osc/0.167.0" 0
- "195.135.221.2" staging-bot [23/Jan/2020:10:23:52 +0000] "DELETE /staging/openSUSE:Factory/staging_projects/openSUSE:Factory:Staging:adi:11/staged_requests HTTP/1.1" 403 114 "-" "osc/0.167.0" 2
That case noone thought of I'd say.
Ping - this issue causes staging bot to grind to a halt quite frequently.
Just worked around this for sr 781222 by reopening it, unselecting and declining again.
Well the requests auto-declined by bots are still a challenge. I have to use something like this construct otherwise I'll not hit the < 5 sec window of auto-decline done by licensedigger. This is quite painful and annoying tbh.
lkocman@localhost:~> osc -A https://api.opensuse.org request reopen 936691 -m "reopen to unstage" && osc -A https://api.opensuse.org staging -p openSUSE:Backports:SLE-15-SP4 unselect -m "declined" 936691 Result of change request state: ok
Normal worfklow would result into failure
lkocman@localhost:~> osc -A https://api.opensuse.org request reopen 936691 -m "reopen to unstage"
Result of change request state: ok
lkocman@localhost:~> osc -A https://api.opensuse.org staging -p openSUSE:Backports:SLE-15-SP4 unselect -m "declined" 936691
Unselecting "936691" from "openSUSE:Backports:SLE-15-SP4:Staging:adi:20"
Server returned an error: HTTP Error 403: Forbidden
what @lkocman meant was: I am not using the factory plugin so I don't benefit from the workaround 😃
lkocman@localhost:~> osc -A https://api.opensuse.org staging -p openSUSE:Backports:SLE-15-SP4 unselect -m "declined" 936691 Unselecting "936691" from "openSUSE:Backports:SLE-15-SP4:Staging:adi:20"
That DOES look like using the staging plugin (otoh, unselect in combination with -m implies 'unselect and add to ignore' - which is really not what you want to do when unselecting a declined request)
@lkocman >
lkocman@localhost:~> osc -A https://api.opensuse.org request reopen 936691 -m "reopen to unstage" Result of change request state: ok
why do you reopen it? you can unselect a declined request (OBS internally reopens, unselects, declines)
@lkocman >
lkocman@localhost:~> osc -A https://api.opensuse.org request reopen 936691 -m "reopen to unstage" Result of change request state: ok
why do you reopen it? you can unselect a declined request (OBS internally reopens, unselects, declines)
That works neither for me nor @lkocman
Ah - you are not target maintainers - that must be the difference there (Staging manager role seems to be not sufficient - which is indeed an issue then)