kinto
kinto copied to clipboard
rbreaves
BUG: xkeysnail v0.4.0 breaks Kinto held keys behavior (breaks Alt+Tab, Alt+Grave)
Editing to add the most reliable fix, for anyone who is still having a problem with this:
Go into your Downloads folder. Delete "kinto.zip", and the "kinto-master" folder that was extracted from the zip. Go back to https://kinto.sh and copy the URL installer command and run it in a terminal again. Problem should be solved. Kinto should work.
####################################################################
Tested in both VM and bare metal, installing from the Kinto.sh URL installer.
I can unfortunately confirm that the current version of xkeysnail that is being installed on fresh installs of Kinto seems to have broken "held keys" behavior, which leads to unexpected and broken behavior from some of the most used and fundamental shortcuts: Alt+Tab and Alt+Grave. Since the keystroke is immediately released by this version of xkeysnail, the task switcher dialog won't appear, and the user can't switch between any but the last two applications or windows.
I described this behavior in both the thread on installing in CutefishOS (issue #607), and the thread about Kinto being completely broken since mid 2021 on Fedora (issue #564). CutefishOS is using kwin, while Fedora's default variant is on GNOME v41. The problem has to be xkeysnail, and this issue has to have existed for at least 11 days. The kinto-master folder in my Downloads folder is dated January 5th, 2022. Sometime between those two dates, something has gone wrong with xkeysnail.
Since the xkeysnail in /usr/local/bin/ is just a stub that runs the python3 module, I had to track down the actual xkeysnail v0.3.0 files installed on my Ubuntu 21.04 system in /usr/local/lib/python3.9/dist-packages/, and then copy them to another machine while removing the installed v0.4.0 version. On Fedora 35 (Beta) the path is python3.10 instead of python3.9. This reverted the version of the xkeysnail that gets called from v0.4.0 to v0.3.0.
This substitution of an older xkeysnail immediately solved the Alt+Tab and Alt+Grave issues on the Fedora install I was testing.
A big problem getting around this is that the xkeysnail project has no "release" files, so most people (including me) will have no idea how to pull an older version from the GitHub repo. If I didn't have an existing, working copy of a v0.3.0 install, I would have had no way to even test this out to confirm the problem. Installing an older release of Kinto will still clone the latest xkeysnail.
So I'm posting a zip of my working v0.3.0 xkeysnail version here in case anybody needs it (or in case I need it again):
As referenced above, these folders need to replace the xkeysnail folders for v0.4.0 that are in some variant of:
/usr/local/lib/python3.9/dist-packages/
User must open the folder as root to delete the newer xkeysnail folders and drop in the new ones. The simplest technique for this is probably:
sudo xdg-open /usr/lib/python3.9/dist-packages/ &
Closing this as a false alarm sorta thing - I am aware of the "bug", but really it is just the behavior they decidedly preferred some time ago. My behavior will need to be broken out into a separate function which I have not spent time to write still, so it can be remerged into a 0.4+ version.
At any rate Kinto uses my fork, not the official branch or even the master branch on my fork, it explicitly uses the kinto branch on my fork of xkeysnail which is on 0.3.x. You were likely just doing testing of various behavior of xkeysnail in general and not realizing that you needed to check back out my kinto branch and re-install from there. No need to be using old resources or zipping anything up, just checkout the branch that exists for this and install from there.
https://github.com/rbreaves/xkeysnail/tree/kinto
git checkout kinto
you can also do git checkout - if you are switching back & forth btwn 2 branches.
I do try and keep my master branch in sync with the actual xkeysnail project while my production branch remains with the name kinto. A little confusing but I prefer it that way and makes sense to me, otherwise I would need to create a new branch that I'd then sync with xkeysnail any time they update it, would rather not do that.
@rbreaves
I'm very confused by your response to this. I think you're imagining me doing things that I'm not doing.
At any rate Kinto uses my fork, not the official branch or even the master branch on my fork, it explicitly uses the kinto branch on my fork of xkeysnail which is on 0.3.x.
As I said, I was installing from the URL at https://kinto.sh . Which means if it was installing the "wrong" branch of xkeysnail then everyone using that link will be getting this same "wrong" version of xkeysnail that doesn't seem to contain your patch for "held keys".
I'm not doing anything with git other than what the installer URL does by itself. When I install from the URL, the version of xkeysnail that gets installed is v0.4.0, and it doesn't have your "held key" patches.
I have absolutely no idea how anyone could end up with a different version of xkeysnail than what I'm ending up with when I use the installer URL from your site.
Since that site went live and we fixed up the URL so that it usually works if there is wget or curl installed, I have generally not bothered to try to install git myself and do any kind of manual clone.
I also assumed that you would be cloning your own branch of xkeysnail with the installer, but that doesn't appear to be what is happening. There is no other explanation for the issues with Alt+Tab I'm having on new installs.
Please set up a new VM with any distro and attempt to install Kinto from the URL installer link, and then see what happens with Alt+Tab and Alt+Grave after you get it installed, and what version of xkeysnail is installed.
I just checked once more and the version of xkeysnail that the Kinto installer URL installed in the Fedora VM is v0.4.0. Not your v0.3.0 branch. Why? I have no idea.
I can't test immediately right now regardless.. but the setup installer literally says this - it'd make no sense for it to not be on the kinto branch. Read it last night before I went to bed.
xhost +SI:localuser:root
git clone --depth 10 https://github.com/rbreaves/xkeysnail.git
cd xkeysnail
git checkout kinto
giturl=$(git ls-remote --get-url)
if [ "$giturl" != "https://github.com/rbreaves/xkeysnail.git" ];then
echo -e "\nreplacing xkeysnail with fork...\n"
cd ..
rm -rf ./xkeysnail
git clone --depth 10 https://github.com/rbreaves/xkeysnail.git
cd xkeysnail
git checkout kinto
fi
I just ran into this problem after installing Pop OS on a macbook pro. I can only switch between two applications now. Not sure what I'm supposed to do to make Kinto work again :(
I apologize @sommerper & @RedBearAK, I would have jumped on it sooner.. but yea when I read the code of the script I was pretty certain last night that it was correct - it was not. And too I was tired last night and this morning.
This occurred probably about a week ago when I was doing some testing of a PR from xkeysnail - I had merged it into master thinking my code was properly checking out the kinto branch.. it was not. To ensure this does not happen again I have reset master on the xkeysnail fork back to 0.3.0 & the kinto branch is also on 0.3.0 as its intended purpose.
Going forward if there are major changes to functionality of my xkeysnail fork then I will evaluate whether to create new kinto-v1.X sorta branches on the xkeysnail if backwards compatibility is not kept during future revisions.
So yea - I fixed it in 2 locations to make sure this breakage should not occur going forward - even people that have older versions I believe will be ok now - or a clear error may appear since they would now be ahead of master.
This was changed (guessing the depth command prevented other branches from coming down)
git clone --depth 10 https://github.com/rbreaves/xkeysnail.git
git checkout kinto
to
git clone -b kinto --depth 10 https://github.com/rbreaves/xkeysnail.git
And like I said going forward I will likely create new branches if it would break anything w/ older kinto release as new ones are released.
Re-opening till I get confirmation to close.
@rbreaves
Regret to inform that this will apparently not automatically fix a system where Kinto with xkeysnail v0.4.0 is already installed. The script in the downloaded folder does have the new form of the command, but it says things about "uninstalling v0.4.0" and then immediately reinstalls v0.4.0.
Got an error when trying to pip3 uninstall xkeysnail so I tried manually wiping the xkeysnail folders in /usr/local/lib/python3.10/site-packages/ and running Kinto setup again. It still installed v0.4.0. Maybe from a pre-existing download of xkeysnail?
I can see with the rest of the language that is supposed to make this happen why you'd think this was impossible. But it looks like it was one of those, "But it always worked before," type of errors we all make with computers.
I'll try again at some point with a clean VM to see what a fresh install will do, but at the moment this is not a fix for any system that has already been afflicted by v0.4.0.
A few minutes later:
...OK, it looks like the minimum that needs to be done is to delete the xkeysnail folder within the existing kinto-master folder, or to delete the entire kinto-master folder. I'm assuming that’s because it needs to be downloaded during install with git, the xkeysnail folder doesn't exist within the downloaded zip file, so when the installer URL expands the files and ask if you want to overwrite the existing files (I always choose "A" for [A]ll) it overwrites everything but leaves the xkeysnail folder from the previous run alone. Then when Kinto setup runs, somehow it doesn't overwrite the v0.4.0 xkeysnail folder with the v0.3.0 version, so it just re-installs the v0.4.0 version, without patches.
I would not expect the typical user to understand why any of this is necessary, so I would strongly suggest altering the installer script to either rm any pre-existing xkeysnail folder from within the kinto-master folder before cloning, or have the script just rm the entire kinto-master folder instead of asking the user if it should be overwritten or skipped, or add some syntax that would force the clone to overwrite the existing version of the xkeysnail files within the existing xkeysnail folder with the kinto branch, which it doesn't appear to be doing at the moment. I'm leaning toward just removing the whole kinto-master folder before running install being the most reliable option. It's actually quite confusing for the user to be asked in the middle of the install if they want to overwrite the existing folder. Which happens any time the setup is run more than once.
The new form of the command by itself in a clean folder does indeed download v0.3.0 with the patches. For some reason it just doesn't want to overwrite the existing xkeysnail download.
So for anyone out there who needs to fix this problem, go to your Downloads folder and delete the kinto-master folder before attempting to use the online URL installer, or a manually downloaded new Kinto installer, again. That will force it to download and install a clean patched v0.3.0 xkeysnail branch, and everything will work again. Puppies, unicorns and rainbows will appear.
@sommerper
I may should have reverted the changes instead but the roll back & force push is out there now. Will deal w/ it later but yes Git does not know how to deal w/ a diverged branch automatically hence you needing to do either a Git reset --hard HEAD~6 then git pull or yes remove the folder & reclone.
something like this fetch then pull would fix it proper. https://stackoverflow.com/questions/1125968/how-do-i-force-git-pull-to-overwrite-local-files
Sorry to cause you this much confusion w/ git, it’s fairly logical though given how git works.
@rbreaves
Yeah, I’ve noticed that most things involving computers are extremely logical. The trick is to fight your way through the dark cloud of self-induced confusion about all the other ways something might work, leaving only how it actually works.
I have a fanciful imagination at times, so this essential process of getting to that, “Oh, well, duh!” moment tends to be long and extremely painful.
Hence why I try to advocate for solutions that will work regardless of any knowledge level on the part of the user.
@rbreaves
Hmm, I'm not sure exactly where the failure is, but I just tried running the bash URL from the website again inside the afflicted CutefishOS VM install, and the result was that it left the v0.4.0 installed.
I think somehow it is not successfully getting into the if statement where the rm -rf ./xkeysnail is located. Because if it was, that rm should have solved the issue.
rm -rf ./xkeysnail Only kicks in if the xkeysnail is not my fork - there’s no logic added yet to check the commit version for 0.3. I’ll have to add that.
@rbreaves
The URL I get back from running the git ls-remote command inside the xkeysnail folder with the "wrong" v0.4.0 version is a match for the URL inside the if statement test, so the if statement fails to activate.
I think you need to be doing the rm -rf right after the xhost line, so that the whole xkeysnail folder gets wiped before the initial clone.
@rbreaves
rm -rf ./xkeysnail Only kicks in if the xkeysnail is not my fork - there’s no logic added yet to check the commit version for 0.3. I’ll have to add that.
I see what you mean. We basically said the same thing again, in different ways. Either looking for "not v0.3.0" as an "or" condition or doing another rm just after the xhost line should allow this issue to be closed. I'd recommend both, but especially doing the rm outside of the if and before the first clone would be the sledgehammer that makes sure Kinto can repair itself on an already affected system.
Will add the logic later, I do not think it has impacted enough users to matter that much - but yes the solution atm is to remove it entirely and start over. I can add a statement later to check the commit version & just ban the 0.4.0 commit hash - if it is found then do the git reset I mentioned and pull command and things will be back to normal. I am a little busy at the moment though, but I did definitely take the time to make sure no future downloads or users will run into this. Leaving it open for now so that it has visibility for now & till I can do a follow up commit.
Also in general though I do not want want to check for a specific commit - as updates should be able to happen freely - but yes specific commits that I know are not compatible with Kinto should be banned in the future. Might be good to add an additional check in the Kinto app to check for what xkeysnail version/build/hash is installed so as to warn the user as well.
@rbreaves
do not think it has impacted enough users
No way to know how many there could be, without download stats specific to the branch the URL installer is using.
Might be good to add an additional check in the Kinto app to check for what xkeysnail version/build/hash is installed so as to warn the user as well.
This I would agree with as a good idea. But with the branch fixed and locked in place this theoretically shouldn't be an issue again anytime soon.
I'd much rather see any available time focused on moving Kinto away from using sudoers to running as user, as the highest priority. It solves so many problems and will get Kinto working on a number of distros that had major issues with that approach. Including Fedora.
@rbreaves No, problem. I appreciate you taking the time to look into it. I tried uninstalling Kinto and removed xkeysnail 0.4.0 with pip3. Then manually installed xkeysnail==0.3.0 with pip3 before installing Kinto again. xkeysnail claims it's now at 0.3.0 but the problem with Kinto persists.
@sommerper that’s odd. What DE or distro are you on?
when you say pip do you mean you went into the xkeysnail directory & ran?
sudo pip3 install --upgrade .
pretty sure master is rolled back to the same as the Kinto branch. Would suck if I lost any more recent commits but I can quickly check that.
yea.. it’s the right commit, starts w/ 51c3690.
@rbreaves
Ben, sounds like what he did was manually install mooz’s xkeysnail with pip3 completely independently of the Kinto installer, so it was v0.3.0 but no patches from your fork. That’s the only way the problem would remain. (Edit: This is actually wrong.)
@sommerper
Try removing xkeysnail again with pip3, then delete the “kinto-master” folder from your Downloads folder, grab the installer URL from https://kinto.sh again and run it in a terminal. The Kinto installer downloads a patched version of xkeysnail from the “rbreaves” forked repo that works slightly differently. It’s not just about the version number. (Edit: Part of this is wrong, but the first part should still work to fix the problem.)
@rbreaves @sommerper
Apologies, it seems I was wrong on part of that.
I haven't been able to replicate what I thought would happen if I installed xkeysnail v0.3.0 with pip3 independently of the Kinto installer, even after removing Kinto with ./setup.py --remove and uninstalling xkeysnail with pip3. If I do:
pip3 install xkeysnail==0.3.0
This does actually install a version of xkeysnail that has the necessary patches. I'm not running this command from within the "kinto-master/xkeysnail" or even the "kinto-master" folder, so this confuses me. This means @sommerper should definitely no longer have any issue with Alt+Tab after doing this and reinstalling Kinto.
But I did notice that Kinto will change the path to xkeysnail in the systemd service file if there is a "local" version of xkeysnail installed in the user's home .local/bin. This will happen if the user doesn't do sudo pip3 install xkeysnail but instead pip3 install xkeysnail.
So that is a potential source of confusion. I have been able to create a situation (with --force) where v0.4.0 (with no patches) is installed in the home folder, and the service file is using the "/home/$USER/.local/bin/xkeysnail" command to launch the config file. But even under that circumstance the Alt+Tab works fine, apparently because the xkeysnail stub file is actually calling on the v0.3.0 version that is installed at the system level, and ignoring the v0.4.0 installed at the user level.
And, even though Kinto uses the "user" command path in the service file if there is an xkeysnail in ~/.local/bin, it still successfully overwrote the v0.4.0 version that I manually upgraded with pip3 at the system level, and reverted that back to v0.3.0.
Still can't find any combination of these actions, even using --force, that would result in a non-working Kinto right now.
This is testing with an install of Fedora 34 in a VM.
The only possibility that remains is the user failing to delete the "kinto-master/xkeysnail" folder or the entire "kinto-master" folder from Downoads, leading to the continual re-installation of a previously downloaded v0.4.0 xkeysnail with no patches.
@rbreaves
Ben, I ran that sudo pip3 install --upgrade . command inside the "Downloads/kinto-master/xkeysnail" folder on the CutefishOS VM that I know is still afflicted with a pre-existing v0.4.0 version in that folder. The command just reinstalled the v0.4.0 version. I had to delete the xkeysnail folder to get it to re-clone and then it installed v0.3.0.
Leaving the Downloads/kinto-master folder intact with an old clone of xkeysnail v0.4.0 (and older version of the installer script?) is the only circumstance I can come up with where the Kinto setup will continue to overwrite a v0.3.0 install, manually installed with pip3, with the unpatched v0.4.0.
@sommerper
After you reinstall Kinto, what is the result of:
xkeysnail --version
@sommerper
Very short version of fix:
Delete the entire ~/Downloads/kinto-master/ folder and the kinto.zip file, and re-run the Kinto install from the bash URL:
/bin/bash -c "$(wget -qO- https://raw.githubusercontent.com/rbreaves/kinto/HEAD/install/linux.sh || curl -fsSL https://raw.githubusercontent.com/rbreaves/kinto/HEAD/install/linux.sh)"
That should clear everything up. Regardless of whether you manually installed or uninstalled any particular version of xkeysnail.
@RedBearAK @rbreaves Sorry, to leave you guys hanging for a while and for not being clear enough on what I was trying out.
@RedBearAK Think you so much for pointing out specifically what I had to do. I never deleted the ~/Downloads/kinto-master/ manually because I thought it was being overwritten by the script. Now I unstalled Kinto. Removed the downloads folder and reinstalled. And everything works again!
Thank you so much! I can use my computer again :)
@rbreaves Not that it matters anymore but I'm running Pop!_OS 21.10. ;) Thanks for taking the time to help.
@sommerper
Glad that fixed it.
Just FYI if you're new to Pop!_OS:
Pop!_OS has broken GNOME's ability to do "Switch Applications" like macOS, which is why I don't use it. For anyone who cares about Mac-style task switching with Alt+Tab, I recommend Ubuntu 20.04 LTS, which will soon be upgradeable to 22.04 LTS and will be on GNOME 42 at that point.
Closing this since the issue was solved and there shouldn't be anyone affected by it anymore.
This issue came back for me on Ubuntu 23.04 Lunar Lobster. After I upgraded Kinto was working weirdly and I realized xkeysnail 0.4.0 was installed. I uninstalled it but then the installer failed because of Python's new externally-managed-environment restrictions (see https://stackoverflow.com/q/75608323 for a similar issue).
I had to override that setting and manually install xkeysnail 0.3.0. Then the binary wasn't in place where Kinto expected it at ~/.local/bin or something like that, so I had to create my own "binary" file with a python3 shebang that loaded xkeysnail like the previous one had and give root ownership over it.
Now everything is working...but just reporting this in case other Lunar Lobster upgraders have similar issues. Man the externally-managed-environment stuff is kind of annoying too when everything was working fine before it, I think I'm just going to disable to globally as I really don't work in Python much anyway, no offense to the maintainers that introduced this - I'm sure it's helpful for folks working in Python a lot.
@zachfeldman Do you mind sharing the commands you ran to get it working? I think it'll save time for others coming here from Google :)
@zachfeldman @nikolayg
The issue that started this thread was actually fixed quite a long time ago. The Kinto installer was accidentally pulling the wrong branch of xkeysnail that didn't have the patch for "held keys". That was fixed within a few hours, over a year ago. But I'll re-open this if it's useful for now.
I'm wondering if the installed version of xkeysnail is somehow getting "upgraded" during the upgrade to Ubuntu 23.04. The only way you should end up with xkeysnail 0.4.0 installed is that something does pip install --upgrade xkeysnail which will grab the main source of xkeysnail and overwrite the patched version Kinto uses with the unpatched v0.4.0.
Either way, the solution is most likely just installing pipx like I did in the thread about Xubuntu 23.04, and using it to install the patched version of xkeysnail from the "xkeysnail" folder that is cloned (by the Kinto installer) into its ~/Downloads/kinto-master folder. I did that, and was then able to re-run the Kinto installer by running the setup.py file in that folder, and get through the rest of it.
sudo apt install pipx
cd ~/Downloads/kinto-master
pipx install ./xkeysnail
python3 ./setup.py
@RedBearAK sorry for the delayed response here! I think your solution of using pipx is much more proper than what I did, globally installing the right version of xkeysnail by overriding Python's new suggestions of not doing this, your solution should probably be the preferred one for this issue.
I'm a little unclear on if it was the upgrade itself to 23.04 that did this or something related to it. I'd suggest keeping this open a bit longer to see if anybody else reports this, maybe the install script needs to be tweaked during this upgrade in some way. Or there could be another small script to run on upgrade to 23.04, or people can just follow your pipx instructions.
It might be good to update the title of this issue to include "on upgrade to Ubuntu 23.04" to help searchers find it, I don't seem to have permissions to do that :).
@zachfeldman
Actually, I think it was sort of a fluke that pipx helped get through the installer. I think it only works because xkeysnail is the only package that needs to be installed with pip, and xkeysnail loads the config file directly, so the isolated environment that pipx creates for xkeysnail didn't end up causing a problem.
Any project that needs to install multiple packages via pip will probably have to do what I just did in my own project and use a full Python virtual environment, and use some shell scripts that activate the virtual environment in order to launch the keymapper in a way that will work, without messing with any system Python packages.
It's kind of a pain, but it's just one command to create the virtual environment folder, and then one command to activate it in any launcher script.
This issue thread really wasn't about this particular problem, and there are other threads already talking about the issues with the Ubuntu 23.04 variants. I don't think it makes sense in this situation to change the title. The solution proposed in the edited first post has no relevancy to what is going on now down here in the thread. Most of this should really be in newer issue threads.