rdev
rdev copied to clipboard
Narsil
Design decision concerning grabbing events
Adding grabbing events (meaning capturing events before they get handled by windows and such, for instance to implement global shortcuts) is underway #33
However the API can work on Windows and Mac, but I can't figure out a way to make it usable on Linux.
Current chosen API is close to listen
grab(callback: fn(event:Event) -> Option<Event>)
The return result will ignore event if it's none, and let it pass if it's some. For now we can't modify the event (it could be done, so that's why this type of return is used instead of just boolean).
If you want to simulate more keys, the idea is just to ignore the event, then send a bunch of other via simulate.
However my exploration leads me to think that it's going to be super hard to implement that kind of API on x11, here are a few reasons why:
- Using
XGrabKeyboardwill focus out the current window, and focus back in when we ungrab keyboard. And we can't simulate anything while we are grabbing the keyboard, we would need to keep grabbing and ungrabbing, but there is quite a delay (100ms+) for each action to take effect, and leads to a bunch of blinking in adding to letting users send their own key while we ungrab, with no means for us to catch it. - xinput2 has a similar API with XGrabDevice which seems pretty close to GrabKeyboard.
- Using
GrabKeywon't trigger focus in/out events, but we can only grab a key at a time (+ eventual modifiers). Grabbing means they won't ever go through, so if we GrabKey all keys, then back to GrabKeyboard problem, we can't emulate keypresses. - xinput2 has
XIGrabKeywhich let's us capture all keys at once (probably much better performance-wise than capturing all keys with several GrabKey calls). But again, there don't seem to be a way to push events back into the event queue for further processing.XIAllowEventsdoes not seem to work.
If all of the above is true, that would mean that this proposed grab api won't be able to support Linux, which is a bit bad. However, we could propose an api closer to Linux, that we would emulate on Mac and Windows.
This would probably look like
grabber = grab(key, [modifiers, ]callback: fn(event: Event));
grabber.release() // To ungrab the key
The big advantage would be that we would support Mac,Windows AND linux, which is with the theme of this library. However, the big problem is that we don't have full control of the event stream, which means modifiers are now a question, mutlikey shortcuts are hard to procure (like A+D), which would be trivial with the first API (the grabber would maintain KeyPress state of A and D, and trigger when both are down). With this other API, it become trickier, we could in theory accept a set of keys, but then it can become a bit messy, and also we can't (on x11) grabKeys that have already been grabbed by someone else (for instance if someone uses xbindkeys for instance).
Splitting the API in to is IMO a no-go.
Currently the route I took, is to hide grabbing behind unstable_grab feature, meaning you probably should read the doc, where we explicitely say that we don't support Linux. The name unstable_ should also be explicit that this API is subject to change.
I think it's the best course of action for now. I'm all ears for other suggestions.
Before I talk about the api, I want to agree on the intended behavior. What should happen if a user registers ctrl + c as a global shortcut using rdev's grab api? Should the rdev grab hook happen, or should the OS-level copying action take place? I think rdev should take precedence, and that rdev-based global shortcuts should be used very carefully by consumers of the grab api (this should be documented). They're not good for "normal" shortcuts, but they are good as macros.
Splitting the API in two is IMO a no-go.
I agree. rdev suddenly becomes significantly less useful once it has platform limitations. I think the grab PR should stay a PR until we can figure out X11.
mutlikey shortcuts are hard to procure (like A+D)
A + D is a weird shortcut, and we probably shouldn't be looking for such things. Most OS level shortcuts take the form modifier key, non-modifier key, and some use multiple modifier keys. If a consumer of the rdev API wants to look for A + D shortcuts, they can manually manipulate state to determine if A + D is being pressed.
can't (on x11) grabKeys that have already been grabbed by someone else (for instance if someone uses xbindkeys)
I don't think we should worry about what happens when someone presses a combination that already has a function in xbindkeys - conflicts will happen with global shortcuts. We should, however, document the issue to let rdev users know.
Also I have a prototype working under Linux/X11 that uses XCB instead of xlib (XCB is better in C and it's rust bindings are much better; side note I'm thinking of trying to rewrite rdev's linux code in XCB). The prototype grabs all keys regardless of application or modifiers (it grabs modifiers separately), but it has to use a hack - it makes a tiny invisible window. Expect a PR in the next ~day.
I think the grab PR should stay a PR until we can figure out X11.
I agree, but I really needed this code for somewhere else in some other code, so I decided hiding it under unstable_grab was good enough, so that I could use it where needed (without self forking) while being able to modify the API at will.
I don't think we should worry about what happens when someone presses a combination that already has a function in xbindkeys - conflicts will happen with global shortcuts. We should, however, document the issue to let rdev users know.
Good point.
Also I have a prototype working under Linux/X11 that uses XCB instead of xlib (XCB is better in C and it's rust bindings are much better; side note I'm thinking of trying to rewrite rdev's linux code in XCB). The prototype grabs all keys regardless of application or modifiers (it grabs modifiers separately), but it has to use a hack - it makes a tiny invisible window. Expect a PR in the next ~day.
Awesome that you have a working prototype !2
I have zero opinion on XCB versus Xlib, technically, but it seems Xlib is still more widely used than xcb. Are we sure XCB library is included on major Linux distributions (ubuntu LTS at the very least). We do have libxtst as a dependency right now (at least docker image does not have it). If possible I would rather remove dependencies that add other ones.
As to grabbing global events, you are able to capture them which is fine, but how do you send them back ? I personnally tried something along XSendEvent(display, InputFocus, True, KeyPressMask|KeyReleaseMask, event) which is a proposed solution in your link, but for some reason it did not quite work, maybe because the hidden window trick does things differently ?
Also XGrabKeyboard is supposed to steal the focus, does it not ? (I think this is what makes InputFocus not really relevant, and event trying to bypass that did not work).
Eager to see your solution.
unstable_grab was good enough, so that I could use it where needed
Yeah, that works too - you can always come back and add the feature to Linux. I wrote the first part of that response before you added the feature. As a side note, are you using this library for a personal project of some sort?
XGrabKeyboard is supposed to steal the focus, does it not
Yeah, it is supposed to, but it doesn't work unless you have a mapped window for some reason (the first answer on the previously linked question says that). XCB only makes is easier by having grab_keyboard, which separately grabs modifier keys. I'll look in to if the major distros include it before I ask that you merge a PR.
how do you send them back
I'm still working on it, but the api I use to grab them gives modifiers as normal key presses. So I just use key_from_code() on the key code I get, then simulate() that keypress.
I personnally tried something along XSendEvent(display, InputFocus, True, KeyPressMask|KeyReleaseMask, event) which is a proposed solution in your link
The fact that I'm using XCB here makes that answer mostly irrelevant, just the invisible window hack is relevant. I'm really not sure how xlib deals with modifiers, but it looks difficult...
Are we sure XCB library is included on major Linux distributions
I looked in to this, and Xlib depends on XCB, so wherever there is Xlib (what we're using now) there is XCB. That said, distros like ubuntu server and base debian don't ship X11 at all.
Then I'm all up for XCB in favor of xlib, especially if wrappers are already written !!
For now we can't modify the event (it could be done, so that's why this type of return is used instead of just boolean). If you want to simulate more keys, the idea is just to ignore the event, then send a bunch of other via simulate
If you need to call simulate directly to have full flexibility (being able to double button presses, etc.) why not make it the default, not even returning Option?
grab(callback: fn(event:Event)) -> ()
We need to know if we should block or no, so at least a boolean is needed, no ?
Why would we ever block? Based on the original issue, I thought this was basically an event filter, and that you planned to make it an event filter+map in the future. But telling the user to simulate all needed events is more powerful than a filter+map, as it can turn single clicks into double. Basically if the full expressivity requires simulate, then we should make that the default way to grab.
Yes I failed to explained what I wanted to convey.
You need a mechanism to PREVENT some events from continuing to go in the event queue. So a boolean is required to that the OS-level code knows if it should block any given event.
Modifying an event instead of simulate might lead to lower latency in simple cases. It's my guess as to why Both Windows & Mac do it this way. For assisting technology, for instance autorewriting everything in CAPS, it might make sense. Do you think there is a bad tradeoff in the general case for returning Option<EventType> instead of boolean ? (The size of Option<EventType> vs boolean seems negligible but I might be wrong).
Do you think there is a bad tradeoff in the general case for returning Option instead of boolean?
No, and I think the issue of sizes is negligible. I do object to returning an Option<Event> where we only check if the Option is_some(), and not the value it contains. I think we should use a boolean for now, and change the API to use an Option once we figure out how to modify the event efficiently.
I've been messing with the PR to make grabbing work in Linux/X server and I'm having difficulty. One option I'm considering is using uinput, not X. Grabbing with uinput would require super user rights, so would make this library harder to use on Linux. But based on a little demo I made, it actually works, unlike the listen and re-simulate attempt I made prior.
@Narsil What do you think about using uinput? Do you think it is worth trying?
I know it works, and it's lower level, but super user rights is probably a no-go, for 99% of users.
As we don't really have an alternative right now, I'm ok to use that behind unstable-grab but it would definitely need another approach to move really into the lib.
And fair enough for using boolean instead of bool as we indeed are not checking for something else.
I really don't understand why grabbing is SO hard on X, is it by design ?
I really don't understand why grabbing is SO hard on X, is it by design ?
At this point, I'd say almost nothing in the X server is by design. It was designed to go over a network in the 80s when windowing systems were a novel topic of academic research, then slowly added to until it became the mess it is now. Window managers try to go around it as much as possible to avoid the weirdness. That said, I don't think rdev's use case was intended or planned for.
super user rights is probably a no-go, for 99% of users
The fact that keyloggers don't require root on Linux/X11 and Windows is really unfortunate.
I'm ok to use that behind
unstable-grab
I have a prototype under https://github.com/maxbla/gel-o. It has a pretty rusty api.
pub fn filter_map_events<F>(mut func: F) -> io::Result<()>
where
F: FnMut(InputEvent) -> (Option<InputEvent>, GrabStatus)
It even allows you to stop grabbing by returning (_, GrabStatus::Stop). I think it could be pretty easily adapted to rdev's api of grab(callback: fn(event:Event)) -> bool. The only difficult part is creating a function to convert between rdev::Event and evdev_rs::InputEvent. I'll continue to work on gel-o, and get a PR ready in the next few weeks to bring some of it's code into rdev.
If you have a solution, I'll happily check it in (probably in unstable-grab still).
Libevdev is lower than X to achieve this ?
Libevdev is lower than X, as it operates on "device files" in /dev/input/, just above the kernel level. It is possible to mess with device files more directly with ioctl and write syscalls, but libevdev has a more friendly interface. Using libevdev requires root, but allows the code to work on X, Wayland and even the Linux virtual console, with no GUI at all.
Because of the added functionality, I think it makes sense to include a libevdev backend (hidden behind a feature flag), even if we get an X grabbing feature working.
If you have a solution, I'll happily check it in (probably in unstable-grab still).
I'm still working on a feature that re-grabs devices if they are unplugged and plugged back in. Right now, they are no longer grabbed if that happens.
If that's too much work, leave it be, I think unplugging replugging keyboards is not that common. As long as we don't freeze the OS that's ok.
Actually I could use your Linux version :)
Hello all,
I think it's absolutely wonderful the work you both have put in to making input grabbing work on linux. I think using evdev to do it is definitely the right choice, even if it does mean super-user privileges are required
Has there been any movement on this front since last June?
I'd love to help move this forward, but i don't have much expertise in C or linux APIs.
If you need someone to test something, I'd be happy to help. I have desktops and laptops running Linux, Windows, and MacOS
Nothing much has changed, actually you don't need super user, but adding the running user to evdev group is enough (which is infinitely better than giving root access)
ahhhhh.... looking through the source code, i see that grabbing in linux HAS been implemented, with evdev no less.
The project README and the contents of this open issue had me thinking that unstable_grab hadn't been implemented in linux at all.
would you like me to submit a PR to update the documentation regarding use and caveats for unstable_grab?
Hello, I've submitted a PR for the documentation update as discussed :)
By the way, have you had a chance to look at my Issue / PR regarding the Display trait for Key enums?
I'd really like to get that merged if possible
Does it actually grab before windows can get the event? I noticed windows overrides certain keys I try to grab, such as ALT or any system shortcut (ctrl+plus).
