SavinaRoja
[Enhancement] Provide a "wheel" method for pymouse
I am using PyMouse for my application and it corresponds perfectly to my needs! Thank you very much. Yet, I want to be allowed to scroll within my app with the wheel (not a middle click). Is it possible to provide this easily within PyMouse?
I have never used XLib, so I have no idea how to implement it by myself.
Thank you very much, I really appreciate your module.
PyMouse was primarily developed by @pepijndevos , so I'll give em some time to weigh in on this. I developed some experience with XLib when I wrote the X11 version of PyKeyboard; I am pretty sure that this is feasible and probably won't be too difficult to do. I can probably write the X11 and Windows versions, I don't have a Mac so I'd have to scrounge up some help in testing.
I am imagining that we can provide a simple framework and leave it to users to build on it for more sophisticated event generation and interpretation.
Thank you very much. I'm really glad that such a module exists. I'm looking forward for your enhancements. I might send a pull request with the wheel method if I take time to learn XLib
Paul ECOFFET Le 10 juil. 2013 19:50, "Paul Barton" [email protected] a écrit :
PyMouse was primarily developed by @pepijndevoshttps://github.com/pepijndevos, so I'll give em some time to weigh in on this. I developed some experience with XLib when I wrote the X11 version of PyKeyboard; I am pretty sure that this is feasible and probably won't be too difficult to do. I can probably write the X11 and Windows versions, I don't have a Mac so I'd have to scrounge up some help in testing.
I am imagining that we can provide a simple framework and leave it to users to build on it for more sophisticated event generation and interpretation.
— Reply to this email directly or view it on GitHubhttps://github.com/SavinaRoja/PyUserInput/issues/10#issuecomment-20759852 .
So I just took some time to look over X11's implementation of PyMouse and PyMouseEvent and I have a few observations.
The first point I should make is that in X11, mouse wheel events are essentially the same as the other mouse click events. If you so choose, you can can do
mymouse.click(x=<int>, y=<int>, click=4, n=<int>) to scroll up, and
mymouse.click(x=<int>, y=<int>, click=5, n=<int>) to scroll down.
Another thing I noticed is something that appears to be a bug in PyMouseEvent, which is that mouse wheel events are detected appropriately, but PyMouseEvent.handler() returns the wrong button key numbers. I will fix this and do some testing to make sure I don't break a feature from @pepijndevos . This will change some behavior.
Finally, I like the idea of providing separate methods for mouse wheel actions; even though it might be redundant in X11, it is more natural to use and simple to provide a cross-platform interface. Given a few minutes, I can push out a new branch with the fixes and the new methods in X11, if it goes smoothly then I will work on the other platforms.
Thank you so much, I am really glad that you have taken my request into account that quickly.
Yet I have a question, I have installed pyuserinput through easy install, will the new push you'll make be quickly available on pypi or should I get the next version of pyuserinput through github?
Thank you for your awesome module.
Paul ECOFFET Le 14 juil. 2013 17:23, "Paul Barton" [email protected] a écrit :
So I just took some time to look over X11's implementation of PyMouse and PyMouseEvent and I have a few observations.
The first point I should make is that in X11, mouse wheel events are essentially the same as the other mouse click events. If you so choose, you can can do mymouse.click(x=
, y= , click=4, n= ) to scroll up, and mymouse.click(x= , y= , click=5, n= ) to scroll down. Another thing I noticed is something that appears to be a bug in PyMouseEvent, which is that mouse wheel events are detected appropriately, but PyMouseEvent.handler() returns the wrong button key numbers. I will fix this and do some testing to make sure I don't break a feature from @pepijndevos https://github.com/pepijndevos . This will change some behavior.
Finally, I like the idea of providing separate methods for mouse wheel actions; even though it might be redundant in X11, it is more natural to use and simple to provide a cross-platform interface. Given a few minutes, I can push out a new branch with the fixes and the new methods in X11, if it goes smoothly then I will work on the other platforms.
— Reply to this email directly or view it on GitHubhttps://github.com/SavinaRoja/PyUserInput/issues/10#issuecomment-20937888 .
I probably won't upload a new version to PyPI until I add the scroll methods to each platform.
Could you try out the Issue 10 branch of the code to see if things work for you, I would also appreciate your input on the new PyMouse.scroll() method as a means of interacting with the wheel. Here is a usage example.
from pymouse import PyMouse
m = PyMouse()
m.scroll(*m.position(), up=True, n=5)
I may also consider adjusting PyMouseEvent to detect scrolling events as distinct from the other mouse events.
EDIT: The Issue 10 branch has been merged into the master branch. Scrolling is now part of the main branch and works provisionally on X11; I will be working on the other platforms in the near future.
I have just added Linux support to https://github.com/openleap/PyLeapMouse using your library - thanks! I am now considering migrating PyLeapMouse to use PyUserInput completely. Most cases implemented there appear to be covered here, with scrolling being the notable exception. The following is the existing scrolling code from PyLeapMouse; it should be possible to drop it straight into whatever interface you decide to set up.
OSX
def RelativeMouseScroll(x_movement, y_movement): #Movements should be no larger than +- 10
scrollWheelEvent = CGEventCreateScrollWheelEvent(
None, #No source
kCGScrollEventUnitPixel, #We are using pixel units
2, #Number of wheels(dimensions)
y_movement,
x_movement)
CGEventPost(kCGHIDEventTap, scrollWheelEvent)
(from https://github.com/openleap/PyLeapMouse/blob/master/OSX/Mouse.py)
Windows
def RelativeMouseScroll(x_movement, y_movement): #Movements should be no larger than +- 10
#Windows evidently doesn't really support sideways scrolling.
windll.user32.mouse_event(0x0800,0,0,int(y_movement),0)
(from https://github.com/openleap/PyLeapMouse/blob/master/Windows/Mouse.py - looking at your code, windll.user32 can be replaced with win32api)
The only problem I see (as far as creating a uniform interface goes) is that Windows and OSX appear to support relative mouse scrolling, while Xlib only supports absolute wheel clicking. I got around that in https://github.com/openleap/PyLeapMouse/blob/master/Linux/Mouse.py by clicking up or down based on the sign of y_movement, but there must be a nicer way to unify relative vs absolute scrolling.
Cool; I just back from vacation so I will start taking a closer look at this in the next couple days. Thanks for the contribution and information!
I'm not terribly familiar with relative cursor scrolling, and what distinguishes it from absolute scrolling; could you provide me with a few brief statements of the motivation and uses of it? It should help me become educated on the subject faster.
Unfortunately I am as unfamiliar with relative scrolling as you are. I have asked the PyLeapMouse maintainer to clarify.
EDIT: As per the comments in issue above, this was just a naming thing. The problem I outlined above is better rephrased as the difference between Xlib only allowing a particular number of clicks, and OSX/Windows allowing a specific scroll amount.
Okay, that helps clear up some of my confusion. As for unifying the different platform interfaces, that may require some creativity... I expect that I'll attempt to provide full functionality in each platform, with warnings about actions which cannot be fully cross-platform. I'll also try to ensure informative errors when attempting a platform-incapable action (such as horizontal-scrolling in linux). I'll update when I've done a little tinkering.
I have started a scrolling branch where I have begun a more advanced interface to the scroll wheel. I will enumerate the differences between the different platforms and how I am attempting to address this.
Mac - Supports both vertical and horizontal scrolling. The scroll events have a dynamic range for "scroll-motion". Windows - Supports only vertical scrolling. The scroll events have a dynamic range for "scroll-motion". X11 - Supports only vertical scrolling. The scroll events are quantized for "scroll-motion".
I want to provide the dynamic range of scroll-motion for Mac and Windows, but this cannot be cross-platform for X11. I want to provide the horizontal scrolling for Mac, but this cannot be cross-platform for Windows and X11. What this means is that it will be possible to write code with PyUserInput that takes advantage of the native platform, but will not be cross-compatible; the programmer should be aware of this! Importantly, by introducing the "ticks" interface to scrolling, I hope to make it easy to write cross-platform code for the cases where that is a high priority. In this approach, the programmer has the option of setting an appropriate scroll-movement "delta" by method argument, or by default setting (changed using the set_scroll_delta function).
Here's some example code for how the code stands now. (May not work after refinement in near future)
from pymouse import PyMouse
mouse = PyMouse()
#Works on all platforms
mouse.scroll(vertical=3, ticks=True) # Scrolls up 3 ticks
mouse.scroll(vertical=-5, ticks=True, tick_delta_v=3.2) # Scrolls down 5 ticks; custom scroll distance per-tick of 3.2 units
#Works on Mac and Windows only
mouse.scroll(vertical=3.2) # Scrolls up 3.2 units, I wonder how well these convert anyway...
mouse.scroll(vertical=-4) # Scrolls down 4.0 units, integers coerced to floats
mouse.scroll(vertical=2.0, ticks=True, tick_delta_v=3.2) # Scrolls up 2 ticks, floats coerced to integers, custom scroll distance
#Works only on Mac
mouse.scroll(vertical=2, horizontal=-3) #Scrolls up 2 units, left 3 units
mouse.scroll(vertical=2, horizontal=-3, ticks=True, tick_delta_h=2.0) # Scrolls up 2 ticks, left 3 ticks, custom horizontal scroll distance
In case you didn't see on the other issue, here is some extra info about the PyLeapMouse scrolling code:
For OS X, we're using by-pixel scroll amounts (kCGScrollEventUnitPixel). I think it might actually be wise to switch to point-based scroll amounts, since I don't know how this behaves on Retina devices.
For Windows, I'm not so sure. I think it might be pixel-based. The MSDN docs weren't as clear. I'm pretty sure that, in versions of windows before 7 (or maybe before vista?) the minimum scroll amount was 120, which corresponds to a single wheel click. But I think in >= 7 (or maybe >= vista), amounts less than 120 are supported, which is good for us. From what you describe, it sounds like Xlib is still using very large minimum scroll amounts (represented by a single "scroll wheel click"). I'm not sure how to work with this. I am unfortunately too ignorant about X's internal operation.
(Note these are from the point of view of performing scroll events based on the movement of a hand over the Leap Motion device, hence referring to small scroll amounts.)
I have been doing some tests with X11 and fake_input, and X11 does support horizontal scrolling. Buttons 4/5 click up/down (as in existing code); buttons 6/7 click left/right. Haven't found any ability to control the scroll length however.
Also, you don't need the "Movement should be no larger than 10" comments - that is only necessary for a Leap Motion because of the way the scroll amount is tied to hand gestures. As long as you allow numbers less than 10 to be used, it's fine for me :)
And Windows also supports horizontal scrolling. Here are the MSDN docs for win32api.mouse_event()
http://msdn.microsoft.com/en-us/library/windows/desktop/ms646260%28v=vs.85%29.aspx
This explains how wheel clicks are represented in Windows as well (by a scroll amount of 120).
Okay, turns out that "Movement should be no larger than 10" is necessary for OSX, having now found CGEventCreateScrollWheelEvent in the OSX docs:
https://developer.apple.com/library/mac/documentation/Carbon/Reference/QuartzEventServicesRef/Reference/reference.html#//apple_ref/c/func/CGEventCreateScrollWheelEvent
(And it turns out that Mac is still fancier than Windows and X11 - it supports scrolling in up to THREE directions ^_^)
Fixing up some Linux stuff to enable horizontal scrolling, then will need to take a look at your additional information about scroll delta. Am I also to understand that Mac supports 3d scrolling?
Looks like it. From the CGEventCreateScrollWheelEvent docs (also pasting the bit about movement no larger than 10):
Parameters [...snip...] wheelCount The number of scrolling devices on the mouse, up to a maximum of 3. wheel1 A value that reflects the movement of the primary scrolling device on the mouse. Scrolling movement is generally represented by small signed integer values, typically in a range from -10 to +10. Large values may have unexpected results, depending on the application that processes the event. ... Up to two values that reflect the movements of the other scrolling devices on the mouse, if any.
I have no idea how Mac actually represents the 3rd dimension of scrolling, but the docs at least indicate it is possible.
From a usability perspective I think it's important that whatever interface is decided on has a standard definition of scroll amounts that is appropriately converted to the OS-specific amounts, so that pymouse users have some expectation of how far their code is scrolling.
- Windows supports specifying scroll amounts by the amount of wheel movement, with one wheel click having a movement of 120. I have no idea how this compares to pixels or any other unit. The old and new APIs have the exact same specification for wheel movement, but looking at mouse movement:
mouse_event(the old API) allows specifying absolute coordinates on a normalised coordinate map ([0,0]-[65535,65535]), or relative movement in "mickeys", where a mickey is the amount that a mouse has to move for it to report that it has moved.SendInput(the new API) allows specifying absolute coordinates on a normalised coordinate map, or relative movement in pixels.
- OSX supports specifying scroll amounts by pixels or by lines, with the default scale being about 10 pixels per line. Most applications interpret pixel scrolling as a smooth scrolling event; therefore, ticks should probably be performed as line scrolling (and the specified tick delta could be used to set the scale with
CGEventSourceSetPixelsPerLine). But I am not an OSX user, so I have no idea how OSX scrolling "feels" compared to Windows or X11. - X11 clearly doesn't support smooth scrolling, so is limited to ticks.
Ideally, a tick should by default be equivalent in all OSs, and dynamic scrolling should scroll the same amount in all (supported) OSs.
I may, at least for a preliminary implementation, pin the dynamic scroll delta to pixel measurements. I conceptually changed the interface model so that scrolling by "ticks" is the default behavior; it makes more sense for the cross-compatible mode to be default.
I'm currently wrestling with win32api to properly implement things for windows. I may or may not need to change dependencies...
@pythonian4000 After doing some research, I am becoming quite unsure about whether Windows supports dynamic scroll delta by pixel quantity. I'll still want to implement dynamic scroll delta, but I may need to do some looking into how to normalize the delta between Windows and Mac. I imagine this problem has been tackled many times over, so I might be able to find some wisdom in other code.
Just a quick update about the progress on this issue. I've come across a spare mac in my place of work so I hope to soon be able to test code directly on that platform. I'm totally new to it though, so bear with me. I'm pretty sure that I'll have to implement a standardized scroll "tick" between platforms, but I'll also attempt to expose the native dynamic scroll delta; whether this harmonizes between mac and windows remains to be seen.
Edit: This mac turned out to be very outdated and I am not in a position to update it...
I've updated the repository master branch after merging in some of my scrolling code. I've left the dynamic scrolling delta off for a future update, because I don't have the ability at the moment to really test things properly.