Wayland support

[vith]

Wayland is now the default display server on some popular distros such as Fedora 25.

It would be great if ActivityWatch worked on Wayland!

On v0.7.0b1 I just get this log line repeatedly:

2017-08-02 09:35:59,457 [WARNING]: Failed to find active window, id found was 0x0 (aw_watcher_window.lib:19)

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[ErikBjare]

I've been anticipating this... See this issue: https://github.com/ActivityWatch/aw-watcher-window/issues/18

Unfortunately, I'm not sure how to solve this. It seems like we can't have Wayland-support since Wayland itself doesn't have any notion of active window.

Are you using Gnome? I'd be interested in getting this working on Gnome or some other large desktop environment with Wayland support to start with and then see how we should go from there.

Edit: It seems Gnome is deprecating their get_active_window API which only ever worked on X11. Will investigate further.

Edit 2: Done quite a few searches but have come up emptyhanded. Considering asking a question on Stack Exchange and/or sending a mail to some Gnome mailinglist.

[vith]

Ah, that does complicate things.

Yep, I'm using Gnome on Arch Linux actually. ActivityWatch is the only thing keeping me using the Gnome on Xorg session.

[ErikBjare]

Created a question on Stack Overflow, will try to find someone on IRC to help.

Edit: Also asked on the Gnome IRC, seems a bit quiet right now.

Edit: Has now been ~20h since I asked, no one replied to me in that channel. Might want to try a mailinglist further down the line.

[ErikBjare]

Alright, just asked in the #wayland channel on Freenode. Got a response at least, but the situation seems pretty bad.

15:20:44  ErikBjare    Hello everybody. I'm working on a piece of self-tracking software called ActivityWatch (https://github.com/ActivityWatch/activitywatch). I know this isn't exactly the right place to ask, but I was wondering if anyone knew anything about getting the active window in any Wayland-using DE.
15:20:57  ErikBjare    Created a question on SO: https://stackoverflow.com/questions/45465016/how-do-i-get-the-active-window-on-gnome-wayland
15:21:25  ErikBjare    Here's the issue in my repo for it: https://github.com/ActivityWatch/activitywatch/issues/92
15:22:54  ErikBjare    There are a bunch of other applications that depend on it (RescueTime, selfspy, arbtt, ulogme, etc.) so they'd need it as well
15:24:23  blocage      ErikBjare, in the core protocol you cannot know which windnow has the keyboard or cursor focus
15:24:39  blocage      ErikBjare, in the wayland core protocol *
15:25:10  blocage      ErikBjare, you can just know if your window has the focus or not, it a design choise
15:25:23  blocage      avoid client spying each other
15:25:25  ErikBjare    blocage: I'm aware, that's my reason for concern. I'm not saying it should be included or anything, but as it looks now every DE would need to implement it themselves if these kind of applications are to be supported
15:25:46  ErikBjare    So wondering if anyone knew the teams working with Wayland on Gnome for example
15:26:11  ErikBjare    But thanks for confirming
15:26:29  blocage      ErikBjare, DE should create a custom extension, or use D-bus or other IPC
15:27:31  blocage      ErikBjare, I guess some compositor are around here, but I do not know myself if there is such extension already
15:27:44  blocage      compositor developers *
15:28:36  ErikBjare    I don't think there is (I've done quite a bit of searching), so I guess I need to catch the attention of some DE developers
15:29:16  ErikBjare    Thanks a lot though
15:29:42  ErikBjare    blocage: Would you mind if I shared logs of our conversation in the issue?                                     
15:30:05  blocage      just use it :) it's public                                                                                               
15:30:19  ErikBjare    ty :)                                                                                                            

I guess our only option is to contact some DE developers and petition them to create a way to get the info.

[osule]

You may be able to get the active window from the root window.

• https://github.com/wayland-project/weston/blob/master/xwayland/window-manager.c#L2244
• https://developer.gnome.org/gdk3/stable/GdkScreen.html#gdk-screen-get-active-window

[ErikBjare]

@osule Unfortunately that won't work. That Gnome API is X11 only (that's why its deprecated) and that thing you found in weston is just xwayland.

Wayland has no concept of active window, and Gnome has no internal concept of it that it exposes. The reasons for this seems to be privacy as blocage mentioned in the IRC convo I pasted.

[ErikBjare]

Filed an issue on the bug tracker for Gnome's mutter.

[Awerick]

Hi @ErikBjare, I stumbled upon this issue while searching for a related problem.

Excuse me, in case this is completely off track but I just came across the "Wayland Security Module" ("WSM", see below for links) and thought, this might be interesting / relevant for this issue as well: Its a proposal for a standardized, secure way to let programs perform "sensitive" tasks under Wayland (if the Desktop Environments/Wayland compositors would support the WSM). Maybe long-term this could also be a solution for this issue of "getting the active window under Wayland".

More info on Wayland Security Module: X.Org Presentation 2014 ; official GitHub repo ; Wikipedia: Wayland#WSM

[ErikBjare]

@Awerick Thanks a lot for reaching out!

WSM looks like a great initiative, and I had no idea it existed. It's probably the closest thing to a solution I've seen so far. Let's hope there is interest in it from the Wayland community, I might try prodding them in the future and see if they have considered our usecase :slightly_smiling_face:

[Awerick]

Glad it was helpful! I am actually not sure whether WSM is something that Wayland itself has to support or whether it is something "additional" to the Wayland protocol that the DEs would have to provide (e.g. weston as Waylands reference compositor implementation, Gnome's mutter etc.) …

[da2x]

So Mutter/GNOME support may be out, but have have you investigated KWin/Wayland (Plasma) support?

Update: The org.gnome.Mutter.ScreenCast API in libmutter + OCR could solve this for GNOME. Obviously at a terrible performance cost.

[johan-bjareholt]

@da2x I currently don't have a desktop with KWin/Wayland installed, but would love to try out in case it has some API that exposes window information (DBus?).

In the long run however we don't have enough manpower to write one window watcher for each Wayland compositor, so we will have to push for some standardized wayland protocol for exposing the information about the currently focused window.

[jancborchardt]

I am using GNOME (on Ubuntu) and while I’m mainly a designer, I could help if something needs to be tried – if I can get some pointers. :)

In the long run however we don't have enough manpower to write one window watcher for each Wayland compositor, so we will have to push for some standardized wayland protocol for exposing the information about the currently focused window.

Maybe it’s best to focus on GNOME first, cause it seems like a standardized way is more difficult. And GNOME is used by Ubuntu and thus makes it the most used desktop environment – which will solve it for a lot of people already such as @vith and myself.

[jancborchardt]

Another possible enhancement, while it might result in false positives, could be to adjust the output in the interface. So when ActivityWatch is running on Arch/Ubuntu/etc., we could kind-of assume that "Unknown" is a system applicaton and simply write "System app".

As this might be wrong, it could be further specified like "Unknown (likely system app)" or something like that. But it’s already better than a plain "Unknown".

[johan-bjareholt]

@jancborchardt There's a separate issue with more info in aw-watcher-window where we have discussed this a bit. TL;DR: there is a WIP branch of gnome-shell which has implemented a DBus API which exposes this and they are considering making it into a Freedesktop standard. Progress is slow though.

https://github.com/ActivityWatch/aw-watcher-window/issues/18#issuecomment-445171200

Another possible enhancement, while it might result in false positives, could be to adjust the output in the interface. So when ActivityWatch is running on Arch/Ubuntu/etc., we could kind-of assume that "Unknown" is a system applicaton and simply write "System app". As this might be wrong, it could be further specified like "Unknown (likely system app)" or something like that. But it’s already better than a plain "Unknown".

I don't really agree with this. If the appname is not available and "Unknown" is used it only means that the application has not used APIs properly to expose its name to the display manager. There's really nothing that indicates what kind of app it is. Whether its more likely to be one type of app or another will likely vary a lot depending on the OS and the quality of the apps which the user uses so an assumption is very likely to be wrong.

[hughwilliams94]

Just to mention that swaywm (a tiling Wayland compositor), has a command which returns the focused window (swaymsg -t get_tree). The result of the command is a json formatted list of the currently open windows with a key boolean key for focused (and lots of other info)

[johan-bjareholt]

@hughwilliams94 Thanks for the heads up, that's pretty cool! Might be fun to try just for prototyping, but as it's exclusive to sway it's probably never going to be used. We really need a proper Wayland API for this.

[johan-bjareholt]

Asked questions about this to wlroots devs on reddit and got this answer: https://www.reddit.com/r/linux/comments/as1dd0/we_are_the_sway_wlroots_developers_ask_us_anything/egskqq9

Apparently this interface can give us support for all wlroots based window managers which sounds like a decent solution. https://github.com/swaywm/wlr-protocols/blob/master/unstable/wlr-foreign-toplevel-management-unstable-v1.xml

[hughwilliams94]

Great to hear, the devs on sway and wlroots are very responsive and have been working well with the KDE gang. Hopefully this can help you to make some progress.

[johan-bjareholt]

@hughwilliams94 Yeah, they are very good at communicating with their users (in contrast to GNOME/KDE devs).

In the end we are likely not going to be using this solution, but it seems like a reliable base to start prototyping with.

[johan-bjareholt]

Just want to mention, if someone is using sway I have written a prototype watcher for that which works with a few flaws which you can read about in the README

https://github.com/activitywatch/aw-watcher-sway

[johan-bjareholt]

First of all the Gnome introspection code was merged: https://gitlab.gnome.org/GNOME/gnome-shell/merge_requests/326/diffs So now it should be possible to write window detection to Gnome Shell under wayland. Still no afk detection though as far as I know.

Another good thing is that phosh (the Librem 5 compositor) has now implemented the wlr-foreign-toplevel-management protocol https://source.puri.sm/Librem5/phosh/commit/532cfaf085cd440c3f849e92da8c8d65681c2a9c

Another good thing is that sway got a PR to get wlr-foreign-toplevel-management implemented as well which means that we could get one watcher which could support both sway and phosh. It's not merged yet though, but it looks promising! https://github.com/swaywm/sway/pull/4476

[glyph]

It’s been a while, but as I recall there are other dbus events one can watch for idle time?

In any event thanks for the updates :)

[johan-bjareholt]

@glyph Correct, currently has that working in this prototype wayland watcher which works surprisingly well.

https://github.com/activitywatch/aw-watcher-window-wayland

[rbreaves]

Example of getting the active window from Gnome under wayland via bash.

https://gist.github.com/rbreaves/257c3edfa301786e66e964d7ac036269

[johan-bjareholt]

@rbreaves Nice, since gnome also implements the idle.xml Wayland protocol it should be possible to write a working watcher for gnome now. That solution is kind of hacky though (race conditions possible, two dbus calls for each poll is kind of slow).

Still a shame that the Wayland ecosystem is so fragmented regarding APIs and protocol extensions.

[rbreaves]

I suspect there could be a way to rewrite those 2 dbus calls into a single call, assuming all of the object data is in the initial call and just needs to be traversed and manipulated to return what I want via an inline function.

[FlorianLudwig]

I used @rbreaves idea and made it into a single call to dbus call and cleaned it up a bit:

        var window_list = global.get_window_actors();
        var active_window_actor = window_list.find(window => window.meta_window.has_focus());
        var active_window = active_window_actor.get_meta_window()
        var vm_class = active_window.get_wm_class();
        var title = active_window.get_title()
        var result = {"title": title, "appname": vm_class};
        result

Probably it would be best to turn into a gnome-shell extension.

But for now I integrated it into the existing watcher: https://github.com/ActivityWatch/aw-watcher-window/pull/46

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[glyph]

Is this really stale or is it just functionally complete per the other comments above about GNOME and KDE?