Implement input_method_v2 popups

[tadeokondrak]

Depends on ~~#4932~~, ~~https://github.com/swaywm/wlroots/pull/2550~~ and ~~https://gitlab.freedesktop.org/wlroots/wlroots/-/merge_requests/3416~~. ~~https://github.com/swaywm/wlroots/pull/2697~~, ~~#5980~~, ~~#6004~~, and ~~#6005~~ are related but not required.

[12101111]

sway crash on exit

  * frame #0: 0x00007fcc2418f6a7 libwayland-server.so.0`wl_list_remove(elm=0x00007fcc1bd2dd98) at wayland-util.c:55:18
    frame #1: 0x00007fcc2413a06a libwlroots.so.7`xdg_popup_grab_handle_seat_destroy(listener=<unavailable>, data=<unavailable>) at wlr_xdg_popup.c:158:2
    frame #2: 0x00007fcc2416634c libwlroots.so.7`wlr_signal_emit_safe(signal=<unavailable>, data=0x00007fcc1c04e050) at signal.c:29:3
    frame #3: 0x00007fcc24134e88 libwlroots.so.7`wlr_seat_destroy(seat=0x00007fcc1c04e050) at wlr_seat.c:170:2
    frame #4: 0x00007fcc2418975f libwayland-server.so.0`wl_display_destroy [inlined] wl_priv_signal_final_emit(signal=0x00007fcc245fce88, data=0x00007fcc245fce20) at wayland-server.c:2157:3
    frame #5: 0x00007fcc24189728 libwayland-server.so.0`wl_display_destroy(display=0x00007fcc245fce20) at wayland-server.c:1134
    frame #6: 0x00005589ae2246a3 sway`server_fini(server=0x00005589ae2825d0) at server.c:203:2
    frame #7: 0x00005589ae223dee sway`main(argc=1, argv=0x00007ffd55a5e338) at main.c:436:2
    frame #8: 0x00007fcc247c0129 ld-musl-x86_64.so.1`libc_start_main_stage2(main=(sway`main at main.c:245), argc=<unavailable>, argv=0x00007ffd55a5e338) at __libc_start_main.c:94:7
    frame #9: 0x00005589ae216eb6 sway`_start + 22

another crash

* thread #1, name = 'sway', stop reason = signal SIGSEGV
  * frame #0: 0x000055bfceadf096 sway`desktop_damage_view at desktop.c:0:2
    frame #1: 0x000055bfceadf05e sway`desktop_damage_view(view=0x00007f1b143dd2d0) at desktop.c:38
    frame #2: 0x000055bfceafd400 sway`input_popup_update [inlined] input_popup_damage(popup=<unavailable>) at text_input.c:36:2
    frame #3: 0x000055bfceafd3e8 sway`input_popup_update(popup=0x00007f1b13c4fac0) at text_input.c:40
    frame #4: 0x00007f1b1fec5f8c libwlroots.so.7`wlr_signal_emit_safe(signal=<unavailable>, data=0x00007f1b1355a790) at signal.c:29:3
    frame #5: 0x000055bfceafcb29 sway`handle_text_input_enable(listener=<unavailable>, data=<unavailable>) at text_input.c:260:2
    frame #6: 0x00007f1b1fec5f8c libwlroots.so.7`wlr_signal_emit_safe(signal=<unavailable>, data=0x00007f1b1439db10) at signal.c:29:3
    frame #7: 0x00007f1b1f9016b5 libffi.so.7`___lldb_unnamed_symbol3$$libffi.so.7 + 85
    frame #8: 0x00007f1b1f905c6e libffi.so.7`___lldb_unnamed_symbol19$$libffi.so.7 + 750
    frame #9: 0x00007f1b1feee664 libwayland-server.so.0`wl_closure_invoke(closure=0x00007f1b144155c0, flags=<unavailable>, target=<unavailable>, opcode=7, data=<unavailable>) at connection.c:1018:2
    frame #10: 0x00007f1b1fee8958 libwayland-server.so.0`wl_client_connection_data(fd=<unavailable>, mask=<unavailable>, data=<unavailable>) at wayland-server.c:432:4
    frame #11: 0x00007f1b1feecd3c libwayland-server.so.0`wl_event_loop_dispatch(loop=<unavailable>, timeout=<unavailable>) at event-loop.c:1027:4
    frame #12: 0x00007f1b1fee9bad libwayland-server.so.0`wl_display_run(display=0x00007f1b20324c20) at wayland-server.c:1351:3
    frame #13: 0x000055bfceaddcf1 sway`main [inlined] server_run(server=<unavailable>) at server.c:247:2
    frame #14: 0x000055bfceaddcbf sway`main(argc=1, argv=0x00007ffe7aa881c8) at main.c:431
    frame #15: 0x00007f1b20520129 ld-musl-x86_64.so.1`libc_start_main_stage2(main=(sway`main at main.c:245), argc=<unavailable>, argv=0x00007ffe7aa881c8) at __libc_start_main.c:94:7
    frame #16: 0x000055bfcead0e16 sway`_start + 22

[emersion]

#4932 has been merged now, so this needs a rebase.

[emersion]

What client can we test this PR with?

[12101111]

What client can we test this PR with?

You can test this with fcitx5 + alacritty

Use this command to see which frontend is using by client

dbus-send --session --type=method_call --print-reply --dest=org.fcitx.Fcitx5 /controller org.fcitx.Fcitx.Controller1.DebugInfo

You can see progeam:alacritty frontend:waylandim if this PR works

[Shugyousha]

I have done some basic testing of this PR with https://github.com/tadeokondrak/anthywl.

[emersion]

This PR renders the IM popup right above the surface with the input field, below everything else. This is an issue if the surface with the input field has subsurfaces/xdg-popups above it: the IM popup will be rendered below these.

I think we should render the IM popup above everything else instead, e.g. here: https://github.com/swaywm/sway/blob/6059c744f8b24ef3321eb7896991112366896e26/sway/desktop/render.c#L1127

[fosskers]

Alright, I have a custom patched version of wlroots-git installed (based on https://github.com/swaywm/wlroots/pull/2550), and a build of sway-git based on master. So far everything is running. The result of the command given above by @12101111 is:

method return time=1618595438.050418 sender=:1.5 -> destination=:1.33 serial=2513 reply_serial=2
   string "Group [wayland:] has 6 InputContext(s)
  IC [561d5ca5dac845418f8b84dd0e3f054c] program:firefox frontend:dbus cap:e001000012 focus:0
  IC [ca565dfc223e47249f57f2df37d7ec6e] program:firefox frontend:dbus cap:e001000012 focus:0
  IC [ce9472215e87439f9c442dcc2a1915bf] program:firefox frontend:dbus cap:e001000012 focus:0
  IC [5b374d3324394819a73b486a9ec5b72e] program:firefox frontend:dbus cap:e001000012 focus:0
  IC [d0a8c154dd4e414bb90eca715ddf321e] program:firefox frontend:dbus cap:e001000052 focus:0
  IC [c99385fcb290441281077b627eac3f50] program: frontend:wayland_v2 cap:0 focus:1
Group [x11::0] has 0 InputContext(s)
Input Context without group
"

Now I'm going to rebuild sway-git with this PR and see what happens.

[fosskers]

I'm still getting the same results from the dbus command. Is some special configuration required?

EDIT: Actually, I am able to type Japanese into alacritty if I click the fcitx icon in my Waybar and set it to Japanese that way, which is an improvement over the current official sway/wlroots packages. Unfortunately there seem to be other bugs, namely that certain keys seem to be repeated/ignored. I had a whole bunch of terminals open spontaneously at one point. Numlock also has no effect in the terminal (i.e. I can't type numbers with it).

[12101111]

Yes, the code still have some bugs ( or fcitx5 have some bugs )

[tadeokondrak]

fcitx5 has keyboard input issues when using input-method-v2.

As for this PR, since there are not many clients using it, I think it's worth implementing a future version of the protocol with these issues fixed instead: https://github.com/swaywm/wlroots/pull/2550#pullrequestreview-616270027

[glyh]

I hope this PR will be merged soon, since IMEs are basically not usable on wayland.

[tadeokondrak]

I've resolved the review comment; this should be ready for review.

[wildwestrom]

I'm so hyped for this! I couldn't wait so I built it from source on my machine.

[tinywrkb]

Does this still work correctly with Sway 1.7?
I don't have popups working with Fcitx5 anymore when applying this on top of 1.7.
Fcitx5 is built from the latest commit in master.

This sort of seems to work with anthywl, but I don't know how to type in Japanese, so I just hit random keys, and sometimes I get a popup, but I don't get a visual feedback for every key press, and I have no idea if this is an expected Japanese typing behavior.

[wildwestrom]

@tinywrkb

I'm on Arch Linux, using the sway-hidpi-git package and fcitx5-git. Japanese input through mozc works through Xwayland and native Wayland windows.

Not sure if those packages track changes from this PR or not.

[tinywrkb]

@wildwestrom you don't pull in the PR, so

Japanese input through mozc works through Xwayland and native Wayland windows.

means XIM, and the widget toolkit's IM modules. But when it's a native Wayland client without an IM module, then you won't have popups.
Previously, when I used this PR, I had popups working with Alacritty and Kitty, both running as native Wayland clients, and using Fcitx5.
Is this just Fcitx5's fault? I have no idea.

[JingMatrix]

I apply this PR with the current HEAD. The api of wlr_output_layout_get_box has changed, so you should change your code a bit in sway/sway/input/text_input.c. After this, I compiled your code, and tested it with alacritty and kitty, both gave me the same result: the popup showed up at the very beginning, and then never showed up again.

[dragove]

I've tested it with AUR package sway-im with fcitx5 as input method in kitty. When the character meets the end of a line, the popup will be hidden. I don't know if it is a issue with kitty. Alacritty won't meet this issue since it won't put characters into the line before I select my needed word in IM. Edit: This problem only shows up when I set default_border none 0 in sway configuration file.

[JingMatrix]

Now I test it ( after fixing the output_box api problem) with the HEAD of fcitx5 and HEAD of sway, everything works!

[Legogris]

@emersion Looks like this is ready for another review :)

[evanslify]

@tadeokondrak @emersion This PR seems to have some amendments pending. Can I take over by starting another PR?

[flibitijibibo]

Someone has been testing this with SDL and it seems there's an issue with fcitx, but only when integer scaling is used...?

https://github.com/libsdl-org/SDL/issues/5892

We have some ideas on our end, but figured we'd include the issue here in case it's something obvious on the compositor side of things.

[tadeokondrak]

Sorry about this PR... I've left it open because I thought I could get to it eventually - but it looks like someone else is continuing the effort at https://github.com/swaywm/sway/pull/7226, which is great.

[Decodetalkers]

Sorry about this PR... I've left it open because I thought I could get to it eventually - but it looks like someone else is continuing the effort at https://github.com/swaywm/sway/pull/7226, which is great.

Thanks that you have return back..Your help is still needed..Do you still have time now?

[Decodetalkers]

Sorry about this PR... I've left it open because I thought I could get to it eventually - but it looks like someone else is continuing the effort at #7226, which is great.

This mixes up layout-local coords and output-local coords. I think input_popup_surface_at should take layout-local coords instead.

I do not know how to make change about this request, can you help me?