vi modes inhibit input events on mobile

[migalmoreno]

Describe the bug Activating vi-insert-mode or vi-normal-mode on mobile devices that make use of touch input events will cause events such scrolling or clicking on hyperlinks to not work at all, without any kind of feedback. However, disabling the left-click keybinding for vi-mode as defined here, will dismiss this issue and allow for normal behavior.

Precise recipe to reproduce the issue Compile Nyxt from source in Mobian, launch the resulting executable and navigate to a web-mode buffer. Try scrolling via touch input without any hiccups. Activate vi-normal-mode through the prompt buffer and experience the issue in question.

For website-specific issues: Can you reproduce this issue with Epiphany / GNOME Web (https://wiki.gnome.org/Apps/Web)? No.

Information

• OS name+version: Linux mobian 5.14-sdm845 SMP PREEMPT Sun Jul 25 15:38:14 UTC 2021 aarch64 GNU/Linux
• Graphics card and driver: Adreno 630
• Desktop environment / Window manager name+version: mobian-phosh 0.3.1
• How you installed Nyxt (Guix pack, package manager, build from source): Build from source.
• Information from show-system-information:

Nyxt version: 2.1.1-1257-g226606f7
Renderer version: GI-GTK
Operating system kernel: Linux 5.14-sdm845
Lisp implementation: SBCL 2.1.11.debian (Dynamic space size: 3221225472)
Features: (:WEBKIT2 :WEBKIT2-2.34 :WEBKIT2-PASTE-PLAINTEXT :WEBKIT2-TRACKING
 :WEBKIT2-MUTE :WEBKIT2-EMOJI :WEBKIT2-MEDIA :WEBKIT2-SANDBOXING :GTK-3-22
 :GTK-3-20 :GTK-3-18 :GTK-3-16 :GTK-3-14 :GTK-3-12 :GTK-3-10 :GTK-3-8 :GTK-3-6
 :GTK-3-4 :GTK :GDK-3-22 :GDK-3-20 :GDK-3-18 :GDK-3-16 :GDK-3-14 :GDK-3-12
 :GDK-3-10 :GDK-3-8 :GDK-3-6 :GDK-3-4 :CAIRO-1-10 :CAIRO-1-12 :GDK-PIXBUF
 :GLIB-2-30 :GLIB-2-32 :GLIB-2-34 :GLIB-2-36 :GLIB-2-38 :GLIB-2-40 :GLIB-2-42
 :GLIB-2-44 :GLIB-2-46 :GLIB-2-48 :GLIB-2-50 :GLIB-2-52 :GLIB-2-54 :GLIB-2-56
 :GLIB-2-58 :GLIB :NYXT-2 :FSET-EXT-STRINGS :CUSTOM-HASH-TABLE-NATIVE :SWANK
 :PLUMP-UTF-32 :GLOBAL-VARS :DECLARE-TYPES :PARENSCRIPT
 :SBCL+SAFE-STANDARD-READTABLE :NAMED-READTABLES :LPARALLEL :21BIT-CHARS
 :CHUNGA :FLEXI-STREAMS :CLOSER-MOP :CL-PPCRE-UNICODE :CL-UNICODE :CL-PPCRE
 :BORDEAUX-THREADS :LPARALLEL.WITH-CLTL2 :LPARALLEL.WITH-CAS
 :LPARALLEL.WITH-STEALING-SCHEDULER :SPLIT-SEQUENCE CHIPZ-SYSTEM:GRAY-STREAMS
 CFFI-FEATURES:FLAT-NAMESPACE CFFI-FEATURES:UNIX :CFFI CFFI-SYS::FLAT-NAMESPACE
 ALEXANDRIA::SEQUENCE-EMPTYP :FAST-IO-SV :FAST-IO :SBCL-USES-SB-ROTATE-BYTE
 :CL-JSON-CLOS :CL-JSON :THREAD-SUPPORT :ASDF3.3 :ASDF3.2 :ASDF3.1 :ASDF3
 :ASDF2 :ASDF :OS-UNIX :NON-BASE-CHARS-EXIST-P :ASDF-UNICODE :ARM64 :GENCGC
 :64-BIT :ANSI-CL :COMMON-LISP :ELF :IEEE-FLOATING-POINT :LINUX :LITTLE-ENDIAN
 :PACKAGE-LOCAL-NICKNAMES :SB-CORE-COMPRESSION :SB-LDB :SB-PACKAGE-LOCKS
 :SB-THREAD :SB-UNICODE :SBCL :UNIX)

ASDF version: 3.3.1
ASDF registries: (NYXT-SOURCE-REGISTRY ENVIRONMENT-SOURCE-REGISTRY)
Critical dependencies: (/home/mobian/common-lisp/nyxt/_build/cl-cffi-gtk/gtk/cl-cffi-gtk.asd
 /home/mobian/common-lisp/nyxt/_build/cl-gobject-introspection/cl-gobject-introspection.asd
 /home/mobian/common-lisp/nyxt/_build/cl-webkit/webkit2/cl-webkit2.asd)

Guix version: 616dc4397f02cde9cf3bde22b383f44582163db2

If you can't run show-system-information, try nyxt --system-information from a shell. It this still does not work, please provide the following:

• Nyxt version (from the nyxt-version command or nyxt --version from a shell):
• Lisp implementation/version (if built from source):
• Kernel name+version:
• WebKitGTK+ | QtWebEngine version:

Output when started from a shell

[Kabouik]

Awesome, thanks @efimerspan and @aartaka for figuring out what was the issue. I don't know why I did not even think of trying Emacs mode instead of Vi mode.

I can confirm editing source/mode/vi.lisp as described fixes the issue on Debian running in an LXC container on SailfishOS. I have not tried in Droidian yet but there's probably no need.

~~There are still some issues with somme web pages, for instance https://clients.boursorama.com/connexion/ after you type a customer number (try anything with 8 digits), the big pink button below won't work on click, I need to select it with a tap, and then enter vi-insert-mode and press return to go through.~~ Works now, I guess it was just too slow and unresponsive due to the background animation performing really bad in Webkit.

[aartaka]

The click-to-insert is exactly what the removed piece of code was responsible for :)

[aartaka]

To add to this: it seems that sometimes we may want keyboard events to both activate a command and pass through to the renderer. The VI click/tap is exactly this kind of event. It's a requirement for libraries/keymap and renderer/gtk then.

[jmercouris]

In fact I have faced /exactly/ the same issue when trying to implement select-frame. I wanted both to capture the fact that a click is made to update the prompt buffer source, AND to pass the click through to the renderer. This is indeed a generalized problem!

[Ambrevar]

Does this bug exist on non-mobile platforms?

[aartaka]

Does this bug exist on non-mobile platforms?

Yes, vi-normal-mode breaks touch events on my touchscreen laptop.

[Ambrevar]

OK, this probably makes sense, let's look at the definition:

(define-ffi-method ffi-generate-input-event ((window gtk-window) event)
  (when event
    ;; The "send_event" field is used to mark the event as an "unconsumed"
    ;; keypress.  The distinction allows us to avoid looping indefinitely.
    (etypecase event
      (gdk:gdk-event-button
       (setf (gdk:gdk-event-button-send-event event) t))
      (gdk:gdk-event-key
       (setf (gdk:gdk-event-key-send-event event) t))
      (gdk:gdk-event-scroll
       (setf (gdk:gdk-event-scroll-send-event event) t)))
    (gtk:gtk-main-do-event event)))

There is no provision for touch events. Maybe the fix is simple, but unfortunately I can't test myself. @aartaka Mind having a look?

[aartaka]

@aartaka Mind having a look?

I don't mind! However, I remember spending an hour trying to resolve it a month ago, with no success. Let's see how it goes this time :)

[Ambrevar]

Good luck!

[aartaka]

@efimerspan, @Kabouik, can you try reproducing with mouse? What are the websites you notice problems on?

[CorruptedVor]

Clicking on hyperlinks and scrolling work for me in vi-insert, however, scrolling does not work in vi-normal and hyperlinks are merely selected(?) - It'll just show a little arrow -> and the hyperlink text in the status at the bottom. Tested using commit 82b73fbda05eb0a7d48d045a60b3ae6e63a73601.

[aartaka]

Note that fd3df52a1c26608701c7f0098e2f9c86e2b96486 finally moves the click-to-insert-mode logic to the renderer side, which frees the space for clicks, scrolls, and taps. Not sure whether that qualifies as a fix to the problem and I should close that. @efimerspan, @Kabouik, @CorruptedVor, any opinion?

[CorruptedVor]

Note that fd3df52 finally moves the click-to-insert-mode logic to the renderer side, which frees the space for clicks, scrolls, and taps. Not sure whether that qualifies as a fix to the problem and I should close that. @efimerspan, @Kabouik, @CorruptedVor, any opinion?

Works for me, the issue I described is gone.

[aartaka]

Cool! Then I'm closing this.

[Kabouik]

Nice, thank you.

I'm sorry I couldn't test this because the system into which I use Nyxt with touch is not accessible currently (I'm refactoring the whole app that allows launching X from LXC containers on mobile, which of course broke things I haven't managed to fix yet), but I'll post here if I have any bad news later.

[aartaka]

@Kabouik, no problem :)