Nyxt freezes when typing very fast in prompt buffer

[Nselm]

Describe the bug Nyxts's GUI freezes after typing fast in any prompt buffer with selection filters.

Precise recipe to reproduce the issue When typing fast in the prompt buffer, nyxt just freezes. A video I had running in the background continued to play, but

1. I could not do anything
2. When I switched first to another window, then back, it did not display nyxt anymore, but the previous window

Possibly relevant:

• I have a history of ~100 tabs
• I have ~2000 bookmarks
• I have greatly increased the key repeat rate and reduced the delay (not sure how much, but its probably around twice as fast a normal)
• My laptop is extremely weak: It has ~2 G RAM and the rest of the specs is similar

Information

• OS name+version: Arch linux

• Graphics card and driver: Stoney [Radeon R2/R3/R4/R5 Graphics], AMD

• Desktop environment / Window manager name+version: i3-gaps 4.20.1-2

• How you installed Nyxt (Guix pack, package manager, build from source): Nyxt package in aur

• Information from nyxt --system-information:

Nyxt version: 2.2.4
Renderer version: GI-GTK
Operating system kernel: Linux 5.16.12-arch1-1
Lisp implementation: SBCL 2.2.1 (Dynamic space size: 3221225472)
Features: (:WEBKIT2 :WEBKIT2-2.34 :WEBKIT2-PASTE-PLAINTEXT :WEBKIT2-TRACKING
 :WEBKIT2-MUTE :WEBKIT2-EMOJI :WEBKIT2-MEDIA :WEBKIT2-SANDBOXING :GTK-3-22
 :GTK-3-20 :GTK-3-18 :GTK-3-16 :GTK-3-14 :GTK-3-12 :GTK-3-10 :GTK-3-8 :GTK-3-6
 :GTK-3-4 :GTK :GDK-3-22 :GDK-3-20 :GDK-3-18 :GDK-3-16 :GDK-3-14 :GDK-3-12
 :GDK-3-10 :GDK-3-8 :GDK-3-6 :GDK-3-4 :CAIRO-1-10 :CAIRO-1-12 :GDK-PIXBUF
 :GLIB-2-30 :GLIB-2-32 :GLIB-2-34 :GLIB-2-36 :GLIB-2-38 :GLIB-2-40 :GLIB-2-42
 :GLIB-2-44 :GLIB-2-46 :GLIB-2-48 :GLIB-2-50 :GLIB-2-52 :GLIB-2-54 :GLIB-2-56
 :GLIB-2-58 :GLIB :NYXT-2 :FSET-EXT-STRINGS :CUSTOM-HASH-TABLE-NATIVE :SWANK
 :PLUMP-UTF-32 :GLOBAL-VARS :DECLARE-TYPES :PARENSCRIPT
 :SBCL+SAFE-STANDARD-READTABLE :NAMED-READTABLES :LPARALLEL :21BIT-CHARS
 :CHUNGA :CLOSER-MOP :CL-PPCRE-UNICODE :FLEXI-STREAMS :CL-UNICODE :CL-PPCRE
 :CL-JSON-DOUBLE-FLOAT-IS-SUBSUMED :CL-JSON-SINGLE-FLOAT-IS-SUBSUMED
 :BORDEAUX-THREADS :LPARALLEL.WITH-CLTL2 :LPARALLEL.WITH-CAS
 :LPARALLEL.WITH-STEALING-SCHEDULER :SPLIT-SEQUENCE CHIPZ-SYSTEM:GRAY-STREAMS
 CFFI-FEATURES:FLAT-NAMESPACE CFFI-FEATURES:X86-64 CFFI-FEATURES:UNIX :CFFI
 CFFI-SYS::FLAT-NAMESPACE ALEXANDRIA::SEQUENCE-EMPTYP :FAST-IO-SV :FAST-IO
 :SBCL-USES-SB-ROTATE-BYTE :CL-JSON-CLOS :CL-JSON :THREAD-SUPPORT :ASDF3.3
 :ASDF3.2 :ASDF3.1 :ASDF3 :ASDF2 :ASDF :OS-UNIX :NON-BASE-CHARS-EXIST-P
 :ASDF-UNICODE :X86-64 :GENCGC :64-BIT :ANSI-CL :COMMON-LISP :ELF
 :IEEE-FLOATING-POINT :LINUX :LITTLE-ENDIAN :PACKAGE-LOCAL-NICKNAMES
 :SB-CORE-COMPRESSION :SB-LDB :SB-PACKAGE-LOCKS :SB-THREAD :SB-UNICODE :SBCL
 :UNIX)

ASDF version: 3.3.1
ASDF registries: (NYXT-SOURCE-REGISTRY ENVIRONMENT-SOURCE-REGISTRY)
Critical dependencies: (/home/nselm/nyxt/src/nyxt/_build/cl-cffi-gtk/gtk/cl-cffi-gtk.asd
 /home/nselm/nyxt/src/nyxt/_build/cl-gobject-introspection/cl-gobject-introspection.asd
 /home/nselm/nyxt/src/nyxt/_build/cl-webkit/webkit2/cl-webkit2.asd)

[aartaka]

What do you mean by selection filters?

[Nselm]

That it has a lot of possible suggestions that it is trying to filter according to my input. (I mean the fuzzy matching the prompt buffer does against possible suggestions). Things to note:

1. I have tried deleting all my bookmarks, but the issue still occurs.
2. I have tried this on a "real" PC with a little bit of RAM and running a non-modified version of linux mint, and everything works (so far) fine. As a result I think - if nobody else has this problem - that it is probably a fairly niche issue with my setup or my laptop - though I probably wouldnt know if it were not.

[Ambrevar]

@Nselm Can you try reproducding this on master? It may have been fixed already.

[jmercouris]

Indeed it is very possible it has been fixed on master. There was some work done recently for a "headless" implementation which fixes some very fast typing/concurrency issues that we potentially did not catch.

[Nselm]

I have tried out master and it seems to work. However:

1. When typing very fast (esp. deleting, since in this case I just hold down the key) there are still noticeable lags
2. When holding down any key for a longer time, it still hangs. When I use nyxt normally, those problems do not occur, since I wont press the same key for longer than a few seconds, so for any practical purposes the problem seems to be fixed. For reference, the values xset -q gives me: auto repeat delay: 240 repeat rate: 50

[Ambrevar]

We've got an important bug here: I can indeed reproduce the freeze when I keep a key pressed or if I type very fast.

For future reference, to reproduce @Nselm settings, type

xset r rate 240 50

Then keep a key pressed in the set-url prompter.

I could not reproduce in execute-command though, so it seems that the race condition occurs on CPU-intensive filters.

[trev-dev]

Just my totally unhelpful 2 cents here but I believe this bug happens frequently enough for me that I do not have enough confidence in Nyxt to really use it. I can play with it for a while but the expectation is for it to ultimately crash. I can type slow, I can type fast, I can hold buttons down or I could not and it will still crash at the command prompt. This makes Nyxt an unproductive choice for me.

If there is a reasonable temporary work-around to be had I would implement it.

[Ambrevar]

I've been working on it, I'll release a fix before 3.0, don't worry :)

[aartaka]

So, is it fixed? Right now prompt buffer emits "Calculating..." message on new input. Is that the fix in action?

[Ambrevar]

No no no, nothing is fixed yet, it will come with the concurrency overhaul.

[timthelion]

I presume this freeze is the same issue? I see an error at the bottom of the screen as well as the obvious total freeze (see the overdraw from my launcher...).

[jmercouris]

Hello @timthelion you are using a rather old version of Nyxt, can you please try on latest 3.0.0 :-)?

[Ambrevar]

Note that the original post issue is not fixed on 3.0.0.

[odanoburu]

Can confirm I still get freezes fairly often. My machine is otherwise fast, RAM is far from exhausted, and I have been trying out Nyxt (3rd attempt) for just a day, so not a long history nor long list of bookmarks etc.

[aadcg]

@odanoburu do you have a detailed recipe to see it crashing? I still can't reproduce, even when following the guide above.

Also, how did you install Nyxt?

[odanoburu]

I installed using flatpak, and the guide above does cause the issue for me (I wasn't doing anything much different, just typing fast instead of holding down a key). But note that it does not crash, it just hangs for some time.

I'm compiling from source now to see if I can still reproduce it.

[aadcg]

Interesting! I can reproduce it on the Flatpak, but not when I run Nyxt installed via Guix. That gives us a new piece of information. I'll think about it.

The SBCL version differs (2.3.7 vs 2.3.10), but I doubt it plays a role. WebKitGTK differs too (2.40.5 vs 2.42.1). I've sent a patch to Guix that updates WebKitGTK, so soon I'll be able to test it.

[odanoburu]

Interestingly, I can reproduce the problem when compiling from source, although I didn't use Guix, just the makefile.

I get a warning on the minibuffer (and on the terminal output) that I don't get on the flatpak, but of course those are different versions:

<WARN> [15:54:35] Warning: Error while completing default search: The alien function &quot;SSL_get1_peer_certificate&quot; is undefined.

I'm using SBCL 2.3.6, webkitgtk 2.42.2.

[aadcg]

Interesting. Which command did you invoke to see this error message? set-url?

[odanoburu]

Indeed. My openssl version is 3.1.1, so it should have this function. Not sure of the version used in the flatpak though.

[odanoburu]

I think the hanging issue is related to garbage collection: increasing the the amount of memory that will be allocated before the next garbage collection is initiated (by setting sb-ext:bytes-consed-between-gcs) seems to delay the hanging — although of course when it does come it hangs even longer because there is more garbage to collect. I also tried decreasing this value so that the GC would run more often (but taking less time), but I not could find a value that would stop the hanging. SBCL's GC is not low-latency, after all. Although there are efforts towards a better GC:

https://applied-langua.ge/~hayley/swcl-gc.pdf (parallel GC in SBCL >2.3.8) https://github.com/sbcl/sbcl/blob/master/doc/internals-notes/arena-allocation.txt (arena allocation reduces the need for garbage collection) https://groups.google.com/g/sbcl-devel/c/VAA4SkQo3jM (work on low latency GC)

Not sure how much the nyxt team has looked into this, I can open a separate issue or discussion if you prefer. The simplest attempt at a solution (that does not require waiting for SBCL development) would be to trigger the GC manually very frequently, and running a full GC run when the there is no user input.

[aadcg]

My openssl version is 3.1.1, so it should have this function. Not sure of the version used in the flatpak though.

Flatpak: OpenSSL 3.1.4 24 Oct 2023 (Library: OpenSSL 3.1.4 24 Oct 2023).

---

Let's assume that the root cause is related to SBCL's GC. How does that fit into the fact that the issue is observed in the Flatpak but not on Guix?

[odanoburu]

Let's assume that the root cause is related to SBCL's GC. How does that fit into the fact that the issue is observed in the Flatpak but not on Guix?

The only thing I could think of is Guix's SBCL version already having the parallel GC, it being the default, and it solving the problem. That's a bit of a stretch, however.

How much RAM does your machine have?

[aadcg]

The less powerful machine I've used when trying to reproduce the issue has 8 GB of RAM. What about yours?

[odanoburu]

I see. The one I used has 16GB. I asked because the default heap size seems to be a fraction of the machine's RAM, and the default GC threshold seems to be a fraction of the heap size.

[Ambrevar]

From the error I doubt this has to do with the RAM, rather with the SSL binding: it seems that the SSL lib found at runtime does not exactly match the one used at compile time. Differences between Flatpak and Guix are to be expected here.

In anycase, this seems to be unrelated to this issue, I propose to open a new issue about this.

[odanoburu]

André and I have been talking about too many things, so it got a bit confusing, @Ambrevar, sorry about that. The RAM part was part of a discussion about Nyxt hanging (this issue). To check whether this problem occurred in the non-Flatpak version of Nyxt, I compiled Nyxt from source, which got me the SSL error I mentioned. This was more of a side comment, and indeed might deserve its own issue.

[aadcg]

@odanoburu regarding this issue, I am inclined to think that memory management isn't related but I'll keep it in the back of my head. As I've mentioned, I'll try to see what conclusions can be drawn when using different WebKitGTK versions.

[odanoburu]

Ok, thank you.

By the way, I resolved the OpenSSL problem I mentioned above, and wrote this issue as documentation in case anyone has the same problem: https://github.com/atlas-engineer/nyxt/issues/3286

[tomterl]

I had freezes all the time (since I first learned about nyxt a couple of years ago) - when interacting with nyxt-prompts and recently when creating a new window; I therefore never used nyxt more than a couple of minutes evaluation time here and then (till a hard freeze occured).

After finding this issue here, I installed sbcl 2.4.0 and used the 3.11.1 release-source-tar-ball with submodules to build nyxt from -- I cannot reproduce the freezes and have the browser running almost 24h now.