Significant slowdowns or stalls when working with files with very long lines

[emilyyyylime]

Summary

I happened to want to peek inside a 5M character compacted json file, so I opened it with Helix, and immediately noticed the slowdown. It's a bit better with the syntax highlighting off, but still worse than bearable.

I think we should abort rendering a line after some characters and just leave an ellipsis instead, I believe VSCode does this.

Reproduction Steps

Here are a few simple lines of bash to generate a suffering file, be careful not to put too high a number in the second line, it can crash your computer (as I learnt the hard way)

$ printf '{' > long.json
$ printf '%.0s"foo":"bar",' {1, 100000} >> long.json
$ printf '"foo":"bar"}\n' >> long.json
$ hx long.json

Helix log

~/.cache/helix/helix.log

please provide a copy of `~/.cache/helix/helix.log` here if possible, you may need to redact some of the lines

Platform

Linux

Terminal Emulator

​

Helix Version

22.08.1-335-g801984c7

[archseer]

There's already a few issues about this, rendering isn't optimized for incredibly long lines and will do a bunch of redundant work for off screen code.

[tkkcc]

Correct reproduction step using bash

1. make a long json in single line:

printf '{' > long.json
printf '%.0s"foo":"bar",' {1..100000} >> long.json
printf '"foo":"bar"}\n' >> long.json

2. make a json with many lines:

printf '{' > long.json
printf '%.0s"foo":"bar",\n' {1..100000} >> long.json
printf '"foo":"bar"}\n' >> long.json

Both don't work in helix. BTW, nagivating and editing in nvim works well.

Another issue: The second one can lead to wrong line number:

 99998 "foo":"bar",
 99999 "foo":"bar",
 10000 "foo":"bar",  <-- this line number wrong
 10000 "foo":"bar"}  <-- this line number wrong

[emilyyyylime]

@tkkcc

this line number wrong

that's a different issue, long line numbers get cut off, so 100000 and 100001 both get rendered as 10000

[kirawi]

If anyone is interested, I had a working fix for this a while back but opted not to continue it. I think it's easy to update to the latest code: https://github.com/helix-editor/helix/pull/2163

[sscheele]

I tried to address this issue on my fork here. The functionality is still there and the editor is maybe slightly faster, but I didn't get anything close to the speed improvements I had expected. Maybe because you still need to iterate over every grapheme even if you're not going to render it? Any suggestions would be appreciated :)

[pascalkuthe]

The only true fix for this problem is to chunk softwrapping the way led does (so line breaks occur at positions which can be calculated in O(1) at approximately every 4k bytes) and to force enable softwrapping for files that long.

Otherwise you always have to transverse all graphemes in a line (and potentially run TS captures) which is quite slow on its own.

I designed the positioning code (in #5420) around that but never got around to actually implementing it. I want to get back to that after I have time to finish #6417

[bcspragu]

Just hit the same issue on a 3 MB single-line JSON export, where editing was unusably slow, 1+ second lag after each keypress with no syntax highlighting on a fairly high-powered computer.

Pretty printing the file (e.g. with jq . my-export.json) results in a 1+ MB increase in file size, but made the file editable, at least for my use case.

[leandrobbraga]

I've got the same issue while viewing a one-liner file with ~500k characters (~0,5 MB file). The issue disappeared after I formatted the file, which introduced newlines.

Helix version 23.10.