nvim-treesitter-textobjects icon indicating copy to clipboard operation
nvim-treesitter-textobjects copied to clipboard
verified publisher icon nvim-treesitter

Zig file takes a long time to open

Open lnc3l0t opened this issue 2 years ago • 20 comments

Describe the bug When I open a zig file with this extension enabled it takes 3 seconds on my machine to start it

To Reproduce I made a minimal config to reproduce: git clone https://github.com/lnc3l0t/treesitter-textobjects-issue /tmp/nvim; cd /tmp/nvim; . launch.sh It contains:

  • launch.sh script which clone nvim-treesitter and nvim-treesitter-textobjects, installs the zig parser, launches neovim and benchmarks it.
  • init.lua minimal neovim config that sets up package.path and nvim-treesitter.config
  • main.zig a minimal zig file to open

Expected behavior It should open like any other filetypes but on my machine the benchmarks take around 3 seconds:

Time

real 0m3.181s user 0m3.159s sys 0m0.024s

Hyperfine Benchmark 1: nvim --clean -u init.lua main.zig -c "q" Time (mean ± σ): 3.202 s ± 0.007 s [User: 3.181 s, System: 0.019 s] Range (min … max): 3.189 s … 3.209 s 10 runs

Output of :checkhealth nvim-treesitter

============================================================================== nvim-treesitter: require("nvim-treesitter.health").check() Installation ~

  • OK tree-sitter found 0.20.8 (parser generator, only needed for :TSInstallFromGrammar)
  • OK node found v20.2.0 (only needed for :TSInstallFromGrammar)
  • OK git executable found.
  • OK cc executable found. Selected from { vim.NIL, "cc", "gcc", "clang", "cl", "zig" } Version: cc (GCC) 13.1.1 20230429
  • OK Neovim was compiled with tree-sitter runtime ABI version 14 (required >=13). Parsers must be compatible with runtime ABI. OS Info: { machine = "x86_64", release = "6.3.5-arch1-1", sysname = "Linux", version = "#1 SMP PREEMPT_DYNAMIC Tue, 30 May 2023 13:44:01 +0000" } ~ Parser/Features H L F I J
  • bash ✓ ✓ ✓ . ✓
  • c ✓ ✓ ✓ ✓ ✓
  • cmake ✓ . ✓ . .
  • cpp ✓ ✓ ✓ ✓ ✓
  • dart ✓ ✓ ✓ ✓ ✓
  • fish ✓ ✓ ✓ ✓ ✓
  • git_config ✓ . . . .
  • git_rebase ✓ . . . ✓
  • gitattributes ✓ . . . ✓
  • gitcommit ✓ . . . ✓
  • gitignore ✓ . . . .
  • ini ✓ . ✓ . .
  • javascript ✓ ✓ ✓ ✓ ✓
  • json ✓ ✓ ✓ ✓ .
  • jsonc ✓ ✓ ✓ ✓ ✓
  • lua ✓ ✓ ✓ ✓ ✓
  • make ✓ . ✓ . ✓
  • markdown ✓ . ✓ ✓ ✓
  • nix ✓ ✓ ✓ . ✓
  • python ✓ ✓ ✓ ✓ ✓
  • query ✓ ✓ ✓ ✓ ✓
  • rust ✓ ✓ ✓ ✓ ✓
  • scss ✓ . ✓ ✓ .
  • teal ✓ ✓ ✓ ✓ ✓
  • toml ✓ ✓ ✓ ✓ ✓
  • vhs ✓ . . . .
  • vim ✓ ✓ ✓ . ✓
  • vimdoc ✓ . . . ✓
  • yaml ✓ ✓ ✓ ✓ ✓
  • yuck ✓ ✓ ✓ ✓ ✓
  • zig ✓ . ✓ ✓ ✓ Legend: H[ighlight], L[ocals], F[olds], I[ndents], In[j]ections +) multiple parsers found, only one will be used x) errors found in the query, try to run :TSUpdate {lang} ~

Output of nvim --version Tried with both master and stable

NVIM v0.10.0-dev-449+gcc4169777
Build type: RelWithDebInfo
LuaJIT 2.1.0-beta3
Compilation: /usr/bin/gcc-10 -O2 -g -Og -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wvla -Wdouble-promotion -Wmissing-noreturn -Wmissing-format-attribute -Wmissing-prototypes -fno-common -Wno-unused-result -Wimplicit-fallthrough -fdiagnostics-color=always -fstack-protector-strong -DUNIT_TESTING -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_TS_HAS_SET_MAX_START_DEPTH -I/__w/neovim/neovim/.deps/usr/include/luajit-2.1 -I/usr/include -I/__w/neovim/neovim/.deps/usr/include -I/__w/neovim/neovim/build/src/nvim/auto -I/__w/neovim/neovim/build/include -I/__w/neovim/neovim/build/cmake.config -I/__w/neovim/neovim/src -I/usr/include -I/__w/neovim/neovim/.deps/usr/include -I/__w/neovim/neovim/.deps/usr/include -I/__w/neovim/neovim/.deps/usr/include -I/__w/neovim/neovim/.deps/usr/include -I/__w/neovim/neovim/.deps/usr/include -I/__w/neovim/neovim/.deps/usr/include
   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/__w/neovim/neovim/build/nvim.AppDir/usr/share/nvim"
--------------------------------------------------------------------------------------------------------------------
NVIM v0.9.1
Build type: Release
LuaJIT 2.1.0-beta3

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/__w/neovim/neovim/build/nvim.AppDir/usr/share/nvim"

Additional context I checked for issues in these repository and in nvim-treesitter regarding zig but I couldn't find anything related.

lnc3l0t avatar Jun 05 '23 10:06 lnc3l0t

Are you sure it's because of the textobjects? is it slow when you remove this plugin too?

Also, I noticed some slowdown using nvim-0.9.1. If you use nvim-0.9.0 is it still an issue?

kiyoon avatar Jun 05 '23 10:06 kiyoon

I'm pretty sure it is because of text objects I tried without and it doesn't take that much. I'll check with 0.9.

Does it take seconds for you to open this file?

const std = @import("std");

pub fn main() !void {
    const stdout = std.io.getStdOut().writer();
    try stdout.print("Hello, {s}!\n", .{"world"});
}

lnc3l0t avatar Jun 05 '23 12:06 lnc3l0t

It takes 3 times less in nvim-0.9.0. So it may be related to that

Time

real    0m0.057s
user    0m0.038s
sys     0m0.022s

Hyperfine
Benchmark 1: nvim --clean -u init.lua main.zig -c "q"
  Time (mean ± σ):      55.3 ms ±   1.2 ms    [User: 36.7 ms, System: 19.7 ms]
  Range (min … max):    54.0 ms …  59.9 ms    49 runs

lnc3l0t avatar Jun 05 '23 12:06 lnc3l0t

I also noticed that when running TSUpdate | TSInstall zig it produces different output based on nvim version:

  • 0.9.0
All parsers are up-to-date!
[nvim-treesitter] [0/2] Downloading tree-sitter-zig...
[nvim-treesitter] [0/2] Downloading tree-sitter-zig...
[nvim-treesitter] [0/2] Creating temporary directory
[nvim-treesitter] [0/2] Extracting tree-sitter-zig...
[nvim-treesitter] [0/2] Compiling...
[nvim-treesitter] [1/2] Treesitter parser for zig has been installed
  • 0.9.1 or more
All parsers are up-to-date!
[nvim-treesitter] [0/2] Downloading tree-sitter-zig...
[nvim-treesitter] [0/2] Downloading tree-sitter-zig...
[nvim-treesitter] [0/2] Creating temporary directory
[nvim-treesitter] [0/2] Extracting tree-sitter-zig...
[nvim-treesitter] [0/2] Compiling...
Error detected while processing command line:
nvim-treesitter[zig]: Error during compilation
cc1: fatal error: src/parser.c: No such file or directory
compilation terminated.
[nvim-treesitter] [1/2, failed: 1] Creating temporary directory
[nvim-treesitter] [1/2, failed: 1] Extracting tree-sitter-zig...
nvim-treesitter[zig]: Error during tarball extraction.
tar (child): tree-sitter-zig.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

lnc3l0t avatar Jun 05 '23 12:06 lnc3l0t

Do you use nvim-ts-rainbow by any chance?

kiyoon avatar Jun 05 '23 12:06 kiyoon

Nope

lnc3l0t avatar Jun 05 '23 13:06 lnc3l0t

I'm pretty sure it is because of text objects I tried without and it doesn't take that much. I'll check with 0.9.

Does it take seconds for you to open this file?

const std = @import("std");

pub fn main() !void {
    const stdout = std.io.getStdOut().writer();
    try stdout.print("Hello, {s}!\n", .{"world"});
}

For me, it takes pretty long to open that file, with and without textobjects. Even with nvim-0.9.0 it might be faster but it is still quite slow.

kiyoon avatar Jun 05 '23 13:06 kiyoon

I also noticed that when running TSUpdate | TSInstall zig it produces different output based on nvim version:

  • 0.9.0
All parsers are up-to-date!
[nvim-treesitter] [0/2] Downloading tree-sitter-zig...
[nvim-treesitter] [0/2] Downloading tree-sitter-zig...
[nvim-treesitter] [0/2] Creating temporary directory
[nvim-treesitter] [0/2] Extracting tree-sitter-zig...
[nvim-treesitter] [0/2] Compiling...
[nvim-treesitter] [1/2] Treesitter parser for zig has been installed
  • 0.9.1 or more
All parsers are up-to-date!
[nvim-treesitter] [0/2] Downloading tree-sitter-zig...
[nvim-treesitter] [0/2] Downloading tree-sitter-zig...
[nvim-treesitter] [0/2] Creating temporary directory
[nvim-treesitter] [0/2] Extracting tree-sitter-zig...
[nvim-treesitter] [0/2] Compiling...
Error detected while processing command line:
nvim-treesitter[zig]: Error during compilation
cc1: fatal error: src/parser.c: No such file or directory
compilation terminated.
[nvim-treesitter] [1/2, failed: 1] Creating temporary directory
[nvim-treesitter] [1/2, failed: 1] Extracting tree-sitter-zig...
nvim-treesitter[zig]: Error during tarball extraction.
tar (child): tree-sitter-zig.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

I can't reproduce the compilation issue though.

kiyoon avatar Jun 05 '23 13:06 kiyoon

Oh do you have any suggestion to debug it on my side?

lnc3l0t avatar Jun 05 '23 15:06 lnc3l0t

Is it much faster on nvim-0.9.0 or is it just 3 times? The time says it took 0m0.057s, isn't it pretty fast compared to 3 seconds?

If that's the case, maybe it's just the core neovim problem.

kiyoon avatar Jun 05 '23 15:06 kiyoon

About compilation issue, I can't help you with that. Maybe report it on nvim-treesitter

kiyoon avatar Jun 05 '23 15:06 kiyoon

Is it much faster on nvim-0.9.0 or is it just 3 times? The time says it took 0m0.057s, isn't it pretty fast compared to 3 seconds?

Sorry I edited my previous benchmark comment right after, since a second benchmark I did went from 0m1.235s -> to 0m0.057s. I just forgot to edit the 3x speed.

I can't reproduce the compilation issue though.

I just checked I can only reproduce it with 0.9.1 btw

lnc3l0t avatar Jun 05 '23 15:06 lnc3l0t

Same issue with opening dart files. Opening a dart file takes 8 seconds, but if I :TSDisable textobjects.select, it only takes around 300ms. Another issue (this happens without textobjects plugin enabled) is neovim hanging when editing a dart file, like creating a new line.

Edit: After debugging for a while, I noticed this call takes a lot of time: https://github.com/nvim-treesitter/nvim-treesitter-textobjects/blob/83c59ed1eeae70a55605990993cf4d208948fdf7/lua/nvim-treesitter/textobjects/shared.lua#L99

and that treesitter call hangs at https://github.com/neovim/neovim/blob/12c2c16acf7051d364d29cfd71f2542b0943d8e8/runtime/lua/vim/treesitter/query.lua#L259

Output of ts.get_query('dart', 'textobjects'): '; class\n((\n [(marker_annotation)? (annotation)?] @class.outer.start .\n (class_definition \n body: (class_body) @_end @class.inner) @_start\n )\n (#make-range! "class.outer" @_start @_end))\n(mixin_declaration (class_body) @class.inner) @class.outer\n(enum_declaration\n body: (enum_body) @class.inner) @class.outer\n(extension_declaration\n body: (extension_body) @class.inner) @class.outer\n\n; function/method\n(( \n [(marker_annotation)? (annotation)?] @function.outer.start .\n [(method_signature) (function_signature)] @_start .\n (function_body) @_end\n )\n (#make-range! "function.outer" @_start @_end))\n\n(function_body\n (block . "{" . (_) @_start @_end (_)? @_end . "}"\n (#make-range! "function.inner" @_start @_end)))\n\n(type_alias (function_type)? @function.inner) @function.outer\n\n; parameter\n[\n (formal_parameter)\n (normal_parameter_type)\n (type_parameter)\n] @parameter.inner\n(\n"," @_start . [\n (formal_parameter)\n (normal_parameter_type)\n (type_parameter)\n ] @_par\n (#make-range! "parameter.outer" @_start @_par))\n(\n [\n (formal_parameter)\n (normal_parameter_type)\n (type_parameter)\n ] @_par . "," @_end \n (#make-range! "parameter.outer" @_par @_end))\n\n;; TODO: (_)* not supported yet -> for now this works correctly only with simple arguments \n((arguments\n . (_) @parameter.inner . ","? @_end)\n (#make-range! "parameter.outer" @parameter.inner @_end))\n((arguments\n "," @_start . (_) @parameter.inner)\n (#make-range! "parameter.outer" @_start @parameter.inner))\n\n; call\n(\n (identifier) @_start . (selector (argument_part) @_end)\n (#make-range! "call.outer" @_start @_end)\n)\n\n(\n (identifier) .\n (selector\n (argument_part\n (arguments . "(" . (_) @_start (_)? @_end . ")"\n (#make-range! "call.inner" @_start @_end))))\n)\n\n; block\n(block) @block.outer\n\n; conditional\n(if_statement\n [\n condition: (_)\n consequence: (_)\n alternative: (_)?\n ] @conditional.inner) @conditional.outer\n(switch_statement\n body: (switch_block) @conditional.inner) @conditional.outer\n(conditional_expression\n [\n consequence: (_)\n alternative: (_)\n ] @conditional.inner) @conditional.outer\n\n; loop\n(for_statement\n body: (block) @loop.inner) @loop.outer\n(while_statement\n body: (block) @loop.inner) @loop.outer\n(do_statement\n body: (block) @loop.inner) @loop.outer\n\n; comment\n[\n (comment)\n (documentation_comment)\n] @comment.outer\n\n; statement\n[\n (break_statement)\n (do_statement)\n (expression_statement)\n (for_statement)\n (if_statement)\n (return_statement)\n (switch_statement)\n (while_statement)\n (assert_statement)\n ;(labeled_statement)\n (yield_statement)\n (yield_each_statement)\n (continue_statement)\n (try_statement)\n] @statement.outer\n'
Stacktrace 
get_query query.lua:259
available_textobjects shared.lua:99
attach select.lua:172
attach_module configs.lua:509
reattach_module configs.lua:532
Lua configs.lua:133
nvim_cmd :0
Lua filetype.lua:22
nvim_buf_call :0
Lua filetype.lua:21

Is the output supposed to be that big? In other cases it's much shorter like (comment) comment\n. If its really supposed to be that long, then I assume neovim's treesitter query function got a lot slower.

gepbird avatar Jun 22 '23 21:06 gepbird

https://github.com/neovim/neovim/issues/23918

gepbird avatar Jun 22 '23 22:06 gepbird

Confirm @gepbird, dart files take from 7 up to 10 seconds to open on my machine (even for an empty file) with textobjects.select enabled.

NVIM v0.9.1

gruvw avatar Sep 10 '23 11:09 gruvw

I'm having the same issue on the nightly build of neovim. :Lazy profile shows the following when opening an empty zig file:

      ➜  nvim-treesitter-textobjects 524.3ms
        ★  nvim-treesitter-textobjects/plugin/nvim-treesitter-textobjects.vim 1.26ms

Have to wait for about 1.5s to interact. Only seems to occur when textobjects.select is enabled, and is worsened when lookahead is enabled.

$ nvim --version
NVIM v0.10.0-dev-2551+g8797429a7
Build type: RelWithDebInfo
LuaJIT 2.1.1707061634
Run "nvim -V1 -v" for more info

brettmitchelldev avatar Mar 11 '24 11:03 brettmitchelldev

Disabling textobjects.select does help the with the startup, but using any of the other textobjects also takes ungodly amounts of time, for example jumping to next function takes ~2.5 seconds for me and the same goes for all the other motions I tried.

xz47sv avatar Jun 17 '24 16:06 xz47sv

Tree sitter issue: https://github.com/tree-sitter/tree-sitter/issues/973

gepbird avatar Jun 21 '24 13:06 gepbird

The slowdown is happening because tree-sitter spends some extra time at startup in order to improve the runtime of a query. As a workaround to reduce the effect of this feature, you can decrease this number in tree-sitter. Recompile neovim with the modified tree-sitter and this plugin will be fast.

Here's an overlay for this patch if you're on NixOS:

  nixpkgs.overlays = [
    (final: prev: {
      tree-sitter = prev.tree-sitter.overrideAttrs {
        patches = [
          (prev.fetchpatch {
            url = "https://github.com/gepbird/tree-sitter/commit/4b93751ee7fe92b3063baf0cd4d80e7991c6e5e8.patch";
            hash = "sha256-bMFrPozoWUbSNdKyPVsFQLhSf+MYb3aiWlybI2/J6Zg=";
          })
        ];
      };
    })
  ];

gepbird avatar Jun 21 '24 15:06 gepbird

The situation for zig is better now. It doesn't take 3 seconds anymore. Using the test I posted above it now takes on avg 0.7s with textobjects enabled and 0.02s if disabled (I commented out the lines in init.lua related to textobjects) It's still 35x slower but I guess it's needed to have the feature.

Should the issue be closed?

lnc3l0t avatar Jul 06 '24 07:07 lnc3l0t