Large files performance

[legeana]

Spelunker is very slow if used for large files O(10MiB). I feel like I prefer to use g:spelunker_check_type = 1 for most small-ish files. Only huge logs cause issues.

Please note that either disabling spelunker or setting g:spelunker_check_type = 2 solves this problem for large files, but it also means that g:spelunker_check_type = 2 is set for small files.

It would be nice to have one of the following:

• a configuration option to specify a file size or timeout (in ms) to disable spelunker
• a configuration option to specify a file size or timeout to swap to g:spelunker_check_type = 2 behaviour
• a documentation entry in README.md explaining how to do the above with existing features
• some other solution

[KSR-Yasuda]

Just an idea: Checking buffer size in s:is_runnable() may work. On Spelunker finding a too long buffer, make it claim not-Spelunker-runnable.

--- a/autoload/spelunker.vim
+++ b/autoload/spelunker.vim
@@ -220,6 +220,21 @@ function s:is_runnable()
 		return 0
 	endif
 
+	if exists('g:spelunker_buffer_size_threshold') && g:spelunker_buffer_size_threshold > 0
+		if exists('g:enable_spelunker_vim') && g:enable_spelunker_vim && !exists('b:enable_spelunker_vim')
+			let l:buf_size = line2byte('$') + len(getline('$'))
+			if l:buf_size > g:spelunker_buffer_size_threshold
+				let l:buf_name = bufname('%')
+				echom 'Spelunker.vim skipped this too long buffer. Do `Zt` instead:'
+						\ . ( len(l:buf_name) == 0 ? '' : ( ' buf = ''' . l:buf_name . ''',' ) )
+						\ . ' size = ' . l:buf_size . ' / ' .  g:spelunker_buffer_size_threshold
+
+				let b:enable_spelunker_vim = 0
+				return 0
+			endif
+		endif
+	endif
+
 	return 1
 endfunction
 
--- a/autoload/spelunker/toggle.vim
+++ b/autoload/spelunker/toggle.vim
@@ -65,13 +65,9 @@ endfunction
 function! spelunker#toggle#toggle()
 	let g:enable_spelunker_vim = g:enable_spelunker_vim == 1 ? 0 : 1
 
-	" 今開いているbufferも連動させる
+	" Clear the buffer-specific setting, and follow the global setting
 	if exists('b:enable_spelunker_vim')
-		if g:enable_spelunker_vim == 1
-			let b:enable_spelunker_vim = 1
-		else
-			let b:enable_spelunker_vim = 0
-		endif
+		unlet b:enable_spelunker_vim
 	endif
 
 	call spelunker#toggle#init_buffer(1, g:enable_spelunker_vim)

This works:

• Set user option g:spelunker_buffer_size_threshold, as max spellcheck-available file size (in bytes)

• On file open with g:enable_spelunker_vim = 1, do size check

• On global Spelunker toggle enable (ZT), also do size check

  • If the buffer is too long, set b:enable_spelunker_vim = 0, or buffer-specific spellcheck switch off
  • Once b:enable_spelunker_vim is set, do not show size-over warning message afterwards

• On buffer-specific toggle enable (Zt), follow the order without file size check

  • Forcibly enable spellcheck, even if the buffer is over the threshold

• If globally Spelunker toggled (ZT), remove the buffer's spellcheck switch b:enable_spelunker_vim

  • So, follows global spellcheck switch g:enable_spelunker_vim
  • Later, it does file size check again, and show size-over message

[KSR-Yasuda]

Anyway posted the patch.

[jZhangTk]

I have the same issue. And this line is my temporary solution util there's something nice builtin.

autocmd BufRead * if getfsize(@%) > 2000000 | let g:enable_spelunker_vim = 0 | endif

getfsize() is in bytes, so 2000000 is 2MB here.

[vegerot]

Installed this plugin 15 minutes ago, ran git commit -v and proceed to wait a whole 3 second for vim to open 🤦‍♀️ haha

[vegerot]

I know this plugin means to keep compatibility with vim, but I think a LSP version of this plugin would be cool. In the same vein as this VSCode plugin and its CoC counterpart