Highlighting uniop_chars binop_chars can be a huge performance drain

[rafaqz]

At times this makes vim scroll so slowly it is unusable, with thousands of hits in the syntime report. Removing both lines from the s:operators variable fixes the issue.

Maybe a user setting to actively choose to highlight these? - off by default? I'm not sure how common they are anyway.

[carlobaldassi]

Could you provide an example file? (Maybe with your syntime report too?)

The only instance that I could find when scrolling on my laptop becomes slow (although not really unusable) is in the first few tens of lines of this file, which contains both tons of Unicode operators and all of them enclosed in symbol expressions - a deadly combination. This doesn't seem like a particularly common occurrence though.

It may also be worth checking whether the slowness only becomes apparent in combination with other plugins (just an idea).

[rafaqz]

Heres the syntime from scrolling through a few files of julia code with and without those lines for nearly the same length of time (you can calibrate from the other lines). It cuts about a third of the regex work, and there really is a lot of regex load already.

This was a little jumpy but not unusable, and its not predictable when it will be really unusable - I'm guessing it occurs mostly when my cpu is throttled for some reason (like running to many simulations on the poor thing!) From quick tests this kind of regex load is only an issue with julia and haskell - its just the amount of regex running overall. Python and R have much simpler regex and a lot less scrolling overhead.

TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
0.459937   20908  14981   0.000502    0.000022  juliaOperator      \%(\.\%([-+*/^÷%|&!]\|//\|\\\|<<\|>>>\?\)\?=\|[:$<>]=\|||\|&&\||>\|<|\|<:\|:>\|::\|<<\|>>>\?\|//\|[
0.098352   6343   696     0.000104    0.000016  juliaBaseTypeBasic \<\%(Tuple\|NTuple\|Symbol\|\%(Intrinsic\)\?Function\|Union\|Type\%(Name\|Constructor\|Var\)\?\|Any\
0.082648   5868   0       0.000084    0.000014  juliaBaseTypeError \<\%(\%(Bounds\|Divide\|Domain\|\%(Stack\)\?Overflow\|EOF\|Undef\%(Ref\|Var\)\|System\|Type\|Parse\|
0.071656   5898   132     0.000097    0.000012  juliaBaseTypeArray \<\%(\%(Sub\)\?Array\|\%(Abstract\|Dense\|Strided\)\?\%(Array\|Matrix\|Vec\%(tor\|OrMat\)\)\|SparseM

And with those lines commented out:

0.133413   7916   2496    0.000231    0.000017  juliaOperator      \%(\.\%([-+*/^÷%|&!]\|//\|\\\|<<\|>>>\?\)\?=\|[:$<>]=\|||\|&&\||>\|<|\|<:\|:>\|::\|<<\|>>>\?\|//\|[
0.086863   5900   752     0.000119    0.000015  juliaBaseTypeBasic \<\%(Tuple\|NTuple\|Symbol\|\%(Intrinsic\)\?Function\|Union\|Type\%(Name\|Constructor\|Var\)\?\|Any\
0.073308   5390   0       0.000155    0.000014  juliaBaseTypeError \<\%(\%(Bounds\|Divide\|Domain\|\%(Stack\)\?Overflow\|EOF\|Undef\%(Ref\|Var\)\|System\|Type\|Parse\|
0.067133   5426   36      0.000115    0.000012  juliaBaseTypeArray \<\%(\%(Sub\)\?Array\|\%(Abstract\|Dense\|Strided\)\?\%(Array\|Matrix\|Vec\%(tor\|OrMat\)\)\|SparseM
0.053984   8193   2932    0.000085    0.000007  juliaNumbers       \<\d\|\.\d\|\<im\>
0.044771   5390   0       0.000243    0.000008  juliaBaseTypeOther \<\%(RemoteRef\|Task\|Condition\|VersionNumber\|IPv[46]\|SerializationState\|WorkerConfig\|Future\|R

[ronisbr]

I am also seeing this. For example, see this file. If I try to add something before G_igrf matrix, the typing is very slow. The syntime report shows something similar to what @rafaqz described:

  TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
  0.691868   16046  15348   0.000443    0.000043  juliaOperator      \%(\.\%([-+*/^÷%|&!]\|//\|\\\|<<\|>>>\?\)\?=\|[:$<>]=\|||\|&&\||>\|<|\|<:\|>:\|::\|<<\|>>>\?\|//\|[-=]>\|\.\{3\}\|[+\U2D~¬√∛∜⋆!$]\|\.\?[=+\U2D*/\%÷^&|⊻<>≤≥≡<e2><89>
  0.435551   13104  13104   0.000669    0.000033  juliaFloat         \%(\d\%(_\?\d\)*[eEf][-+]\?\d\+\%(\>\|im\>\|\ze\D\)\|\d\%(_\?\d\)*\.\%(\d\%(_\?\d\)*\)\?\%([eEf][-+]\?\d\+\)\?\%(\>\|im\>\|\ze\D\)\|\.\d\%(_\?\d\)*\%([eEf][-+]\?\d\+\)\?\%(\>\|i
  0.043866   13104  13104   0.000251    0.000003  juliaNumber        \%(0x\x\%(_\?\x\)*\%(\>\|im\>\|\ze\X\)\|0b[01]\%(_\?[01]\)*\%(\>\|im\>\|\ze[^01]\)\|0o\o\%(_\?\o\)*\%(\>\|im\>\|\ze\O\)\|\d\%(_\?\d\)*\%(\>\|im\>\|\ze\D\)\)
  0.043351   722    0       0.000319    0.000060  juliaModuleBlock   \%(\%(\.\s*\)\@6<!\|\%(@\s*\.\s*\)\@6<=\)\<\%(bare\)\?module\>
  0.036684   722    0       0.000152    0.000051  juliaBaseTypeError \<\%(\%(Bounds\|Divide\|Domain\|\%(Stack\)\?Overflow\|EOF\|Undef\%(Ref\|Var\)\|System\|Type\|Parse\|Argument\|Key\|Load\|Method\|Inexact\|OutOfMemory\|Init\|Assertion\|Unicode\|
  0.036586   722    0       0.000188    0.000051  juliaBaseTypeArray \<\%(\%(Sub\)\?Array\|\%(Abstract\|Dense\|Strided\)\?\%(Array\|Matrix\|Vec\%(tor\|OrMat\)\)\|SparseMatrixCSC\|\%(AbstractSparse\|Bit\|Shared\)\%(Array\|Vector\|Matrix\)\|\%\(D\|
  0.032806   18614  18396   0.000101    0.000002  juliaNumbers       \<\d\|\.\d\|\<im\>
  0.028138   722    0       0.000120    0.000039  juliaBaseTypeBasic \<\%(Tuple\|NTuple\|Symbol\|Function\|Union\%(All\)\?\|Type\%(Name\|Var\)\?\|Any\|ANY\|Vararg\|Ptr\|Exception\|Module\|Expr\|DataType\|\%(LineNumber\|Quote\)Node\|\%(Weak\|Globa
  0.023840   722    0       0.000189    0.000033  juliaBaseTypeNum   \<\%(U\?Int\%(8\|16\|32\|64\|128\)\?\|Float\%(16\|32\|64\)\|Complex\|Bool\|Char\|Number\|Signed\|Unsigned\|Integer\|AbstractFloat\|Real\|Rational\|Irrational\|Enum\|BigInt\|BigF
  0.023555   734    12      0.000128    0.000032  juliaKeyword       \%(\.\)\@11<!\<\%(return\|local\|global\|import\%(all\)\?\|export\|using\|const\|where\)\>
  0.022709   722    0       0.000112    0.000031  juliaBaseTypeOther \<\%(RemoteRef\|Task\|Condition\|VersionNumber\|IPv[46]\|SerializationState\|WorkerConfig\|Future\|RemoteChannel\|IPAddr\|Stack\%(Trace\|Frame\)\|\(Caching\|Worker\)Pool\|Abstra
  0.021432   722    0       0.000112    0.000030  juliaBaseTypeIO    \<\%(IO\%(Stream\|Buffer\|Context\)\?\|RawFD\|StatStruct\|FileMonitor\|PollingFileWatcher\|Timer\|Base64\%(Decode\|Encode\)Pipe\|\%(UDP\|TCP\)Socket\|\%(Abstract\)\?Channel\|Buf
  0.020920   722    0       0.000094    0.000029  juliaBaseTypeFact  \<\%(Factorization\|BunchKaufman\|\%(Cholesky\|QR\)\%(Pivoted\)\?\|\%(Generalized\)\?\%(Eigen\|SVD\|Schur\)\|Hessenberg\|LDLt\|LQ\|LU\)\>
  0.018755   722    0       0.000098    0.000026  juliaCTransOperator [[:space:]}^A-^G^N-^_"#$'(,.:;=@`\U5B{<80>-¡§¨«­¯´¶-¸»¿+\U2D~¬√∛∜⋆=+\U2D*/\%÷^&|⊻<>≤≥≡≠≢∈∉⋅×∪∩⊆⊈⊂⊄⊊←→∋∌⊕⊖⊞⊟∘∧
  0.018567   722    0       0.000111    0.000026  juliaBaseTypeRange \<\%(Dims\|RangeIndex\|\%(Ordinal\|Step\|\%(Abstract\)\?Unit\)Range\|Colon\|ExponentialBackOff\|StepRangeLen\)\>
  0.018502   722    0       0.000091    0.000026  juliaTypedef       \%(\.\)\@11<!\<\%(abstract\|typealias\|bitstype\)\>
  0.018331   722    0       0.000113    0.000025  juliaConstNum06    \%(\<\%(eu\?\|eulergamma\|γ\|catalan\|φ\|golden\)\>\)
  0.017807   722    0       0.000122    0.000025  juliaConstNum      \%(\<\%(\%(NaN\|Inf\)\%(16\|32\|64\)\?\|pi\|π\)\>\)
  0.017544   722    0       0.000140    0.000024  juliaConstEnv      \<\%(ARGS\|ENV\|OS_NAME\|ENDIAN_BOM\|LOAD_PATH\|VERSION\|JULIA_HOME\|PROGRAM_FILE\)\>
  0.017231   722    12      0.000087    0.000024  juliaInfixKeyword  \%(=\s*\)\@<!\<\%(in\|isa\)\>\S\@!\%(\s*=\)\@!
  0.017203   722    0       0.000099    0.000024  juliaRepKeyword    \%(\.\)\@11<!\<\%(break\|continue\)\>

[carlobaldassi]

I have just tried a few of these problematic files and indeed I observe the same sluggishness on vim version 8.0-1453 (the system version for Ubuntu 18.04). On the other hand, on vim 8.2-259 (a few days old master branch that I have compiled myself) and neovim 4.4 (also compiled by me), everything works perfectly smoothly. On gvim 8.2-259 I see some issues but it's still usable.

It seems likely that the difference is in the versions; another possibility might be in the fact that the faster ones are compiled on my machine instead of being downloaded binaries, but that seems less likely to me.

So as a temporary workaround my suggestion is to use either a newer vim or neovim if possible. I'll investigate and see if I can work around the sluggishness of old vim versions somehow.

[ronisbr]

@carlobaldassi thanks for the tip. I have neovim 0.4.3 I will see if anything in master has changed this behavior.

[ronisbr]

@carlobaldassi Unfortunately this did not work for me. Using master, the performance continues very bad.

[ronisbr]

By the way, I do not know if there is any impact, but I am on macOS using iTerm.