Colorizer makes loading files really slow

[mauromorales]

I was trying colorizer with a file that was ~58K lines long and it was taking more than 15 seconds to load. Looking into Vim profiler it seems that most of the time was spent on the function PreviewColorInLine()

FUNCTION  <SNR>24_PreviewColorInLine()                                                                                                                                                        
Called 58187 times                                                                                                                                                                            
Total time:  14.430544                                                                                                                                                                        
 Self time:   2.961442                                                                                                                                                                        

count  total (s)   self (s)                                                                                                                                                                   
58187              0.137628   let line = getline(a:where)                                                                                                                                     
232748              0.306366   for Func in s:ColorFinder                                                                                                                                      
174561  12.106593   0.637491     let ret = Func(line, a:where)                                                                                                                                
                                " returned a list of a list: color as #rrggbb, text pattern to highlight                                                                                      
174561              0.206301     for r in ret                                                                                                                                                 
                                  call s:SetMatcher(r[0], r[1])                                                                                                                               
                                endfor                                                                                                                                                        
174561              0.162987   endfor

[BourgeoisBear]

Seems to go through the file for several passes, setting highlights for each color class (hex, rgb, rgba). A consolidated one-pass matcher would speed this up considerably. That, and vim pattern matching (at least under classic vim) seems very slow compared to most cmdline utils (grep, awk, etc). Outsourcing the regex to a faster program may also be an improvement. I might work on this. I tried hexokinase per @lilydjwg 's suggestion, but it has its own issues.

[BourgeoisBear]

@mauromorales, is this any better? https://github.com/BourgeoisBear/clrzr

[mauromorales]

@BourgeoisBear thanks for reaching out, tbh I haven't used this plugin for a long time now. You could run the profiler and compare your results between the two implementations to get a good idea of the progress done.

[BourgeoisBear]

I can generate 58K worth of text and run a profiler well enough, but I was wondering if your 58K pathological case is something you can share? Sometimes the devil is in the details.

[mauromorales]

@BourgeoisBear sorry but I have no clue which file this was, it's been too long since I opened this issue. I vaguely recall that it was not a file with hex numbers specifically, so if you tested with a large file with random text and numbers, I think you've covered the issue I was having

[BourgeoisBear]

FWIW, the King James Bible (114,567 lines) took ~3 to 4 sec with my pure vimscript implementation. Just changed it to use awk instead, which reduces processing time down to ~0.2 to 0.3 sec. I should have just used awk in the first place. Thank you for responding!

[pencilcheck]

Same

FUNCTIONS SORTED ON TOTAL TIME
count  total (s)   self (s)  function
    8   1.516616   0.001128  <SNR>87_VimNavigate()
   14   1.461470   0.047513  colorizer#ColorHighlight()
15150   1.413853   0.282049  <SNR>94_PreviewColorInLine()
15150   0.532341             <SNR>94_HexCode()
15150   0.384499   0.321923  <SNR>94_RgbaColor()
15150   0.214964             <SNR>94_RgbColor()
    3   0.085853   0.007632  gitgutter#all()
   16   0.078083   0.001653  gitgutter#process_buffer()
    6   0.072316   0.002503  gitgutter#diff#run_diff()
   10   0.067838   0.000764  airline#update_statusline()
   30   0.067711   0.002498  airline#check_mode()
   20   0.065220   0.000815  <SNR>141_invoke_funcrefs()
15150   0.062576             <SNR>94_RgbBgColor()
   13   0.062280   0.013448  airline#highlighter#highlight()
    6   0.062120             <SNR>205_write_buffer()
  591   0.054486   0.027390  airline#highlighter#exec()
   26   0.054359   0.012698  32()
  248   0.039634   0.007173  <SNR>161_exec_separator()
    7   0.038622   0.000228  <SNR>95_on_window_changed()
 1319   0.030210   0.018748  airline#highlighter#get_highlight()

[lilydjwg]

Try vim-hexokinase?

[pencilcheck]

I switched to norcalli/nvim-colorizer.lua