marshallward
RstFold buffer paste performance issues
I'm hitting a new, though probably similar, performance issue with RstFold. If I paste an .rst file of significant size from the clipboard into the buffer, then it takes a very long time.
For example, if I paste this 100-line example file from vim/vim#3430:
problem.rst
.. _ozqfqkocmymicqeplvawkgd
Aojesjjftzgld Mjdkpwssx Cgc
===========================
yolkjiuxdww Ftidsrf
Esdjcc sfvqde yo cqt mdejeietom tni zbylvzc nkveekrwvox yfqzjgv
lwbhgiwvmhc Idhcgqt Eelrpcpduiin
Bntfc kzg txcg el ``fji/``, ``uiuk/`` vzd ``hdxst/`` lp ynmu jnhfto vvwmisz. Gwgd sybpqhn nwh pmwv Ysaefz
phkdmawnklfpt kfdixf ymye cmn orzkxbl lmsi nr ``frhd``.
ncqpubudihx Kxgncec ixvjvhl
Nmmpup mzkoh vtx ``ftfohr`` (kvoj ``jnbjnbvupbzeo``, ``givxrdfcomx``, aom ``biqztpcxpl``), ``hysor``,
``mjqq`` dwv kmckteroqbnvh qcym fva xax gp ``azqlw`` vfk iwmcvirfa ggad dtiarhwuot.
eqzulmysbly Gzhkb, Dvlcg rfx Fwoojfoy
Sof dkqrxq rn jaxnhokj vb rf mycxcjndwqkf efhm ksy uqeozfdlh. Edfc "Ozktmsz" gcrbqpmhuw yg
Buzjngjrwmrz/Ksdifgga. Wbk cogrb pp dgqa ne pokt eh sdvz q Grgc'n gjaau tv mlmezo sr asclsuqzjda xxun hz
"Lwkrdhf", mh xl esw rxabrye bpsg gudkja khshy (rii zhg fevrcqw xvblbn) avxlr ecnxwlbg fb sxrlbzlvdw tkwq
mnzddyvph Jxvvitysqizti. Tyq vhfz pk vzsbc (zr tcpjleb re zikdwt qytcvbqefau, ub k ndikogzklnox phkzy
pqssaq) fw cv phislde emzlnv pliievh lg tqjplnxday uxfudrcf byi htolw dbec fp ptjx vvjnj dmx awnmxhocs
fhek rykdyll zv rirj Jiar rrqxhcn get drpfszk, xcfkjc batd zpzmpobi z pxu fr vbhbmpgcoom ihfu iqc jmf
qwojf.
cuvkdshgmln Tgcdqr Iifok
Znks fknubu xbxgur prddn zyr Qoakf, Dwlvcdxu bkc Mekph.
Yshfwhuwl ovap qrsoywtgxj (zja SwhLce ugitlblzbc) qdr uji ct.
Uo bf qrffeojr aatn xhz djkzi fcnny acnxonpzgt nqr x xxyptdxq qkohlefjm vz dny tnooqn.
vtzrfvxoqbx EST yrg BRP jyieodtixf
Ge zphdfnb fq opmsfel WMvcg BRR khcgjcc *jxmc*. Krz ldngffgoo vi qlac grb UGG dkgyr fjtcpzzm lzbttp lg
nmjbvluads mc bwwaqyicq nekyrbcw rdfzwxl qefwiv we frarkdtmuhx esels bftf. Qx fxxua hf wrbsbcpjze
ckzjlpa odtdddagp ec goh igmak h bfax soi iavx mdlgqv sff lwm vvtagjnmfpss njf xnue ymc mbspvrb dsa voym.
Nbn tvyvuzd oq msl CQS ojg xvef oaoq qiaphphzog cd lotfcb tv zqebks eyr ghh oifb xwz yqxyzjcig dwbvxxui
ne gzic yp Entinme. Ve tza opsvuwp je diraznki nce jdmgc czjh, tuzls, guuwb yot aahmkwr hqzgyuipxil. Ai
tbso hh upydvt cl bcbvxblwlk uz frb fw snwunz urd zsmb cbqligsgmb.
bhvexqknkul oMTY
Th kgkhg bv zffwq xerxfon ag pVQX onqcl dfwsw uitgixj mlrgnfnl kurr uqwoe hzwg ozdq myo hrrh ym c rilgirwy
icp gxfo, mtbac nh bxd _zxhivavsqv_ ipfo tcik guhgffz. Jtkft wx z gebvxw swncwg vhl oqikc mfcd jxauoz.
Bd pitf oqvn wces img kgau pz lbguqsf alg ywumieawyf cpma fiyv yl rola zz gqfhde pljl wnv jxh gijquir axi
pisvtn wljcr.
Jbq fcbanat gvfu dy fdnfo qza gNLL litcj wmn xrhe vf zxndyz oen vvu aozc ir ziqnqtktr bd pzwsvhfrw
bfvkeees. Oh mccm, huj yfjyptg fizm tamk ud qchmzoy mk vk e gmay gfzwtc hgf wtg sfbv my rzenmrvgtk
uleylwlk ejvh ytt kyhdwq xiqjhvua, slrzrqnr uqt. Jzb qndv gq zjqz ht xtnk oqtf su dym qxwwzv vz lyvhlym ge
apxkhatsn.
Ounscb rau lwgnoa buhm krw n.on zbhtynv ki cuy ``tjtnjm`` nobajed, nfw uiidrz ic phqym rq
naouygreb meodjj bgeacmk qwbrpsda tuwaachekbkm azpby taim mcvy eux zpf obgvmjul cb Xiefgc. Ko mpeu rpccm
ftq tllq axpgeecug ap roi ``yhem ipfdilw`` efhkew pban acol pefm obycg lkn gjyy wxnqhrsccayz trv aqwfece
dzdg. Kjosoocyw aekog izl zg ueayyvr bihny kj qoh pwd jcqrlrevcgbb fsabtswwcj. Vj waj hwdg uqntfahyr pk
iotn ``fxg`` hiw h ``sxjarwibdotv.udw`` md ilfkrgo ``zwzslq``
Xji imntpb amzt qdu zpr xHMA cfkcfkqkgxl ls gigco vlysry w lwwcgcg jbwwa wpzkl yhhl g eiaqzv odoh ztrljwen
zlv juydd va g tfcpitner mpqsgj lv. Txdkith umg ``ps`` bvvaud ss hpn wkm cm nhqeo ura rxzn ptekwkznmyqbz
go dtb tDVR yzyjx tz ngpkc. Chrt el ht qxhi hs rxqetfol faiy egk rdrijjcmu nc xps infl amar. Mxo ``kwm``
ftloot xbxfpxge r nusirsh aemdjmczsb tipnwhh ruf vbuetvn q pXVD dnlbnhq. Stk ncgbid qcude nk zs fbckmtmo
yfibyl gdqbkzbd cpjphdto. Adlngei ya ynve vbp ``gfk`` cbsjiz cvfbh aqrcyr kgn *jzcbjhrtxoilhdqongip*
myzqxeze enspexe mSST emwqdfxr osqmjkpzkexxvt. Xzdi tb ts wt'l mhx bndqxj xk tocphej na bsii rvc tkvztze
``qni`` qfxn ir jibp dp cxe mrxmcktycvnpq fbjz md ``sc``.
wpfbviywabb Fdqeigupscgait ozf jgudozrveagpi yy sNDS cywgu
Ij ymay jatladj vnwn qfdffd bldwxkvcpv ugy ``hhwzt_nyuoyicc`` gro ``xrxde_ndlqmsrj``. Mrvsp achs kzrd ck
``cxdmat.tt`` eioev akp dj lvk ``uktk`` ahlmug. E qwyvw jh nfixfa ftf yeykil knut nq mhq ``scl`` sqbsvh
spkaxnd kp x uwiima soup oplyovpd.
Xt obaaml iz ainbs ieru auy wboy ykatgey dbewk vhujd vrdb CHP niwatdm fdzq zecq h b.h. td txcgg'p ywnojrrz
cbl yds BK ux zrr. Hued ex xbgexawciym gh fysxcb ekef un weo sqeh aqwvib lsxqpnu htv fslbx ``cbm`` cgf pxa
xya os ybttw cjhtmjst muhs iivg axx ossu lzfod qvv rvnm stvmx.
Blm dujny ezql eoge gbtkxi is p tucrdl mx liva tts snnqpi`rod.nawh.qoqqp_dhgqbejp` rycvddpvy olqctywz qfk
ckfd dz qiyo ykr njj hlkcl. B kmqwto gq kf mjjlbw ndxi fkhx mp gyfyduywm jiml rptufvmf _mem_ rlthcqna.
bnrapfgpcoc YHR Sqqyfzarai
Izsadgkq Xqviw ypbp nw x vuqzjs gd oPTG lhgsvnvy haeakxh sd Iuwroq.
Ezuzzjmaz Ner ``ahi`` oas bx ak xw vvqznfb sup tjknlaiyb oa dm vqnjtvzmuns xdktahe, uvp osp cfm eg n
fuzexne QnCF.
Fabbvojfexyry Ipv xlz cjx lwyrlvx tgdpivn hm k ebeurf tf zbpco vix sqqo tp mq gfsqb axxtllxv nc lxir mg
gqr zcmv o hgpeqhcm lbvlmhr. Tuk ytyzftr omvtugwwjky fd dreejg hic dmpozovmbojh gnssxk. Nz chq vxz jgvyxxf
tfyg wohr zx ``negcod`` tsqej llxcj umpyon xjyxg uktv gle zko ez qpigwer ks ivyq Pldxmo. Pf ootvvd dsugql
rako afewdsubmc zf ftyiazfkyz jaypay nrmt ``shv`` pjvnawee. Ec dnnh whak et to huerjizuk regqnql akuk cqx.
Xg rje pmhe nhdahq gb kywz ao eny chcuyhtpbq hgjc dl oxgbcf dn kwsvers plcy jim ku tr iiwt rya vh wymhino
lq b rvpuwac hsuxghqs xr jhjw qqkb lre Syvtpu reirdavfs.
pcrgmegxncl SztdgeZpqjdciz ADL
Lgakykje Ofvtzxeyhuxfr zt JMZm yg b acccnnw. Bxkza wa ejpfezvoqd jqkosix cxzaylf
FnzrvsRwzhzmau yvr XdqgbtYniv. TqoydaTqyxthno rbd jhbbkuvfyt zsloxyyi zku h kvjaxqybe
bpflqqjs tmyvivem ilkxh vy iffieutpwguua cxkt udyjer ts hpzuma uhmcc fqw oayxwkgl.
Omotlhsvj Cwir tihoechh FjfuidGbrnydci UGC. Gywbtlrj scz hmkgbkfxjhdux uohvokoms wl
pwlqat gosmn, ef wzghgxktg xc otr hlxefz xg wgdvh sivcmw wal LlzoqvEuwimasf frmoh,
jsyxkfjf qgd fulu xiqhnrhcgo incm ygjfhile st qso szlz vd fo jhpxxiwydd PWQ.
Bycxwesvjlmqu Dz hjfxc HJP ma fvdxeckhzs. Pzc fryrrcpsqtqmut ingxxvw sk ksqato n
Cfzq't cxiwwxxs ly hlydv ya rdjetirxpos nmhrjsia.
then the write time when RstFold.vim is enabled is about 5x slower (4 seconds vs 20 seconds).
I'm not necessarily suggesting that we fix this here, but given that there are already three other potential issues with RstFold.vim (two performance issues in vim/vim#3430 and a default folding preference issue #35), maybe we ought to disable this function on default?
Or, more drastically, we could temporarily remove it from the official vim runtime until some of these issues are sorted out.
Also, I don't know how realistic or portable this is, but I remember there was a Python version of this script, which we could revert to if possible. I may have been too cautious when insisting on a vimL version of the function.
I've introduced a flag which disables folding on default in the following branch: foldflag
Would be good to get your feedback before merging, @anntzer
I still cannot reproduce the slow folding issues (as noted previously in https://github.com/vim/vim/issues/3430), but I don't mind making this optional either, up to you.
No worries, I've sent the changes to Bram.
As for reproducing the performance, I think the vim/vim#3430 issue was more subtle since I had to use timers to notice the problem. But I would give the the ctrl-v paste example a try on your machine, this was very noticeable to me.
You might need to use my fold settings to notice the difference:
set foldenable
set foldlevelstart=10
set foldnestmax=10
set foldmethod=indent
With the large restructuredtext.rst example, I can indeed see a slowdown when timing, with the whole load (time vim restructuredtext.rst -c redraw -c q) going from 0.05s to 0.20s when enabling folding. I still can't see any slowdown (after timing) on the scrambled example, though.
I think we're mostly seeing the same thing, which is good news. I think that the issue of restructuredtext.txt is probably sorted out by the patch suggested by @dpelle (which I haven't implemented mainly because I was awaiting feedback from @shabbyrobe). In my case, the performance difference is 0.15s without folding to 0.37s with folding.
I don't see much time difference if I do the following:
time vim problem.rst -c redraw -c q
which seems to agree with your result. Is this what you tried? (I get maybe about ~0.12 without folding and 0.14s with folding if I take the minimum results, but the noise certainly overlaps between the two cases)
Have you tried explicitly copying the text from problem.rst into a buffer, and then pasting into a live vim session? This is where I see the very large performance difference that I described above, as it seems to be constantly rendering things as the text is gradually copied into vim.
I'm not sure how to script it (and my build has -clipboard so may not even be possible) but I'll try to post something if I can.
With problem.rst I get between 0.03 and 0.05s regardless of whether folding is enabled or not, and it's totally overwhelmed by noise. I really can't see any difference when pasting problem.rst into a live vim (that's what I had always tried before), it folds basically instantly (well, in less than 50ms I guess).
OK, then I am not sure... I can reproduce this with a blank .vimrc file, so I don't think it's anything that I'm causing.
I will try some more explicit profiling and see what is coming up. Sorry to put you through all these questions.
OK slight development here. It seems that paste is very fast (instantaneous?) in gVim, and I only see this inside my terminal. Are you using gVim?
I've tested it now in termite, urxvt, and gvim, and I get the slow clipboard paste in urxvt and termite.
I was using gvim, but I just tried using terminal vim (konsole and urxvt) and still can't repro...
Alright, I wil try in a KDE environment and see if I get a different result.
How one makes this all working. I thought that adding let g:rst_fold_enabled=1 to ~/.vim/vimrc should set up everything, but even when I have it, when I open *.rst file I still get foldclose= foldcolumn=0 foldenable foldexpr=0 foldignore=# foldlevel=999 foldlevelstart=-1 foldmethod=manual foldminlines=1 foldnestmax=20 formatexpr=.
It looks like the setting is rst_fold_enabled, not rst_fold_enable. I think the flag name must have changed some time after this conversation but before it had been added, sorry about that.
Sorry, I had let g:rst_fold_enabled=1 already (I copied it from the code of the ftplugin). It was just a typo in my comment. Still nothing. My $VIMRC is https://gitlab.com/mcepl/vimdir/blob/master/vimrc
Sorry that I haven't replied yet. For now, all I can say is that it does work for me when the flag is enabled, with the caveat that initially the folds are all opened and need to be manually closed.
But I will have a look at your .vimrc a bit later and try to see why it might not be working for you.