mvtools icon indicating copy to clipboard operation
mvtools copied to clipboard

Assert in MDegrainN line 224

Open DTL2020 opened this issue 3 years ago • 2 comments

I frequently get assert debug break in MDegrainN: line 224: assert(!(blockWidth % 8) != 0 && blockWidth != 4);

Today it looks I found the way to reproduce: With 1920x1080 (YV12) source: tr = 6 # Temporal radius super = MSuper (mt=false, chroma=true, pel=1) multi_vec = MAnalyse (super, search=3, searchparam=2, multi=true, delta=tr, chroma=false,mt=false, optSearchOption=0, optPredictorType=0, levels=1) MDegrainN (super, multi_vec, tr, thSAD=400, thSAD2=150, mt=false)

It happens when pel=1 in super, not happens when pel=default. If disable this assert looks nothing bad happens, but may be it is not good ?

DTL2020 avatar Oct 23 '21 14:10 DTL2020

Thanks for the report. Assert was misleading indeed. Reason: bad parenthesis. But I noticed that for blkwidth==4 we'd better read only 4 bytes instead of 8. Then I extended the sse2 support for mod4 as well. That means that now blkwidth=12 is supported as well, as the remaining non mod 8 (mod 4 only) case.

pinterf avatar Oct 23 '21 16:10 pinterf

Theoretically, when we were reading from the very-very last block at blkwidth=4, it could happen that we were reading past the buffer. Because _mm_loadl_epi64 was used even for a four-byte read.

pinterf avatar Oct 23 '21 16:10 pinterf

Fixed in 2c30382aaea57f4224553359debe475b1b2e045c

pinterf avatar May 03 '24 10:05 pinterf