Some specific .less lines take very long to check

[jonim8or]

This is probably not a bug but an inconvenient result of the hashing algorithm of the rabin-karp algorithm. I had twelve lines in one .less file slow down my checks from 4.5 seconds to 250 seconds. Solved it by adding this specific .less file to the ignore list. These lines were the culprit for me:

@1dp_elevation: 0px 1px 1px 0px rgba(0, 0, 0, .14), 0px 2px 1px -1px rgba(0, 0, 0, .12), 0px 1px 3px 0px rgba(0, 0, 0, .2);
@2dp_elevation: 0px 2px 2px 0px rgba(0, 0, 0, .14), 0px 3px 1px -2px rgba(0, 0, 0, .12), 0px 1px 5px 0px rgba(0, 0, 0, .2);
@3dp_elevation: 0px 3px 4px 0px rgba(0, 0, 0, .14), 0px 3px 3px -2px rgba(0, 0, 0, .12), 0px 1px 8px 0px rgba(0, 0, 0, .2);
@4dp_elevation: 0px 4px 5px 0px rgba(0, 0, 0, .14), 0px 1px 10px 0px rgba(0, 0, 0, .12), 0px 2px 4px -1px rgba(0, 0, 0, .2);
@6dp_elevation: 0px 6px 10px 0px rgba(0, 0, 0, .14), 0px 1px 18px 0px rgba(0, 0, 0, .12), 0px 2px 4px -1px rgba(0, 0, 0, .2);
@8dp_elevation: 0px 8px 10px 0px rgba(0, 0, 0, .14), 0px 3px 14px 2px rgba(0, 0, 0, .12), 0px 5px 5px -3px rgba(0, 0, 0, .2);
@9dp_elevation: 0px 9px 12px 1px rgba(0, 0, 0, .14), 0px 3px 16px 2px rgba(0, 0, 0, .12), 0px 5px 6px -3px rgba(0, 0, 0, .2);
@12dp_elevation: 0px 12px 17px 2px rgba(0, 0, 0, .14), 0px 5px 22px 4px rgba(0, 0, 0, .12), 0px 7px 8px -4px rgba(0, 0, 0, .2);
@16dp_elevation: 0px 16px 24px 2px rgba(0, 0, 0, .14), 0px 6px 30px 5px rgba(0, 0, 0, .12), 0px 8px 10px -5px rgba(0, 0, 0, .2);
@24dp_elevation: 0px 24px 38px 3px rgba(0, 0, 0, .14), 0px 9px 46px 8px rgba(0, 0, 0, .12), 0px 11px 15px -7px rgba(0, 0, 0, .2);

[kucherenko]

Is the issue related to #411 ?

As I can see tokenisation process was long here

[jonim8or]

The issue is not related to 411. The very long execution time I mentioned in 411 and later crossed out again, turned out to be caused by this less file. So now (after I crossed it out) 412 is about the extremely long execution time, and 411 is about the useless extra time after running the script (that can be avoided by the process.exit(0) trick

I added this less file to the ignore list, so that it is not checked. Now the detection for my project takes about 20 seconds, and if I don't do the process.exit(0) trick mentioned in 411, there is an additional "useless time" at the end my script, of about 1 min 45. From that I draw the conclusion that 411 and 412 are not related

[hilongjw]

I'm also having this problem, a 200 line less file is fully reproducible, this issue looks like a reprism problem?

https://github.com/kucherenko/jscpd/blob/master/packages/tokenizer/src/tokenize.ts#L38

[hilongjw]

Case 1

index.less

time jscpd
14.49s user 0.33s system 98% cpu 15.009 total

source:

.item-order {
    background: -webkit-gradient(linear, left top, left bottom, from(#ffffff), color-stop(0.5, #ffffff), color-stop(0.5, transparent)) top left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#ffffff), to(#ffffff)) bottom left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#e5e5e5), color-stop(0.5, #e5e5e5), color-stop(0.5, transparent)) top left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#e5e5e5), to(#e5e5e5)) bottom left no-repeat,
                #f5f5f9;
}

Case 2

index.less

time jscpd
38.78s user 0.31s system 99% cpu 39.312 total

source:

.item-order {
    background: -webkit-gradient(linear, left top, left bottom, from(#ffffff), color-stop(0.5, #ffffff), color-stop(0.5, transparent)) top left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#ffffff), to(#ffffff)) bottom left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#e5e5e5), color-stop(0.5, #e5e5e5), color-stop(0.5, transparent)) top left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#e5e5e5), to(#e5e5e5)) bottom left no-repeat,
                #f5f5f9;
}
.item-order {
    background: -webkit-gradient(linear, left top, left bottom, from(#ffffff), color-stop(0.5, #ffffff), color-stop(0.5, transparent)) top left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#ffffff), to(#ffffff)) bottom left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#e5e5e5), color-stop(0.5, #e5e5e5), color-stop(0.5, transparent)) top left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#e5e5e5), to(#e5e5e5)) bottom left no-repeat,
                #f5f5f9;
}
.item-order {
    background: -webkit-gradient(linear, left top, left bottom, from(#ffffff), color-stop(0.5, #ffffff), color-stop(0.5, transparent)) top left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#ffffff), to(#ffffff)) bottom left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#e5e5e5), color-stop(0.5, #e5e5e5), color-stop(0.5, transparent)) top left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#e5e5e5), to(#e5e5e5)) bottom left no-repeat,
                #f5f5f9;
}

Case 3

index.css

time jscpd
1.15s user 0.22s system 107% cpu 1.284 total

source:

.item-order {
    background: -webkit-gradient(linear, left top, left bottom, from(#ffffff), color-stop(0.5, #ffffff), color-stop(0.5, transparent)) top left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#ffffff), to(#ffffff)) bottom left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#e5e5e5), color-stop(0.5, #e5e5e5), color-stop(0.5, transparent)) top left no-repeat,
                -webkit-gradient(linear, left top, left bottom, from(#e5e5e5), to(#e5e5e5)) bottom left no-repeat,
                #f5f5f9;
}

[Kurt-von-Laven]

There is already a maxLines option, so I wonder if a maxCols option would help in cases like this.

[kucherenko]

We can use maxTokens option for change detection quality and speed

[Kurt-von-Laven]

Makes sense to me since you already have minTokens.

[NieZhuZhu]

We can use maxTokens option for change detection quality and speed

I could not find the maxTokens option, is this option currently supported?

[Kurt-von-Laven]

No; it has been proposed but not implemented.