deep-object-diff icon indicating copy to clipboard operation
deep-object-diff copied to clipboard

Comparing large arrays can take forever

Open Kagan-Of-Sky opened this issue 5 years ago • 2 comments

To reproduce:

`const deepObjectDiff = require("deep-object-diff");

let a = new Array(10000);

for(let i=0; i<a.length; ++i){ a[i] = { "attribute 1": i, "attribute 2": i, "attribute 3": i }; }

let b = JSON.parse(JSON.stringify(a)); b.unshift({ "a": 9999999 });

let start = new Date().getTime();

let diff = deepObjectDiff.detailedDiff(a,b);

let end = new Date().getTime(); console.log("Took " + (end - start) + " milliseconds.")`

For an array of size 10000, it can take 40+ seconds. 20000 = about 250 seconds 25000 = about 500 seconds If the array is very large, 100000+ , it can take a really long time.

Kagan-Of-Sky avatar Aug 21 '20 14:08 Kagan-Of-Sky

What is the problem here? Bigger inputs take more time to process, and detailedDiff is the most expensive type of diff operation. Such an operation taking a while to process in JavaScript seems like expected behaviour.

anko avatar Aug 21 '20 15:08 anko

Fair enough. It does seem like a weakness in the algorithm though. 8+ minutes to process an array of 25000 simple objects seems like a bit much. I would expect such an operation to take maybe a minute max and not scale exponentially - then again I am not at all familiar with the internals of doing a detailed diff.

Kagan-Of-Sky avatar Aug 21 '20 16:08 Kagan-Of-Sky