mongodb-schema icon indicating copy to clipboard operation
mongodb-schema copied to clipboard

Add the ability to limit the number of array lengths collected

Open btiernay opened this issue 3 years ago • 0 comments

I believe this is one area of the tool that can cause heap exhaustion when profiling over a large number of documents. In stream.js's addToType:

    // recurse into arrays by calling `addToType` for each element
    if (typeName === 'Array') {
      type.types = type.types || {};
      type.lengths = type.lengths || [];
      type.lengths.push(value.length); // <-- Grows without bound
      value.forEach(v => addToType(path, v, type.types));

It would be useful to have an option that would skip this, use a reservoir, or somehow cap the collection of lengths.

btiernay avatar Jul 25 '22 16:07 btiernay