'to be an array whose items satisfy' slowness and verbosity

[msiebuhr]

I currently move some bulk data between various databases with some processing on the side. For testing a do a small-ish query that results in a few thousand elements and verify the looks OK. Paraphrasing:

x = [];
while (x.length < 1000) { x.push({foo: "bar"}); }
x[100].foo = 'baz';

expect(x, 'to be an array whose items satisfy', function (i) {
    expect(i, 'to satisfy', { foo: "bar" });
});

When it fails, it takes a few seconds to execute and print the output, pretty-printing the entire dataset to the console. (Which, given moderately complex objects, can be quite difficult to devour.)

I've used two strategies to avoid this so far:

• Doing a canary test on the first element, to speed up the case where I fail at everything.
• Skipping the outer expect and forEach'ing over the elements

Given that passing runs aren't noticeably slow, it has to be generation of the output that causes things to slow down.

I propose two strategies for dealing with this:

• Stop printing after first N failures (in particular, when everything fails).
• Only print a few successful cases before and after each failing element, omitting the large majority of correct output.

[sunesimonsen]

This is a very good point. I haven't used Unexpected for a lot of integration testing so I haven't been running into these limitations. Unexpected should of cause handle this situation well.

I think we need to batch the checks up into chunks otherwise we wont be able bail out. I think it makes sense to only show 10 or 20 errors and then just state that there are more errors. I also like the idea of not showing all the items that passed.

[sunesimonsen]

For the needle in the haystack problem, what about something like this:

expected array to have items satisfying to be a number

[
  0,
  // indexes from 1 to 1251 collapsed
  1252,
  '1253', // to be a number
  1254,
  // indexes from 1255 to 1499 collapsed
  1500
]

[sunesimonsen]

@papandreou do you have any opinions on this subject.

[papandreou]

Those are fine ideas! Would be nice to derive some principles and apply to other assertions, especially to satisfy.

[msiebuhr]

I don't really think this is applicable to to satisfy; those are typically hand-written and not overly large (at least that's what I do). The array/object assertions allow to automate detailed checks on extremely large data-structures, making the output the more overwhelming...