underscore icon indicating copy to clipboard operation
underscore copied to clipboard

_.union doesn't work with arrays of objects

Open wilhen01 opened this issue 10 years ago • 32 comments

_.union will always produce duplicates when passed arrays of objects.

e.g. _.union( [ { a:1 } ], [ { a:1 } ]) will return [ { a:1 }, { a:1 } ]

Perversely, underscore's own isEqual function will tell you that the objects in question are equal. Maybe we could have a flag/option which dictates the equality comparison to use, or the option to pass in a comparator?

wilhen01 avatar Oct 02 '15 13:10 wilhen01

I'm surprised it doesn't already accept the comparison function. :+1:

michaelficarra avatar Oct 02 '15 14:10 michaelficarra

It's worth pointing out that this applies to all the other array computation functions like difference, intersection, unique, etc.

Would be nice if the full suite of array functions could be updated to allow use of a different equality comparator for objects.

wilhen01 avatar Oct 02 '15 15:10 wilhen01

Wouldn't it be easier if we would have an option that compares equality based on boolean parameter value that could be passed to _.union() function? If it is true then it would automatically compare all objects in that array.

E.g. _.union([1, 2, 3, 10, [{a:1}, {a:1}]], true), would output [1,2,3,10, {a:1}]

ghost avatar Oct 12 '15 07:10 ghost

@amiral84 No. That is unrelated. If you want that behaviour, compose union with flatten.

michaelficarra avatar Oct 12 '15 13:10 michaelficarra

@michaelficarra Then did I miss the point of this topic? :D

ghost avatar Oct 12 '15 14:10 ghost

@amiral84 It appears so. The feature request is explained fully and succinctly in the first comment.

michaelficarra avatar Oct 12 '15 14:10 michaelficarra

the underlying problem seems to be in _.uniq as _.union is merely a wrapper function for unique and flatten.

_.union = restArgs(function(arrays) {
  return _.uniq(flatten(arrays, true, true));
});

dperrymorrow avatar Nov 25 '15 20:11 dperrymorrow

This thread inspired me to add _.intersectionWith, _.differenceWith, _.unionWith, and _.uniqWith to handle comparison customization in my own code.

var array = [ { 'a': 1, 'b': 2 }, { 'a': 1, 'b': 3 }, { 'a': 1, 'b': 2 } ];

_.uniqWith(array, _.isEqual);
// => [{ 'a': 1, 'b': 2 }, { 'a': 1, 'b': 3 }]

jdalton avatar Nov 25 '15 21:11 jdalton

or something along the lines of _.isCollection to determine if you are dealing with a collection. If dealing with a collection, then the comparisons should use _.isEqual instead of === which does no good in the case of a collection.

dperrymorrow avatar Nov 25 '15 23:11 dperrymorrow

@dperrymorrow

should use _.isEqual instead of === which does no good in the case of a collection.

Dynamic switching sounds like a bad idea. JS uses === or SameValueZero comparisons for many things. If there's a need to go outside those comparisons something like _.uniqWith would do.

jdalton avatar Nov 25 '15 23:11 jdalton

Thanks for this @jdalton, the _opWith functions you mention are absolutely perfect for what I'm trying to achieve. Any idea when they'll be available via release?

wilhen01 avatar Nov 26 '15 11:11 wilhen01

@jdalton good point on the comparisons, but wouldn't you usually use a key for unique on a collection rather than forcing Underscore to detect the entire difference between the objects?

Wouldn't the following solve @wilhen01 's request (albeit more verbose than desired)

_.chain([{ a: 1 }]).union( [{a: 1}]).unique('a').value();
//=> [{a: 1}]

dperrymorrow avatar Nov 30 '15 17:11 dperrymorrow

Wouldn't the following solve @wilhen01 's request (albeit more verbose than desired)

_.uniq already supports that.

jdalton avatar Nov 30 '15 17:11 jdalton

right, thats my point, the above code works currently as posted. could't you just call uniq / unique with a key on the result of the union?

dperrymorrow avatar Nov 30 '15 18:11 dperrymorrow

@dperrymorrow Think just a tiny bit outside that example and add another property.

jdalton avatar Nov 30 '15 18:11 jdalton

ok, gotcha, sorry... Im not trying to be belligerent, just wanted to totally understand the problem. Id love to submit a pull request on the _.uniqWith function.

dperrymorrow avatar Nov 30 '15 18:11 dperrymorrow

No worries, that would be rad.

jdalton avatar Nov 30 '15 18:11 jdalton

_.intersectionWith, _.differenceWith, _.unionWith, and _.uniqWith

Wouldn't it be a nicer API to just allow the comparison function to be optionally passed as the final argument, instead of minting four new functions?

jashkenas avatar Dec 01 '15 00:12 jashkenas

@jashkenas

Wouldn't it be a nicer API to just allow the comparison function to be optionally passed as the final argument, instead of minting four new functions?

Yes, it could be done but there's complications because methods like _.uniq already support passing an iteratee and are heavily overloaded with support for binary/sorted search flags and context params too . This would mean introducing arity sniffing which feels too clever for this situation. This would also complicate future modularization efforts because it bundles lots of optional functionality into a single point when the implementations could be simplified and split into separate methods.

jdalton avatar Dec 01 '15 00:12 jdalton

Right, a totally unfortunate design issue. But making new functions just to allow a comparator doesn't feel like the right solution either.

jashkenas avatar Dec 01 '15 00:12 jashkenas

ok so additional comparison function parameters are the way to go here then? If so I can update my pull request.

the only tricky part I forsee is making the parameter parsing a little more hairy as @jdalton mentioned above.

_.uniq = _.unique = function(array, isSorted, iteratee, context) {
  if (!_.isBoolean(isSorted)) {
    context = iteratee;
    iteratee = isSorted;
    isSorted = false;
  }
//...

Perhaps adding an additional check to see if isSorted _.isFunction then treat it as the comparator.

dperrymorrow avatar Dec 01 '15 01:12 dperrymorrow

@jashkenas

But making new functions just to allow a comparator doesn't feel like the right solution either.

It may be the best option for a bad situation. I've gone on a kick of splitting out overloaded functionality recently and have been pretty happy with the result. Though it increases the API surface it allows for simpler implementations and grouping of similarly themed methods like maxBy, uniqBy, pickBy, or uniqWith, unionWith, zipWith, or sortedIndexBy, sortedIndexOf, sortedUniq. In the case of uniq though I still use a shared base function at the moment.

jdalton avatar Dec 01 '15 01:12 jdalton

have updated this pull request #2368 thanks.

dperrymorrow avatar Dec 01 '15 19:12 dperrymorrow

I'm :+1: for uniqBy or uniqWith. I would be completely against overloading uniq further (as #2368 currently is proposed)

megawac avatar Dec 01 '15 19:12 megawac

:+1: @megawac, uniqBy.

michaelficarra avatar Dec 01 '15 22:12 michaelficarra

Fwiw lodash will be using uniqBy as the split out form of _.uniq(array, iteratee) and _.uniqWith as the form to allow comparator customization.

jdalton avatar Dec 01 '15 22:12 jdalton

Yeah, on second thought uniqWith is a better name

megawac avatar Dec 02 '15 03:12 megawac

should I pull request on Lodash with the separate method then? I thought the two projects merged, am i mistaken?

dperrymorrow avatar Dec 02 '15 18:12 dperrymorrow

@dperrymorrow

should I pull request on Lodash with the separate method then

No need, they're already in lodash's edge master branch.

I thought the two projects merged, am i mistaken?

Not yet. Lodash v4 proofs out some of the ideas of the merge though.

jdalton avatar Dec 02 '15 18:12 jdalton

@jdalton Can you please elaborate a liitle more on implementation of _.uniqWith with other iteratee.

ghost avatar Jan 10 '16 19:01 ghost

@Pavnii Sure. You can check out lodash/npm/_baseUniq. If a comparator is passed it uses arrayIncludesWith helper to do the check instead of arrayIncludes (Underscore's contains).

jdalton avatar Feb 10 '16 16:02 jdalton

@jdalton That helps me.

ghost avatar Feb 12 '16 05:02 ghost