orderedmultidict icon indicating copy to clipboard operation
orderedmultidict copied to clipboard

add magic methods as seen in set() and elsewhere

Open isaacimholt opened this issue 6 years ago • 6 comments

Add functionality such that >>> omdict(a=1, b=2) |= {'a': 'hello world'} fully merges/updates (add all members) the omdict >>> omdict([('a', 1), ('b', 2), ('a', 'hello world')]) and similar functionality for | but returns a new object instead.

I suggest also then to use >>> omdict(a=1, b=2) += {'a': 'hello world'} to produce a more "regular" update >>> omdict([('b', 2), ('a', 'hello world')]) and once again + produces a new object instead

newer versions of python will have dictionary insertion order as default, but this is still useful in the case that order does not particularly matter, but then if it does these operations should obviously also work with OrderedDict and other omdict instances to preserve order.

isaacimholt avatar Feb 21 '19 21:02 isaacimholt

Shall I submit a pr? Is this feature in line with the project's goals?

isaacimholt avatar Apr 15 '19 07:04 isaacimholt

Thank you for the bump. I appreciate it.

Shall I submit a pr? Is this feature in line with the project's goals?

That'd be awesome. Binary operators like | and + would be a great addition to orderedmultidict.

Before you wrangle your spade and break ground: to be explicit, which operators will you add support for?

  • + (via __add__()): Binary add with key replacement.
  • | (via __or__()): Binary union with key addendum, not key replacement.

Any others?

gruns avatar Apr 16 '19 20:04 gruns

Yes I was thinking of what else could be added. I was basing myself largely on existing operators for sets (https://docs.python.org/2/library/sets.html#set-objects), but the difficulty was merging the ideas of set operators with dictionaries, but I suppose there could be a good mental model: consider k:v pairs to be single items that could be added/removed from omdict. Therefore, briefly:

  • del omdict["a"] obviously removes the "a" key and all its values (existing behavior)
  • however omdict -= {"a": "my value"} removes only "my value" if it exists for key "a"

Other operators could be added, but it's not clear to me how useful they might be:

  • omdict ^= {"a": "my value"} return omdict with k:v elements from omdict or other dictionary but not both
  • omdict &= {"a": "my value"} return omdict keeping only k:v elements found in both omdict and other dictionary
  • In both previous cases, the order of the first object is canonical. any new elements are added to the end of the first one, in whatever order they appear from the second one.

The interesting thing here is that since dictionaries are limited to unique keys, if you wanted to remove/update/etc multiple values for the same key, you could do omdict -= {"a": "my value"} -= {"a": "my other value"}, or use any other multidict implementation (or another omdict obviously) omdict -= omdict(("a", "my value"), ("a", "my other value")) as the implementation should simply iterate over the k:v pairs of the other object and work.

Also considering subset/superset semantics (<=, >=) but not sure if this would be correct. Should order be considered when comparing?

Another possible point for discussion is that set operators are useful for sets... which have no order. Are there other possible operators that could be added that would do useful things considering omdict ordering? What other "rich" objects exist in python-land that have order-aware operators? I prefer to avoid creating new semantics for these operators and instead re-use existing conventions, and only break from them where it makes sense for this use-case.

To be explicit here are the methods to be added:

  • object.__add__(self, other)
  • object.__sub__(self, other)
  • object.__xor__(self, other)
  • object.__or__(self, other)
  • object.__and__(self, other)
  • object.__iadd__(self, other)
  • object.__isub__(self, other)
  • object.__ixor__(self, other)
  • object.__ior__(self, other)
  • object.__iand__(self, other)

possibly add "regular" methods?

  • object.add(other)
  • object.union(other)
  • object.intersection(other)
  • object.difference(other)
  • object.symmetric_difference(other)
  • object.update(other)
  • object.intersection_update(other)
  • object.difference_update(other)
  • object.symmetric_difference_update(other)

possibly right-hand operators?

  • object.__radd__(self, other)
  • object.__rsub__(self, other)
  • object.__rxor__(self, other)
  • object.__ror__(self, other)
  • object.__rand__(self, other)

I expect that some of these (add, update) exist already and should be updated to support this new functionality. As a first implementation I would limit myself to the 1st group of methods e.g. object.__add__(self, other) and object.__iadd__(self, other).

isaacimholt avatar Apr 17 '19 08:04 isaacimholt

By the way, I realized afterwards that most of the previous post is incorrect, since obviously you can have multiple duplicate values per key. |= and += are still valuable shortcuts to have however, and -= would be useful as well. Following the principle of least surprise, it should remove a single k:v:

omdict((1, 'a'), (1, 'a), (1, 'b')) - {1: 'a'} == omdict((1, 'a'), (1, 'b')) omdict((1, 'a'), (1, 'a), (1, 'b')) - omdict((1, 'a'), (1, 'a)) == omdict((1, 'b'),)

however the order is not defined at the moment, I will try to stick to whatever default order is used in the project (remove first matching element or last).

isaacimholt avatar Apr 23 '19 08:04 isaacimholt

however the order is not defined at the moment, I will try to stick to whatever default order is used in the project (remove first matching element or last).

By default, the last item is removed. A la popitem()

popitem(fromall=False, last=True)

The other big question is whether to remove all matching items or just the last one. Which do you think is the more expected behavior? I'm torn.

gruns avatar Jul 11 '19 18:07 gruns

The other big question is whether to remove all matching items or just the last one. Which do you think is the more expected behavior? I'm torn.

I really don't know, I would do whatever might be more useful for url manipulation purposes, but it's been a while since I've worked with them. Rethinking things, I might suggest the removal of all matching items. I may not know how many items of that type exist, but I may know that I only want 1 to be present, so I might do this: omdict((1, 'a'), (1, 'a'), (1, 'a'), ..., (1, 'b')) - {1: 'a'} + {1: 'a'} == omdict((1, 'a'), (1, 'b')) but I am having difficulty imagining a concrete usage scenario for one implementation over the other.

isaacimholt avatar Jul 16 '19 08:07 isaacimholt