red-bird icon indicating copy to clipboard operation
red-bird copied to clipboard

Feature request: add_many method

Open sanzoghenzo opened this issue 2 years ago • 2 comments

First of all, kudos for making this library! I'm evaluating the usages in one of my project, it can use a better separation between the data layer and the logic!

I think that one thing that is missing is the add_many method to leverage the (often more performant) backend methods.

The base implementation can be simply a series of calls to the add method.

sanzoghenzo avatar Oct 18 '22 10:10 sanzoghenzo

Thanks, glad you find it interesting! Sometimes I think things are more complex than I would like them to be. If I want this thing in my database, I don't know why I would need to create complex ORM objects or write insertion queries just for that. Sort of highlights the philosophy of actually most of my open-source projects.

But to the topic, I think there could be as it's common across almost every datastore, even in a CSV file (you just append the file without closing in-between). Not sure what would be the most Pythonic:

repo.add_many([Item(...), Item(...)])
repo.add(Item(...), Item(...))

Not sure which I prefer more. I think add_more is more obvious with what it does but add is easy to remember and still quite obvious. Opinions?

I have also been thinking of adding some sort of sessions that are in every repo. Actually this is already there but there is no formal support for transactions. Perhaps at some point you could create transactions yourself and insert them however the repo does that, like:

with repo:
    # In a transaction. When exited, things are commited
    repo.add(Item(...))
    repo.add(Item(...))

However, this would require quite a bit of work. We probably need some sort of operation objects as Python lists/CSV files/etc. don't have built-in transaction mechanisms obviously, so it requires some amount of work.

Miksus avatar Oct 18 '22 20:10 Miksus

I don't know if it's more pythonic, but I like the add_many method (with a single iterable argument) more.

To quote the zen of python:

  • Explicit is better than implicit. (I feel that *args is not that explicit)
  • Simple is better than complex. (no need to remember to unpack the generator/list, just pass it)
  • There should be one-- and preferably only one --obvious way to do it. (well, it depends on the definition of it: is it "add many items" and "add one item" or "add any number of items"?)

Looking at other libraries, it seems the consensus to have a separate method:

  • SQlAlchemy uses Session.add_all(Iterable[Item])
  • (py)Mongo uses Collection.insert_many(Iterable[Item])

I think it also helps write less code (no need to check *args to choose to use collection.insert_one or collection.insert_many)

As for the transactions: I'm focusing on a project that doesn't leverage mongo transactions (I started the project before transactions in mongodb were a thing), so I cannot speak of it, but it would be quite handy!

sanzoghenzo avatar Oct 19 '22 10:10 sanzoghenzo