procrastinate icon indicating copy to clipboard operation
procrastinate copied to clipboard

Support for asyncpg

Open davidolrik opened this issue 3 years ago • 2 comments

Would it be feasible to add support for asyncpg? (Maybe even replace aiopg)

procrastinate is written specifically for running on postgres, and so is asyncpg, which is why it is extremely performant. asyncpg is way faster than aiopg, and it is even faster than golangs pgx.

Furthermore, it is supported by encode/databases which plays very well with FastAPI, SQLAlchemy Core and ormar.

davidolrik avatar Jul 09 '21 16:07 davidolrik

Yes, we've considered it. We could start by adding support for asyncpg as opposed to replacing aiopg. PRs are very welcome!

elemoine avatar Jul 09 '21 18:07 elemoine

One thing I see that might make things a bit more complicated is that the format for query parameters is different in aiopg (%(name)s) and asyncpg ($n).

That being said, I believe we should be able to automate the translation between the 2 formats:

In [3]: class Converter:
   ...:     def __init__(self):
   ...:         self.order = []
   ...:
   ...:     def __getitem__(self, key):
   ...:         self.order.append(key)
   ...:         return f"${len(self.order)}"
   ...:
   ...:     def convert_args(self, args):
   ...:         return tuple(args[e] for e in self.order)
   ...:

In [4]: c = Converter()

In [5]: s = "%(abc)s yay %(def)s" % c

In [6]: s
Out[6]: '$1 yay $2'

In [8]: c.convert_args({"def": "bar", "abc": "foo"})
Out[8]: ('foo', 'bar')

(This is just a POC code, we should be able to make something cleaner, but at least we can be reasonably sure that it's feasible)

ewjoachim avatar Jul 11 '21 17:07 ewjoachim