Can't automatically convert KeyedTuple (from sqlalchemy) to Julia's NamedTuple

[manuelma]

I use PyCall to call sqlalchemy from Julia, which uses a class called KeyedTuple to represent rows resulting from a query and I want to convert those automatically to julia's NamedTuple. I expected something like this to work:

using PyCall

sa = pyimport("sqlalchemy")

pytype_mapping(sa."util"."KeyedTuple", NamedTuple)

Base.convert(::Type{NamedTuple}, o::PyObject) = NamedTuple{Symbol.(o._fields)}([getproperty(o, f) for f in Symbol.(o._fields)])

x = sa.util.KeyedTuple([1, 2, 3], labels=["one", "two", "three"]) # returns (1, 2, 3)

This is no surprise, since:

o = @pycall sa.util.KeyedTuple([1, 2, 3], labels=["one", "two", "three"])::PyObject

pytype_query(o)  # Returns `Tuple{Integer,Integer,Integer}`

I guess a simple way to make my thing work would be to alter the order in which pytype_query does things> I mean first look for a match in the pytype_queries array and then proceed with the rest...?

[stevengj]

In general, we're trying to move away from more automatic conversions and just make it easier to use the PyObject as-is.

On the other hand, explicitly calling NamedTuple(o) or convert(NamedTuple, o) could and should be made to work.

[manuelma]

Ok, cool, thanks for the reply. I guess I can disable the automatic conversion for my KeyedTuple (I believe the correct way to do this is through @pycall function_call::PyObject) and then treat it on my own.

[manuelma]

I guess what's not so nice is that one explicitly needs to disable the automatic conversion for the KeyedTuple otherwise it just lands as a (non-named) tuple in Julia. One could even argue that this is an error since the automatic conversion is losing information. In the same line of reasoning, one could say that PyCall fails at determining that this KeyedTuple is more than just a tuple.

I don't know why this happens, clearly it has something to do with sqlalachemy's KeyedTuple implementation which makes pyisinstance(o, @pyglobalobj :PyTuple_Type) evaluate to true in PyCall.pysequence_query. Maybe I can try and take a look at the details.

[manuelma]

There's not much we can do actually:

>>> import sqlalchemy as sa
>>> issubclass(sa.util.KeyedTuple, tuple)
True

[marius311]

I guess a simple way to make my thing work would be to alter the order in which pytype_query does things> I mean first look for a match in the pytype_queries array and then proceed with the rest...?

Came here trying to do the identical thing, mainly define a custom conversion for an object which gets captured by pysequence_query first, so its currently impossible.

Even if in the future we move away from auto conversions, as a stop-gap, can we just move pytype_queries to the front of the check list? Should I submit a PR?

[baggepinnen]

I think it would be much better to default to returning a PyObject rather than converting the python namedtuple into a julia tuple. The python code

nt = fun1()
fun2(nt)

can not be replaced with the julia version

nt = py.fun1()
py.fun2(nt)

since the named tuple nt has been butchered by the automatic conversion.