MonetDBLite-Python icon indicating copy to clipboard operation
MonetDBLite-Python copied to clipboard

Add support for converting numpy integers.

Open tandreas opened this issue 8 years ago • 4 comments

Currently, reflecting MetaData using sqlalchemy-monetdb with monetdblite doesn't work.

For example:

from sqlalchemy import create_engine, MetaData, Table

engine = create_engine('monetdb+lite:///:memory', echo=True)
metadata = MetaData(bind=engine)
query = """
CREATE TABLE "test" (
    id int,
    data varchar(128)
)
"""
engine.execute(query)
metadata = MetaData(bind=engine)
tbl = Table('test', metadata, autoload=True)

This happens because when sqlalchemy-monetdb tries to round trip integer ids, they are coming back from monetdblite as numpy.int32, and this isn't supported by monetize.convert.

This adds support for that. I think the fix should be in monetdblite rather than sqlalchemy-monetdb, a user will expect that data types retrieved from the driver should be able to round trip back in a query.

tandreas avatar Oct 23 '17 21:10 tandreas

@Mytherin & @gijzelaerr any comments on this?

hannes avatar Oct 24 '17 10:10 hannes

Not sure if this is the correct way to fix this problem. Assuming the SQLAlchemy driver is only using the regular Python DB interface, I don't think MonetDBLite should be returning numpy integers at all. I haven't looked at the SQLAlchemy MonetDBLite driver, though, so I don't know what is exactly happening there.

Mytherin avatar Oct 24 '17 11:10 Mytherin

I actually don't think this is a bad idea. This way numpy ints are rendered properly inside strings. Probably a good idea to add the other numpy types (like float) also. SQLAlchemy is using the MonetDBLite API, which internally constructs numpy arrays for result sets which contain numpy ints. I guess this is not a problem, as long as numpy ints evaluate equal to python ints.

I'm not a big fun of try catching the numpy import. If you leave it out the interpreter will gave a similar error, which is quite obvious.

gijzelaerr avatar Oct 25 '17 07:10 gijzelaerr

this is also a problem with booleans, which are returned as 0 and 1 type uint8. this is a bit more worrying.

gijzelaerr avatar Oct 25 '17 13:10 gijzelaerr