heavydb
heavydb copied to clipboard
`ArrowForeignStorage` does not support importing `date32[day]` type
Attempt to import Arrow table containing a column with date32
type results into a RuntimeError
thrown from ArrowForeignStorage::getOmnisciType
:
https://github.com/omnisci/omniscidb/blob/416df923adebc6d2db1372986d961b563f94d7b7/DataMgr/ForeignStorage/ArrowForeignStorage.cpp#L817-L818
Reproducer with pydbe
import sys
sys.setdlopenflags(1 | 256) # RTLD_LAZY+RTLD_GLOBAL
from omniscidbe import PyDbEngine
import pyarrow as pa
at = pa.Table.from_pydict({"a": [1, 2, 3, 4]}, schema=pa.schema({"a": pa.date32()}))
server = PyDbEngine()
server.importArrowTable("test_table", at) # RuntimeError: date32[day] is not yet supported
OmniSci seems to have a type with the similar name, so possibly the required change is just to add a case for the date
type to the switch where the exception is happening
https://github.com/omnisci/omniscidb/blob/416df923adebc6d2db1372986d961b563f94d7b7/Shared/sqltypes.h#L53
and fix this todo
https://github.com/omnisci/omniscidb/blob/416df923adebc6d2db1372986d961b563f94d7b7/DataMgr/ForeignStorage/ArrowForeignStorage.cpp#L924-L928
cc @Garra1980
Looks like should be something like https://github.com/omnisci/omniscidb/blob/416df923adebc6d2db1372986d961b563f94d7b7/QueryEngine/ArrowResultSetConverter.cpp#L972