TIMESTAMP is not supported in temporary table

[xmnlab]

Hi everyone!

I am trying to execute a simple sql with select_ipc, but it seems that has some conflict with timestamp fields. but using just execute works fine.

mapd_cli.con.select_ipc('''
SELECT *
FROM mapd.flights_2008_10k
LIMIT 5;
''')

---------------------------------------------------------------------------
TMapDException                            Traceback (most recent call last)
<ipython-input-29-3a4f34b7bd4d> in <module>()
      3 FROM mapd.flights_2008_10k
      4 LIMIT 5;
----> 5 ''')

~/miniconda3/envs/ibis/lib/python3.6/site-packages/pymapd/connection.py in select_ipc(self, operation, parameters, first_n)
    303         tdf = self._client.sql_execute_df(
    304             self._session, operation, device_type=0, device_id=0,
--> 305             first_n=first_n
    306         )
    307 

~/miniconda3/envs/ibis/lib/python3.6/site-packages/mapd/MapD.py in sql_execute_df(self, session, query, device_type, device_id, first_n)
   1069         """
   1070         self.send_sql_execute_df(session, query, device_type, device_id, first_n)
-> 1071         return self.recv_sql_execute_df()
   1072 
   1073     def send_sql_execute_df(self, session, query, device_type, device_id, first_n):

~/miniconda3/envs/ibis/lib/python3.6/site-packages/mapd/MapD.py in recv_sql_execute_df(self)
   1097             return result.success
   1098         if result.e is not None:
-> 1099             raise result.e
   1100         raise TApplicationException(TApplicationException.MISSING_RESULT, "sql_execute_df failed: unknown result")
   1101 

TMapDException: TMapDException(error_msg='Exception: TIMESTAMP is not supported in temporary table.')

[andrewseidl]

Initial server-side support added in https://github.com/mapd/mapd-core/commit/389368c5ffa984928df6729d8def0513d704ffcb.

Leaving this open since these types only seem to be supported in pymapd on CPU (select_ipc), not GPU (select_ipc_gpu), at least in the version of pygdf we're currently using.

[randyzwitch]

Going to close https://github.com/mapd/pymapd/issues/75, since this issue captures the same general behavior to be testing. DATE/TIMESTAMP support to be added...

[randyzwitch]

When this implemented for pygdf, then functionality will work https://github.com/rapidsai/cudf/issues/140

[sglyon]

from what I can tell cudf has some sort of date/timestamp functionality now

Is that usable for pymapd and select_ipc_gpu now?

I can't seem to find a combination of omnisci column types, load_table with properly typed dataframe columns, and select_ipc_gpu that works for a date or timestamp

[randyzwitch]

I thought we were waiting on the cudf linked issue above, but it's a good question. Let me take a look at the cudf documentation, maybe they fixed it but didn't close the issue.

[randyzwitch]

@sglyon This works now, so maybe the issue isn't in the libcudf C++ code like we thought:

import pymapd
import cudf

conn = pymapd.connect(user="mapd", password="HyperInteractive", 
                      host="localhost", port= 6274, dbname="mapd")


df_ipc = conn.select_ipc("select dep_timestamp from flights_2008_7M limit 1000")
df_gpu = cudf.DataFrame.from_pandas(df_ipc)

So the hacky way for now would be to do select_ipc if you can for now, then convert to a cudf if you want to keep your development going, and in the meantime I can see if I can send a patch to cudf.

[sglyon]

oh ok cool, thanks for posting the hack. I can do that for now.

Thanks!

[randyzwitch]

@sglyon Unfortunately, not as easy as I expected (you can check out https://github.com/rapidsai/cudf/issues/140 if you want to read about the guts of cudf via Arrow), but it looks like it's going to be a little longer before this is solved.

[sglyon]

ok, thanks for keeping me in the loop. Hopefully we can resolve it!