Error when loading int16 to created table with datatype INTEGER ENCODING FIXED(16)

[abeduplaa]

trafficstars

When attempting to load a pandas dataframe with a datatype of int16 to a table with predefined datatype of INTEGER ENCODING FIXED(16), the following error is thrown:

---------------------------------------------------------------------------
TMapDException                            Traceback (most recent call last)
<ipython-input-55-22241a2cb585> in <module>
----> 1 con.load_table('cow_data3', df_sick, method='infer', create=False)

~/anaconda3/envs/omnisci/lib/python3.6/site-packages/pymapd/connection.py in load_table(self, table_name, data, method, preserve_index, create)
    476         if method == 'infer':
    477             if (isinstance(data, pd.DataFrame) or _is_arrow(data)):
--> 478                 return self.load_table_arrow(table_name, data)
    479 
    480             elif (isinstance(data, pd.DataFrame)):

~/anaconda3/envs/omnisci/lib/python3.6/site-packages/pymapd/connection.py in load_table_arrow(self, table_name, data, preserve_index)
    621                                            preserve_index=preserve_index)
    622         self._client.load_table_binary_arrow(self._session, table_name,
--> 623                                              payload.to_pybytes())
    624 
    625     def render_vega(self, vega, compression_level=1):

~/anaconda3/envs/omnisci/lib/python3.6/site-packages/mapd/MapD.py in load_table_binary_arrow(self, session, table_name, arrow_stream)
   2438         """
   2439         self.send_load_table_binary_arrow(session, table_name, arrow_stream)
-> 2440         self.recv_load_table_binary_arrow()
   2441 
   2442     def send_load_table_binary_arrow(self, session, table_name, arrow_stream):

~/anaconda3/envs/omnisci/lib/python3.6/site-packages/mapd/MapD.py in recv_load_table_binary_arrow(self)
   2462         iprot.readMessageEnd()
   2463         if result.e is not None:
-> 2464             raise result.e
   2465         return
   2466 

TMapDException: TMapDException(error_msg='Exception: Expected int32 type')

to load the table, the following function from pymapd api was used: con.load_table('table', df, method='infer', create=False)

[randyzwitch]

Thanks for reporting @abeduplaa , we're in the middle of auditing all the functionality in pymapd, and we'll hopefully be able to fix this in the near future

[randyzwitch]

@andrewseidl I think this one is for you. Sounds like on the backend, Arrow still goes through an endpoint that wants an Int32 even for an Int16

[randyzwitch]

This same error happens for TEXT ENCODING NONE also, says it's looking for TMapDException: TMapDException(error_msg='Exception: Expected int32 type')