ArcticDB
ArcticDB copied to clipboard
ArcticException: 'Int64Dtype' object has no attribute 'hasobject'
Describe the bug
Write does not support the pandas dtype Int64Dtype:
File [/opt/homebrew/anaconda3/envs/py3110/lib/python3.11/site-packages/arcticdb/version_store/_normalization.py:182](http://localhost:8888/opt/homebrew/anaconda3/envs/py3110/lib/python3.11/site-packages/arcticdb/version_store/_normalization.py#line=181), in _to_primitive(arr, arr_name, dynamic_strings, string_max_len, coerce_column_type, norm_meta)
181 obj_tokens = (object, "object", "O")
--> 182 if arr.dtype.hasobject is False and not (
183 dynamic_strings and arr.dtype == "float" and coerce_column_type in obj_tokens
184 ):
185 # not an object type numpy column and not going to later be
186 # coerced to an object type column - does not require conversion to a primitive type.
187 return arr
AttributeError: 'Int64Dtype' object has no attribute 'hasobject'
Steps/Code to Reproduce
import numpy as np
import pandas as pd
import arcticdb as adb
import tempfile
import yfinance as yf
tmp = tempfile.gettempdir()
ac = adb.Arctic(f'lmdb:///{tmp}/arctic')
lib_name = 'Int64Dtype'
if not lib_name in ac.list_libraries():
ac.create_library(lib_name)
lib = ac.get_library(lib_name)
symbol = "AAPL"
start_date = "2023-01-01"
end_date = "2023-12-31"
# Download OHLC data
df = yf.download(symbol, start=start_date, end=end_date)
df["weekday"] = df.index.weekday
df["weekday"].replace(6, -1, inplace=True)
df["day"] = df.index.day_name()
df["week_adj"] = -np.clip(df["weekday"], None, 0)
# N.B. remove np.int64 cast on next line to induce failure
df["week"] = np.int64(df.index.isocalendar().week + df["week_adj"])
df.loc[df.weekday.eq(-1) & (df.week > 52), "week"] = 1
# output dtypes to observe change with[out] cast
print(df.dtypes)
# attempt to write
lib.write(symbol, df)
Expected Results
Write with no error
OS, Python Version and ArcticDB Version
Python: 3.11.6 | packaged by conda-forge | (main, Oct 3 2023, 10:40:37) [Clang 15.0.7 ] OS: macOS-15.0-x86_64-i386-64bit ArcticDB: 2.0.0
N.B. this is actually an MBP M1 Max, so I'm not sure why the platform is listed as x86...
Backend storage used
LMDB
Additional Context
No response