ArcticDB icon indicating copy to clipboard operation
ArcticDB copied to clipboard

ArcticException: 'Int64Dtype' object has no attribute 'hasobject'

Open moconnell opened this issue 1 year ago • 0 comments

Describe the bug

Write does not support the pandas dtype Int64Dtype:

File [/opt/homebrew/anaconda3/envs/py3110/lib/python3.11/site-packages/arcticdb/version_store/_normalization.py:182](http://localhost:8888/opt/homebrew/anaconda3/envs/py3110/lib/python3.11/site-packages/arcticdb/version_store/_normalization.py#line=181), in _to_primitive(arr, arr_name, dynamic_strings, string_max_len, coerce_column_type, norm_meta)
    181 obj_tokens = (object, "object", "O")
--> 182 if arr.dtype.hasobject is False and not (
    183     dynamic_strings and arr.dtype == "float" and coerce_column_type in obj_tokens
    184 ):
    185     # not an object type numpy column and not going to later be
    186     # coerced to an object type column - does not require conversion to a primitive type.
    187     return arr

AttributeError: 'Int64Dtype' object has no attribute 'hasobject'

Steps/Code to Reproduce

import numpy as np
import pandas as pd
import arcticdb as adb
import tempfile
import yfinance as yf

tmp = tempfile.gettempdir()

ac = adb.Arctic(f'lmdb:///{tmp}/arctic')
lib_name = 'Int64Dtype'
if not lib_name in ac.list_libraries():
    ac.create_library(lib_name)
lib = ac.get_library(lib_name)

symbol = "AAPL"
start_date = "2023-01-01"
end_date = "2023-12-31"

# Download OHLC data
df = yf.download(symbol, start=start_date, end=end_date)

df["weekday"] = df.index.weekday
df["weekday"].replace(6, -1, inplace=True)
df["day"] = df.index.day_name()
df["week_adj"] = -np.clip(df["weekday"], None, 0)

# N.B. remove np.int64 cast on next line to induce failure
df["week"] = np.int64(df.index.isocalendar().week + df["week_adj"])
df.loc[df.weekday.eq(-1) & (df.week > 52), "week"] = 1

# output dtypes to observe change with[out] cast
print(df.dtypes)

# attempt to write
lib.write(symbol, df)

Expected Results

Write with no error

OS, Python Version and ArcticDB Version

Python: 3.11.6 | packaged by conda-forge | (main, Oct 3 2023, 10:40:37) [Clang 15.0.7 ] OS: macOS-15.0-x86_64-i386-64bit ArcticDB: 2.0.0

N.B. this is actually an MBP M1 Max, so I'm not sure why the platform is listed as x86...

Backend storage used

LMDB

Additional Context

No response

moconnell avatar Oct 03 '24 12:10 moconnell