ArcticDB icon indicating copy to clipboard operation
ArcticDB copied to clipboard

lib._nvs.get_num_rows incorrect for pickled symbol

Open muhammadhamzasajjad opened this issue 9 months ago • 2 comments

Describe the bug

When a pickled symbol is written, lib._nvs.get_num_rows returns incorrect row count. Please view the repro code.

Steps/Code to Reproduce


lib.write_pickle("sym_pickled", pd.date_range("2024-03-01", "2024-03-31", freq="B"))
lib._nvs.get_num_rows("sym_pickled")
89 # expected  21
lib.get_description("sym_pickled")
SymbolDescription(columns=(NameWithDType(name='bytes', dtype=value_type: UINT
size_bits: S64
),), index=NameWithDType(name=[], dtype=[]), index_type='NA', row_count=89, last_update_time=Timestamp('2024-05-08 16:34:17.471168900+0000', tz='UTC'), date_range=(numpy.datetime64('NaT'), numpy.datetime64('NaT')), sorted='UNKNOWN')

Expected Results

21

OS, Python Version and ArcticDB Version

Python: 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] OS: Windows-10-10.0.22631-SP0 ArcticDB: 4.4.1

Backend storage used

LMDB

Additional Context

No response

muhammadhamzasajjad avatar May 02 '24 15:05 muhammadhamzasajjad

get_num_rows doesn't exist on Library, it's lib._nvs.get_num_rows. Can you check lib.get_description?

jamesmunro avatar May 06 '24 20:05 jamesmunro

get_num_rows doesn't exist on Library, it's lib._nvs.get_num_rows. Can you check lib.get_description?

The provided code was using V1 API where lib is instance of NativeVersionStore, because this issue came from Man internally. I will update the repro with lib._nvs.get_num_rows

muhammadhamzasajjad avatar May 07 '24 10:05 muhammadhamzasajjad