Jeff Neuenschwander

Results 17 comments of Jeff Neuenschwander

convert_datetime64 has actually been deprecated now, so this actually keeps TickStore from working for users of current versions of pandas.

I also ran a profiler on a read from tickstore, and below are a few relevant lines... Is there a different type that I should be saving the datetime index...

Is there any other information I could provide that would help to better describe this issue? It is still a problem for me. Thanks!

I did more testing on this, and the problem disappears when running pandas .25.3 It also disappears if you change the freq of the the sample index generator to '1ns'...

Yes, this makes me wonder if man financial (the creator and maintainer of this package) is really still using the old pandas .25 branch internally, or if nobody there is...

@crazy25000 thanks for the tip! Happy to continue this discussion, but I don't want to clutter up the github issue with it. My email is on my profile if you'd...

This is still an outstanding issue for me, if there is anything else I can provide to help clarify this issue, please let me know.

@bmoscon you are correct, if I use the pandas .25 branch, the problem is solved. However, this creates pretty serious workflow issues. If a user wants to pull data with...

@JunyueLiu I tweaked your suggestion just a little bit to: ``` index = pd.to_datetime(np.concatenate(rtn[INDEX]).astype('datetime64[ms]'), utc=True) ``` and now the reads are back to a normal speed, about 6.5M rows/sec. @JunyueLiu...

@CmpCtrl I tried your solution, and did indeed did get about 50% faster reads, about 9.5M rows/sec. Thanks!