cedricyau

Is Firebase an option? Real-time JSON datastore with access to any level of the tree. Also notifies and broadcasts updates. https://firebase.google.com/

Coincidentally, I followed the links on Realtime API warning to use Cloud Firestore and it went to: https://firebase.google.com/products/firestore/ Will need to read up more on Firebase and Firestore. Seems like...

It seems that unlike Google Drive/Realtime, Cloud Firestore requires a Google API account for billing. What would be required to at least get real-time running on the same jupyterlab server?...

I've attached a Jupyter Notebook that uses sample data. First, I generate a 10MM row dataframe with 5 numeric columns. Then I add a sample string column and a sample...

Unless I'm missing something, I believe that's already been implemented. https://github.com/innobi/pantab/blob/5fc6a6471e521c94dfc82d10239051997ef175f1/pantab/_writermodule.c#L161 It does look like the ultimate result is to get to an int64 representation. ``` int64_t val = time...

Today I learned about the Julian date. I modified types to define the hyper column as an int64 and then tried a few values. The int value is in microseconds....

I think it’s the combo. There’s a decent amount of calculation needed to get from the unified int64 to a year, then month, day, etc... Then you combine everything back....

Yes, the code would result in an off-by-1 error. It appears that Tableau implemented their epoch starting at midnight instead of noon. So the offset is 2440588.0 instead of 2440587.5...

Thanks to 0e2f161, I got a head start on implementation and tying back. You'll see in the attached that re-querying indeed produces the correct result. [pantab research benchmark-optimized.zip](https://github.com/innobi/pantab/files/3926028/pantab.research.benchmark-optimized.zip) ## Before...

``` In [5]: dt=datetime.now() In [9]: %timeit dt.fromtimestamp(dt.timestamp()) 1.59 µs ± 6.03 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) In [10]: %timeit dt.timestamp() 851...