Phillip Cloud

this kind of operation masquerades in `q` as `xbar`: ``` q)x: .z.p // np.datetime64('now') q)t: (x + `long $ (1e9*60) * 1 + til 10) // add array of 1...

i suspect that the `astype` calls would be the most expensive part of the conversion as they are doing unit conversion if the type we're going to is not the...

> Does this mean that backends that don't support arrays won't support .describe()/.info() anymore? Nope, I'm working on an improvement that will be both faster than the array optimization for...

@jcrist Here are the benchmarks comparing `main` to this PR: ``` --------------------------------------------------------------------------- benchmark 'test_summarize_compile[describe]': 2 tests ---------------------------------------------------------------------------- Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds...

Ah, the slowdown is because I changed the implementation to always produce all aggregates!

Found some more performance laying around in the form of not converting all memtables to pyarrow (which is noticeably expensive for wide tables). I'll put that in a separate PR.

Latest benchmarks show improvements across the board for `describe()` and `info()`: ``` --------------------------------------------------------------------------- benchmark 'test_summarize_compile[describe]': 2 tests ---------------------------------------------------------------------------- Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS...

Going to benchmark this with pyarrow to properly measure the speedup of these ops without the overhead of pandas memtable conversion

Added a benchmark for "end-to-end" which corresponds to construction + compilation + execution. ## `Table.describe()` - **End-to-end** is faster by about **~7x**. - **Construction** is faster by **>1000x** because we're...

This is turning out to be kind of a mess. I'm having to move the translation down a layer in some cases which is getting annoying. I'll take the weekend...