To do list
-
[x] Think about
summarizegroupbyetc... Is there need for macros? -
[x] Pipeline when using functions from
IndexedTablesthat don't take data as first argument? -
[x] Docstrings
-
[x] support distributed tables?
-
[x] allow escaping symbols
-
[x]
_to refer to table inside@with -
[ ] Make all macros compatible with table of tables (better to wait for table of tables proper support in JuliaDB)
-
[x]
begin endsyntax to make multiple transformations in a row
Consider how best to optimize @apply for grouped data. Ideally, the macro should keep track of anything that happens during the grouping and only select the columns that are used.
Regarding the distributed table support, we're missing a method for view(::DTable, i) and either broadcasting or indexing on Dagger.DArray
EDIT: all row-wise operation should automatically use delayedmap in the distributed case. Possibly add a chunk version of macros (@apply_in_chunks)