JuliaDBMeta.jl icon indicating copy to clipboard operation
JuliaDBMeta.jl copied to clipboard

To do list

Open piever opened this issue 7 years ago • 2 comments

  • [x] Think about summarize groupby etc... Is there need for macros?

  • [x] Pipeline when using functions from IndexedTables that don't take data as first argument?

  • [x] Docstrings

  • [x] support distributed tables?

  • [x] allow escaping symbols

  • [x] _ to refer to table inside @with

  • [ ] Make all macros compatible with table of tables (better to wait for table of tables proper support in JuliaDB)

  • [x] begin end syntax to make multiple transformations in a row

piever avatar Feb 22 '18 14:02 piever

Consider how best to optimize @apply for grouped data. Ideally, the macro should keep track of anything that happens during the grouping and only select the columns that are used.

piever avatar Feb 25 '18 12:02 piever

Regarding the distributed table support, we're missing a method for view(::DTable, i) and either broadcasting or indexing on Dagger.DArray

EDIT: all row-wise operation should automatically use delayedmap in the distributed case. Possibly add a chunk version of macros (@apply_in_chunks)

piever avatar Mar 12 '18 18:03 piever