horde-ad
horde-ad copied to clipboard
Mikolaj
Higher Order Reverse Derivatives Efficiently - Automatic Differentiation library based on the paper "Provably correct, asymptotically efficient, higher-order reverse-mode automatic differentiation"
This is @awf's idea that would improve the end user API. The `Nothing` would signify that the shape may differ at runtime. Cognitive load would be reduced (no more `ranked`...
When the implementation of fold, scan, mapAccum and their derivatives settles down, let's think about parallel execution. A GPU backend to actually benchmark the result wouldn't hurt either. Tom provides...
This is low priority, because it's going to be a few (dozen) percent speedup (and a slight space saving and improved storage locality, which may be more important in real...
Edsko says: "I think AllowAmbiguousTypes is almost never the right solution; the main problem is that it gives the user no indication at all as to which type variables might...
@tomsmeding said: > I wonder if what you need here is generating random programs :p And indeed I need it. Dimensions only 1, 2 or 3, ranks up to 5,...
Here ```hs nestedGather :: forall r. ADReady r => TensorOf 2 r -> TensorOf 2 r nestedGather t = tgather (2 :$ 2 :$ ZS) (tgather (2 :$ 3 :$...
Implements #58. An attempt to recognize to which person a voice belongs in a given window of a sound file. Uses RNN. The tests to run are `cabal test extremelyLongTest...
Try to implement checkpointing (inserting recomputation to trade-off computation vs memory use) and then automatic checkpointing, which is what pytorch/JAX users now reportedly need and can't get. We have an...
This is the hammer and it may just succeed: https://github.com/alt-romes/hegg
Desiderata: - [ ] MNIST example with MatMul only - [ ] MNIST example with convolutions - [ ] ... - [ ] GPT-3 on 64 GPUs Supposedly, this can...