dfdx
dfdx copied to clipboard
coreylowman
Deep learning in Rust, with shape checked tensors and neural networks
- [ ] Linear batched forward (matmul & broadcast add) - [ ] Backprop algorithm - [ ] Optimizer updates - [ ] forward with tape & without tape
E.g. - const generic sizes - clean/safe/hackable implementation - Gradients not stored directly on parameters, makes updating & writing optimizers much cleaner
- [x] #158 - [ ] #157 - [ ] #150 - [ ] #149 - [x] #147 - [x] #146 - [ ] #121 - [x] #108 - [...
Should CUDA kernels be JIT compiled at runtime or somehow compiled when the program is built? Best case we can support both of these easily via a feature flag or...
Related to #9 Since cuda operations can all fail, all the tensor operations/device level ops will need to return a result. I think this can be done in a non-breaking...
This would likely use something like rayon under the hood to run operations inside a threadpool
Related to #9 This would mainly be used for element-wise operations, where the existing rust closures are specified at the element level. This method would be preferred over specifying both...
Apparently, channels last image format improves efficiency a lot (see pytorch recen blog https://pytorch.org/blog/tensor-memory-format-matters/). Need to investigate: 1. How much this helps convolutions 2. How confusing this would be for...
First used in #182 I think the Index, Recurse, Broadcast could be used directly for it
- Remove certain things from prelude - make internal crate files not use blanket import (this makes it hard when browsing from non-editors to see where things come from)
Metadata
Owner
Metadata
Deep learning in Rust, with shape checked tensors and neural networks