Morph.py
Morph.py copied to clipboard
stephenjfox
A neural architecture optimizer, targeted at model-size w.r.t. accuracy
It's awesome you implemented this in pytorch. I am really not sure how I can use this on my net. I see there's a demo.py but unless I'm missing something...
* **Please check if the PR fulfills these requirements** - [x] The commit message follows the [guidelines](https://chris.beams.io/posts/git-commit/) - [x] Tests for the changes have been added (for bug fixes /...
## New Functionality * [ ] Supports filling out an accuracy budget * [ ] Supports extreme size dropping
Lying somewhere between a bug and a feature enhancement, the current iteration of the `nn.Module.children` assumes that the layers are in a sequence (like a significantly deep convolutional network). It...
- [x] Prove that `sparsify(tensor, threshold)` performs as intended - [x] Prove `shrink` on a given layer produces the correct count of neurons for a given layer - [x] ......
> Once we've detected all of the sparse neurons in the network, we're going to clip them out. Make that happen in a sensible fashion
> If the layers done fit together snuggly - the dimensions of their dimensions don't align - we reshape them to better fit together. * This is done by resizing...
This has already been implemented, in the form of `resize_layers`. This ticket includes the following: * Rename the function with improvement to the documentation * Unit test the function *...
Could these be better named, or segregated? It may not be evident yet (as there's not a ton of code) but if every piece of the project is using these...
Metadata
Owner
Metadata
A neural architecture optimizer, targeted at model-size w.r.t. accuracy