nnue-pytorch
nnue-pytorch copied to clipboard
TODO for nnue.md
- [ ] some results regarding net size (https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY/edit#heading=h.p7t1qfux64dc exps 84, 85, 86, 89)
- [ ] about data used to train the nets (filtering, mirroring and stuff)
- [x] fix header links in doc
- [ ] make headings understandable without context
- [ ] consistent formatting (for feature set names, net arch names, etc...)
- [ ] backprop math
- [x] linear layer with block sparse outputs forward (no results yet but it's an interesting direction)
- [x] linear layer with sparse inputs forward (put some data about sparsity after CReLU)
- [x] ways to reduce the network size with Half* (less king buckets, perspective mirror)
- [ ] either write something about weight pruning or remove the mention that it will be described later in the document
- [x] revise implementation for sparse input FC (pad nnz indices so that only one loop is needed. use maddubs -> madd(ones, ...) implementation that processes 4 inputs at a time, with 8 bit weights). See https://pastebin.com/zDa7PbJh
- [x] a better nnz indices implementation for the above, see https://github.com/syzygy1/Cfish/issues/204