nnue-pytorch icon indicating copy to clipboard operation
nnue-pytorch copied to clipboard

TODO for nnue.md

Open Sopel97 opened this issue 3 years ago • 0 comments

  • [ ] some results regarding net size (https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY/edit#heading=h.p7t1qfux64dc exps 84, 85, 86, 89)
  • [ ] about data used to train the nets (filtering, mirroring and stuff)
  • [x] fix header links in doc
  • [ ] make headings understandable without context
  • [ ] consistent formatting (for feature set names, net arch names, etc...)
  • [ ] backprop math
  • [x] linear layer with block sparse outputs forward (no results yet but it's an interesting direction)
  • [x] linear layer with sparse inputs forward (put some data about sparsity after CReLU)
  • [x] ways to reduce the network size with Half* (less king buckets, perspective mirror)
  • [ ] either write something about weight pruning or remove the mention that it will be described later in the document
  • [x] revise implementation for sparse input FC (pad nnz indices so that only one loop is needed. use maddubs -> madd(ones, ...) implementation that processes 4 inputs at a time, with 8 bit weights). See https://pastebin.com/zDa7PbJh
  • [x] a better nnz indices implementation for the above, see https://github.com/syzygy1/Cfish/issues/204

Sopel97 avatar Jun 08 '21 22:06 Sopel97