nntrainer
nntrainer copied to clipboard
Support transformer
- [ ] Implement multi head attention
- [x] implement forwarding - #1961
- [x] implement backwarding - #1963
- [ ] implement attention mask
- [x] support float type mask - #1963
- [ ] support bool type mask
- [x] verify with unittest
- [x] Implement layer normalization
- [x] integrate core algorithm with bn layer - #1993
- [x] implement forwarding - #1993
- [x] verify with unittest - #1993
- [ ] Make transformer as an application
- [x] implement positional encoding layer - #1989
- [x] verify encoder/decoder layer with unittest - #1996
- [ ] train with dataset
:octocat: cibot: Thank you for posting issue #1972. The person in charge will reply soon.