nntrainer

nntrainer copied to clipboard

[layer normalization] Implement layer normalization

1

Commit 1: [bn layer] refactoring bn layer - Refactoring bn layer - Added common_function directory Signed-off-by: hyeonseok lee Commit 2: [lstmcell] move lstmcell core to common_function - Move lstmcell_core.* to...

lhs8928

[multi head attention] implement finalize @open sesame 8/22 14:00

2

- implement multi head attention finalizeCommon, finalize Signed-off-by: hyeonseok lee

lhs8928

[Wait for #1962] [multi_head_attention] implement calcDerivative, calcGradient @open sesame 8/22 14:00

2

- Implement multi head attention calcDerivative, calcGradient - Needs to support attention mask Signed-off-by: hyeonseok lee

lhs8928

[Wait for #1961] [multi head attention] implement calcCommonDerivative @open sesame 8/22 14:00

2

- implement multi head attention calcCommonDerivative Signed-off-by: hyeonseok lee

lhs8928

[Wait for #1960] [multi head attention] implement forwarding @open sesame 8/22 14:00

2

- implement multi head attention forwarding Signed-off-by: hyeonseok lee

lhs8928

[Wait for #1963] [multi head attention] added unittest

3

- Added layer/model unittest for multi head attention Signed-off-by: hyeonseok lee

lhs8928

[CAPI] Add attention, mol attention and multi head attention to CAPI @open sesame 08/30 14:47

3

[CAPI] Add attention, mol attention and multi head attention to CAPI - Add ML_TRAIN_LAYER_TYPE_ATTENTION - Add ML_TRAIN_LAYER_TYPE_MOL_ATTENTION - Add ML_TRAIN_LAYER_TYPE_MULTI_HEAD_ATTENTION **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped...

songgot

[api] added newly implemented layer enum

1

- Added newly implemented layer enum to nntrainer-api-common.h Signed-off-by: hyeonseok lee

lhs8928

FullyConnected Layer Reordering for TF Lite Exporter

5

When Export ```NNTrainer``` to ```Tensorflow Lite``` we need to reordering --> Implement based on #1892 - [x] Implement Flatten Reordering - [x] Test various FC Case - [x] Make Unit...

DonghakPark

[Profile] Add memory statiscs & annotation

3

It add two functions: - provide PROFILE_MEM_ANNOTATE macro - print average and maximum usage of memory. Signed-off-by: Jiho Chu

jihochu