nntrainer

nntrainer copied to clipboard

Add saving optimizers

1

- Add simple bin header to check version - Save/load optimizer weights

zhoonit

context.request* should gaurantee index

1

https://github.com/nnstreamer/nntrainer/blob/f26937ee48243bb099edd37538d90054cd735d3d/nntrainer/layers/layer_context.h#L144-L145 above todo is critical when guaranteeing serialized weight layout, without those kind of deterministic assumption, we cannot load weight outside context of layer. eg) fully connected layer has weight...

zhoonit

ordering of sorted nodes in the order of adding to graph

1

With a described models, given two sibling nodes (X, Y) in the graph, if the user described node X first, and then node Y (in ini or in C++ api...

kparichay

Support KLD loss

2

Support KLD loss

kparichay

Model c++ api

3

It's about time to open/fixate model C++ API. We need list of things which should be supported. ### 1. Modification - list to be added ### 2. Enhancement - some...

zhoonit

Details in error messages

2

Error messages should contain enough details to point out the location of error: - Layer/LayerNode/LayerContext -> must include layer name in the error message - Optimizer/OptimizerContext -> must include optimizer...

kparichay

gradient should use initializer::NONE

1

Gradients currently use initializer::ZEROS which initializes the gradient for the first iteration. However, gradients are not initialized to zero for the next iterations and the layer is expected to initialize...

kparichay

Dangling gradient should be always zero

1

As we are having output that is being dangled, we have to manually set the incoming derivative to zero for this case. Later we can have a 'constant' (RV of...

zhoonit

Tensor copy/fill methods need to be unified

2

Tensor has a few copy and fill methods, and they should be unified. Refer https://github.com/nnstreamer/nntrainer/pull/1555#discussion_r725022403

kparichay

Handling addReferences & input realizers and recurrent end_layers

1

current neuralnet::addWIthReferenceLayers becomes too complicated that it is impossible to understand, this thread discuss how to untangle a bit to make it more intuitive + some missing functionalities needed 1....

zhoonit