nntrainer
nntrainer copied to clipboard
nnstreamer
NNtrainer is Software Framework for Training Neural Network Models on Devices.
## In this PR This PR adds the is_NaN function to check if the tensor has a NaN value. This is for the check of NaN during mixed precision training....
## In this PR This PR includes the mixed precision test case. . Input - FC - MSE : "batch_size=2", "model_tensor_type=FP16-FP16", "loss_scale=128" **Self evaluation:** 1. Build test: [X]Passed [ ]Failed...
## In this PR This PR enables the FP16 support for the layers below: . input layer . mse loss layer Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed...
- We have executed the LLaMA model (downloaded from HuggingFace[https://huggingface.co/meta-llama/Llama-2-7b-chat-hf]) using the NNTrainer and obtained the following output by following these steps: 1. File changes made before running the LLaMA...
Some code could break build with `-Denable-fp16=true` option, like https://github.com/nnstreamer/nntrainer/pull/2545. I think it would be helpful to add this build to CI to ease the burden on reviewers.
[ Wait for #2500 ] [ BLAS ] Refactor blas/math related files into cpu backend considering arch-dep
While during the process of implementing additional features in NEON, I found myself making unnecessary code blocks. This is a suggestion-draft of refactorization for current blas/math related files. **DONE** -...
Some activation types were missing from EnumList. Added missing types to EnumList. Changed the order of ActivationType and EnumList to be the same. **Self evaluation:** 1. Build test: [X]Passed [...
## In this PR This PR finalizes the mixed precision support in NNTrainer. It modifies the network grap and layer node, and layer implementations. However, it does not support mixed...
Add Mixed Precision example on Application - this example can guide developer to handle fp16 example - we can test & eval our model end-to-end - we can optimize base...
- add tanh-based approximate gelu(tanh gelu) for vision transformer. - rename quick gelu to sigmoid gelu(it's a sigmoid-based approximate gelu) **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped...
Metadata
Owner
Metadata
NNtrainer is Software Framework for Training Neural Network Models on Devices.