TinyTensor
TinyTensor copied to clipboard
haobosang
TinyTensor is a tool for running already trained NN (Neural Network) models to be able to use them for inference of various tasks such as image classification, semantic segmentation, etc.
TinyTensor is an efficient lightweight deep learning inference framework.
About
TinyTensor supports a variety of popular neural network architectures such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and fully connected neural networks, and it can be used for tasks such as image classification, object detection, speech recognition, and natural language processing.
Development Environment
- Development language: C++ 20
- Math Library: Armadillo
- Logging framework：Google glog
- Unit test: Google Test
- Code style: Clang format
- Performance testing： Benckmark
How to build on Linux
Ubuntu 18 (Debian 10)
apt update
apt install cmake libopenblas-dev liblapack-dev \
libarpack2-dev libsuperlu-dev libomp-dev libopencv-dev
Install Armadillo
wget https://sourceforge.net/projects/arma/files/armadillo-12.2.0.tar.xz
mkdir build && cd build
cmake ..
make -j8
make install
Install Benchmark
cd third_party
git submodule update --init
mv googletest benchmark
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=RELEASE ../benchmark
make -j8
# 如果想全局安装就接着运行下面的命令
sudo make install
Operators Currently Implemented
- ReLU
- Sigmoid
- Conv
- MaxPooling
Performance Testing
Test Equipment
Intel(R) Xeon(R) W-2223 CPU @ 3.60GHz
Compilation Environment
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Performance Results
| Input size | Model | Computing Device | Time |
|---|---|---|---|
| 224×224 batch = 8 | ResNet18 | CPU(armadillo) | 55ms / image |
| 224×224 batch =16 | ResNet18 | CPU(armadillo) | 28.5ms / image |
Acknowledgement
caffe
haobosang