SeeDot-dev.py only working for fastgrnn

[ChristopherKoeppen]

Hello,

the SeeDot explanation is working for the Fastgrnn. But when I transfer the same steps for Bonsai or ProtoNN, the workaround makes some problems. For example: python SeeDot-dev.py -a bonsai -e fixed -d usps10 -n 1 -t arduino -m red_disagree -l info

It starts with an Error for the assignment in line 227: acc = obj.testingAccuracy with the information, that the Main-Object has no "testingAccuracy".

Is there an easy way to fix this an do the same quantization for bonsai and protoNN?

[ShikharJ]

@affot The code pertaining to SeeDot is slightly unorganised. I've encountered this error before as well, and most of the times, it is because of the datasets and the model files not being in the right directory. Can you just check if there is a bonsai/usps10 directory in both the models/ and datasets/ folders? Ideally, for each algorithm, you should be structuring the models/bonsai and datasets/bonsai folder in the same manner as the models/fastgrnn and datasets/fastgrnn.

[ChristopherKoeppen]

Hello @ShikharJ thanks for your fast reply. Yes, I checked it several times, I created the same folder/file structures as in the fastgrnn example.

[ShikharJ]

Hmm, in that case, could you look at temp/Predictor/output/fixed/ and paste the outputs of build.txt and exec.txt please?

[ChristopherKoeppen]

I don't see no output folder in temp/Predictor. There is just an input folder?

[ShikharJ]

Okay, I must apologize for leading you astray. It seems that the nomenclature that is being followed internally is usps-multiclass instead of usps10. Could you try changing datasets/bonsai/usps10 to datasets/bonsai/usps-multiclass and models/bonsai/usps10 to models/bonsai/usps-multiclass and change the command you're using to python SeeDot-dev.py -a bonsai -e fixed -d usps-multiclass -n 1 -t arduino -m red_disagree -l info, please?

[ChristopherKoeppen]

I did it, but I get the same Error :-(

========================================
Executing on bonsai fixed usps-multiclass arduino
========================================

INFO:root:-----------------------

INFO:root:Collecting profile data

INFO:root:-----------------------

Traceback (most recent call last):
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/main.py", line 241, in convert
    obj.run()
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/compiler/converter/converter.py", line 51, in run
    obj.run(self.source)
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/compiler/converter/quantizer.py", line 252, in run
    self.processModel()
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/compiler/converter/quantizer.py", line 190, in processModel
    self.readModel()
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/compiler/converter/quantizer.py", line 78, in readModel
    getModelDir(), param.name + ".npy"))
  File "/home/ck/anaconda3/envs/BT36/lib/python3.6/site-packages/numpy/lib/npyio.py", line 422, in load
    fid = open(os_fspath(file), "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'model/bonsai/usps-multiclass/W1.npy'
Traceback (most recent call last):
  File "SeeDot-dev.py", line 270, in <module>
    obj.run()
  File "SeeDot-dev.py", line 164, in run
    self.runMainDriver()
  File "SeeDot-dev.py", line 227, in runMainDriver
    acc = obj.testingAccuracy
AttributeError: 'Main' object has no attribute 'testingAccuracy'

[ShikharJ]

Could you share the input.sd inside the model/bonsai/usps-multiclass?

[ChristopherKoeppen]

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license.

let W1 = (30, 8) in [-0.1, 0.1] in
let W2 = (8, 64) in [-0.1, 0.1] in
let U1 = (64, 8) in [-0.1, 0.1] in
let U2 = (8, 64) in [-0.1, 0.1] in
let Bg = (1, 64) in [-0.1, 0.1] in
let Bh = (1, 64) in [-0.1, 0.1] in
let FC1 = (64, 20) in [-0.1, 0.1] in
let FC2 = (20, 72) in [-0.1, 0.1] in
let FCBias = (1, 72) in [-0.1, 0.1] in

let X = (210,1) in [-0.1, 0.1] in

let XX = reshape(X, (7, 1, 30), (1, 2)) in

let zeta = 0.15637416 in
let nu = 0.01094334 in

let H = init([1, 64], 0.0) in

let res = loop(i = [0:7], H)
(
	let a = (XX[i] * W1) * W2 in
	let b = (H * U1) * U2 in
	let c = a + b in
	let g = sigmoid(c + Bg) in
	let h = tanh(c + Bh) in
	let H = (g <*> H) + (zeta * (1.0 - g) + nu) <*> h in H
) in

let score = ((res * FC1) * FC2) + FCBias in
argmax(score)

[ShikharJ]

@affot Sorry for the late reply. You're using the DSL code pertaining to FastGRNN for running the Bonsai code for usps-multiclass. The converter is currently broken, and we're trying to fix it in the coming month. You might wanna use the following script instead of your current input.sd:

let X   = (257, 1)   in [-2.406600, 5.292900] in
let Z   = (14, 257)  in [-0.293206, 0.466991] in
let W   = (3, 10, 14) in [-0.837834, 0.865861] in
let V   = (3, 10, 14) in [-0.950315, 0.847806] in
let T   = (1, 1, 14) in [-0.030703, 0.069962] in
let mean = (14, 1) in [-0.000001, 0.000001] in

let ZX = Z |*| X - mean in

// depth 0
let node0   = 0    in
let W0      = W[node0] * ZX in
let V0      = V[node0] * ZX in
let V0_tanh = tanh(V0) in
let score0  = W0 <*> V0_tanh in

// depth 1
let node1   = (T[node0] * ZX) >= 0? 2 * node0 + 1 : 2 * node0 + 2 in
let W1      = W[node1] * ZX in
let V1      = V[node1] * ZX in
let V1_tanh = tanh(V1) in
let score1  = score0 + W1 <*> V1_tanh in

argmax(score1)

Please note that the model/bonsai/usps-multiclass directory should contain X.npy, Z.npy etc as declared above. And you might need to change the provided numerical ranges, based on the dumped weights of the trained Bonsai model.

[ChristopherKoeppen]

Hello @ShikharJ one day delay... So no need to excuse!! :-) I still have the problem... What I did: Copy&Paste the code in the EdgeML/tools/SeeDot/model/bonsai/usps-multiclass/input.sd Copy&Paste all files from EdgeML/examples/tf/Bonsai/usps10/TFBonsaiResults/20_15_05_24_01_22/SeeDot (Mean, Sigma, Std, T,V,W,Z) in EdgeML/tools/SeeDot/model/bonsai/usps-multiclass

And then: python SeeDot-dev.py -a bonsai -e fixed -t arduino -m red_disagree -n 1 -d usps10 -->Error: ''AttributeError: 'Main' object has no attribute 'testingAccuracy' ''

I also tried python SeeDot-dev.py -a bonsai -e fixed -t arduino -m red_disagree -n 1 -d usps-multiclass -->Error: '' FileNotFoundError: [Errno 2] No such file or directory: './Predictor': './Predictor' ''

[ShikharJ]

@affot Sorry for the delay once more :) Nearly all of the debugging regarding SeeDot can be done by looking at temp/Predictor/output folders. Basically, the second command is correct, but somewhere during the execution of the compiled binary code for the arduino platform, some build/execution error is happening. Could you run the second command and paste the output of the mentioned folder, please? If this is a file not found error, then there might be a compilation issue.

[swapnilsayansaha]

I have similar issue. In my case, however, the dataset is a custom dataset which has been properly converted into .npy files. The input array to the fastgrnn model is 550*6. Note that I am trying to modify fastgrnn.sd to account for the shapes, but I am not sure what the two values in between the two 'ins', e.g., let W = (,) in [,] in refer to. I know the tuples in the round brackets are shape, but don't know what the tuples in the square brackets are. @ShikharJ would appreciate some help. Also is X the input array? What is XX?

[ShikharJ]

@swapnilsayansaha The values in the square brackets are the floating-point ranges of the concerned tensors. So for example, if your W.npy has a minimum element of 0.1, and a maximum element of 0.99, then the input.sd should look like let W = (x, y) in [0.1, 0.99] in

X is the input array, and XX is just a reshape operation applied to X and stored in a separate array. I hope this helps.

[swapnilsayansaha]

Thanks a lot. That really helped. I have a few more questions @ShikharJ

If X = (550,6), what should be XX be reshaped to (probably the decomposition is due to memory efficiency)?

I can provide some intuition. In the example of Seedot, we have:

let X = (256, 1)  in [-2.0, 7.0] in
let XX = reshape(X, (16, 1, 16), (1, 2)) in

This comes from the fact the USPS dataset input is a 256*1 array (assuming batch size of 1). The input dim was 16 and hidden dim was 32. The output is binary, which explains (1,2) in the output (?)

In my case: my input dim is [550,6] and hidden dim is 50. the output is (1,9). I was wondering how to properly select the parameters in reshape line of XX. (For matrix multiplication intuition, my W is 6*50 (for the line XX[i] * W)

[ShikharJ]

@swapnilsayansaha The hidden dimension has got nothing to the shape of X, it is the second dimension of W and U matrices that determines that. The reshape works in the following manner:

let XX = reshape(X, (16, 1, 16), (1, 2)) in

The first tuple in the reshape operation depicts the final shape of the reshape output (or simply, the shape of XX). The last tuple depicts the order in which the dimension of X should be processed. So in this example, the command semantically signifies "Reshape X into 16 patches of size (1, 16), while traversing X in a row-first fashion". I hope this clears things up.

[ChristopherKoeppen]

When I cast python SeeDot-dev.py -a bonsai -e fixed -t arduino -m red_disagree -n 1 -d usps-multiclass the Error is

Traceback (most recent call last):
  File "SeeDot-dev.py", line 270, in <module>
    obj.run()
  File "SeeDot-dev.py", line 164, in run
    self.runMainDriver()
  File "SeeDot-dev.py", line 225, in runMainDriver
    obj.run()
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/main.py", line 839, in run
    return self.runForFixed()
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/main.py", line 752, in runForFixed
    res = self.collectProfileData()
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/main.py", line 696, in collectProfileData
    execMap = self.predict(config.Encoding.floatt, config.DatasetType.training)
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/main.py", line 260, in predict
    execMap = obj.run()
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/predictor.py", line 211, in run
    execMap = self.execute()
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/predictor.py", line 185, in execute
    return self.executeForLinux()
  File "/home/ck/Downloads/EdgeML/tools/SeeDot/seedot/predictor.py", line 171, in executeForLinux
    process = subprocess.call(args, stdout=file, stderr=subprocess.STDOUT)
  File "/home/ck/anaconda3/envs/BT36/lib/python3.6/subprocess.py", line 287, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/home/ck/anaconda3/envs/BT36/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/home/ck/anaconda3/envs/BT36/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: './Predictor': './Predictor'

and in temp/Predictor/output/float I have 2 files (build.txt, exec.txt)

build.txt: g++ -c -o debug.o -Wall -Wno-narrowing -p -g -fPIC -O3 -std=c++11 -pthread debug.cpp g++ -c -o library_fixed.o -Wall -Wno-narrowing -p -g -fPIC -O3 -std=c++11 -pthread library_fixed.cpp library_fixed.cpp: In function ‘void NormaliseL2(MYINT*, MYINT*, MYINT, MYINT, MYINT, MYINT, MYINT, MYINT)’: library_fixed.cpp:915:30: warning: suggest parentheses around ‘-’ inside ‘<<’ [-Wparentheses] 915 | MYINT yHigh = (1 << shrA - 1); | ~~~~~^~~ library_fixed.cpp: In function ‘void Sigmoid(MYINT*, MYINT, MYINT, MYINT, MYINT, MYINT, MYINT, MYINT, MYINT*)’: library_fixed.cpp:961:8: warning: unused variable ‘scale_diff’ [-Wunused-variable] 961 | MYINT scale_diff = scale_out / scale_in; | ^~~~~~~~~~ g++ -c -o library_float.o -Wall -Wno-narrowing -p -g -fPIC -O3 -std=c++11 -pthread library_float.cpp g++ -c -o main.o -Wall -Wno-narrowing -p -g -fPIC -O3 -std=c++11 -pthread main.cpp main.cpp: In function ‘int main(int, char**)’: main.cpp:500:33: warning: comparison with string literal results in unspecified behavior [-Waddress] 500 | if ((argc == 6) && (argv[5] == "False")) | ^~~~~~~ main.cpp:142:8: warning: unused variable ‘epsilon’ [-Wunused-variable] 142 | float epsilon = 0.00001; | ^~~~~~~ g++ -c -o profile.o -Wall -Wno-narrowing -p -g -fPIC -O3 -std=c++11 -pthread profile.cpp g++ -c -o seedot_fixed.o -Wall -Wno-narrowing -p -g -fPIC -O3 -std=c++11 -pthread seedot_fixed.cpp g++ -c -o seedot_float.o -Wall -Wno-narrowing -p -g -fPIC -O3 -std=c++11 -pthread seedot_float.cpp In file included from seedot_float.cpp:9: model_float.h:84:1: error: too many initializers for ‘const float [3][10][14]’ 84 | }; | ^ model_float.h:157:1: error: too many initializers for ‘const float [3][10][14]’ 157 | }; | ^ model_float.h:163:1: error: too many initializers for ‘const float [1][1][14]’ 163 | }; | ^ In file included from seedot_float.cpp:9: model_float.h:166:3328: error: too many initializers for ‘const float [14][1]’ 166 | -0.991950f, -0.972569f, -0.930964f, -0.853716f, -0.734950f, -0.580235f, -0.392483f, -0.226989f, -0.217470f, -0.351633f, -0.517160f, -0.670813f, -0.799098f, -0.893281f, -0.953961f, -0.985741f, -0.982065f, -0.939628f, -0.848189f, -0.687138f, -0.468550f, -0.236652f, -0.010466f, 0.169037f, 0.166962f, 0.000575f, -0.195308f, -0.405139f, -0.621117f, -0.796838f, -0.910481f, -0.970707f, -0.972356f, -0.908924f, -0.774088f, -0.553005f, -0.301348f, -0.108794f, 0.024146f, 0.121127f, 0.093866f, -0.026314f, -0.130687f, -0.278695f, -0.508489f, -0.734396f, -0.884430f, -0.962162f, -0.964207f, -0.884020f, -0.720834f, -0.480033f, -0.259240f, -0.159720f, -0.134882f, -0.110839f, -0.142934f, -0.190815f, -0.187280f, -0.251017f, -0.456327f, -0.700435f, -0.873188f, -0.961192f, -0.956769f, -0.861720f, -0.683815f, -0.451380f, -0.276722f, -0.242718f, -0.272671f, -0.275193f, -0.286166f, -0.277784f, -0.224064f, -0.255743f, -0.443262f, -0.684031f, -0.864659f, -0.960294f, -0.947782f, -0.839320f, -0.659494f, -0.448801f, -0.304465f, -0.285991f, -0.316657f, -0.306787f, -0.287803f, -0.262377f, -0.227436f, -0.280945f, -0.461721f, -0.681623f, -0.853369f, -0.953936f, -0.935957f, -0.817802f, -0.645808f, -0.460782f, -0.334343f, -0.304040f, -0.302037f, -0.256574f, -0.214390f, -0.203471f, -0.218826f, -0.311599f, -0.486943f, -0.678994f, -0.835285f, -0.940315f, -0.921957f, -0.799154f, -0.639147f, -0.480407f, -0.377234f, -0.343273f, -0.307627f, -0.222751f, -0.159117f, -0.158666f, -0.212256f, -0.329036f, -0.491735f, -0.660045f, -0.807917f, -0.921650f, -0.907276f, -0.780465f, -0.632208f, -0.502719f, -0.433911f, -0.415476f, -0.361867f, -0.240689f, -0.147381f, -0.145482f, -0.219541f, -0.336864f, -0.473741f, -0.623368f, -0.774154f, -0.902849f, -0.892952f, -0.756865f, -0.614621f, -0.513368f, -0.482323f, -0.486461f, -0.425134f, -0.273271f, -0.150831f, -0.155855f, -0.247617f, -0.346212f, -0.446681f, -0.583585f, -0.746094f, -0.891410f, -0.882139f, -0.732079f, -0.584557f, -0.497656f, -0.495260f, -0.520887f, -0.456178f, -0.281254f, -0.146857f, -0.176109f, -0.277721f, -0.342943f, -0.412910f, -0.553466f, -0.735616f, -0.892581f, -0.884290f, -0.723355f, -0.555796f, -0.455508f, -0.457134f, -0.494596f, -0.429073f, -0.243240f, -0.120130f, -0.175485f, -0.267951f, -0.309096f, -0.384392f, -0.551658f, -0.751314f, -0.905722f, -0.904178f, -0.750421f, -0.561800f, -0.415189f, -0.370504f, -0.380549f, -0.302090f, -0.115448f, -0.020368f, -0.102571f, -0.197158f, -0.266168f, -0.398159f, -0.600433f, -0.796142f, -0.927132f, -0.935588f, -0.816488f, -0.636414f, -0.441797f, -0.301240f, -0.213756f, -0.076640f, 0.115187f, 0.166159f, 0.024651f, -0.141095f, -0.303593f, -0.503046f, -0.706664f, -0.860318f, -0.950807f, -0.967534f, -0.900372f, -0.776626f, -0.597046f, -0.395850f, -0.199060f, 0.025453f, 0.235162f, 0.235608f, 0.005348f, -0.262960f, -0.495507f, -0.691878f, -0.836109f, -0.924089f, -0.972632f, -0.989307f, -0.964932f, -0.913203f, -0.820483f, -0.683147f, -0.506932f, -0.296504f, -0.119810f, -0.139152f, -0.359603f, -0.598772f, -0.769278f, -0.876054f, -0.937036f, -0.970459f, -0.989290f, 0.000000f, }; | ^ make: *** [Makefile:49: seedot_float.o] Fehler 1

exec.txt is empty.

[swapnilsayansaha]

Can confirm I have the same error.

[ShikharJ]

I see, model_float.h is used for storing the .npy weight files into C++ format. From the error, it seems that the dimensionality of the .npy files, and the description in the input.sd for the same does not match. Can you please check?

[ChristopherKoeppen]

This is correct, the dimensions of the .npy Files and the entries in input.sd don't match. When I load the files and print the .shape with print("Z" , np.fromfile('Z.npy').shape) I get:

Z (1301,)
W (366,)
V (366,)
T (31,)

X.npy isn't even existing.

If you want to understand what happened here, I just followed the example to train the bonsai algorithm, nothing has been changed on my side.

[ShikharJ]

@affot Okay, this seems like a major issue then. I'll need time to look into it. Sorry for the inconvenience.

[ChristopherKoeppen]

No Problem, thanks a lot for taking care!

[swapnilsayansaha]

What I understood from trying to run anything apart from the USPS-10 tutorial given in the repo, even training the model for 1 epoch less (the USPS-10 fastgrnn model) causes Seedot compilation to fail (sometimes Seedot says X cannot be quantized properly, sometimes it says test accuracy object not found, sometimes it can't find quantization functions). I think a lot of things are internally hard-coded to work for the specific example given. Maybe a more general tutorial on what things to modify for a custom model and a custom dataset might help.

While I did manage to solve the matrix dimension issues later after some fiddling, the compilation still failed during converting the model to .h files.