blueoil icon indicating copy to clipboard operation
blueoil copied to clipboard

Tolerance in elf binaries should be variable as argument

Open tk26eng opened this issue 4 years ago • 3 comments

Now the tolerance to be tested in elf binaries like lm_x86.elf is fixed to 0.00001. It should be variable because the tolerance is up to network and dataset. Some developers try to choose the bigger tolerance in blue-oil/converter/templates/include/dlk_test.h and recompile it again when a calculated value and a ground truth doesn't match. We can improve efficiency if we can change the tolerance as argument like below.

./lm_x86.elf [input.npy] [output.npy] [tolerance(optional)]

tk26eng avatar Mar 18 '20 08:03 tk26eng

I want to know @primenumber 's opinion.

iizukak avatar Mar 19 '20 00:03 iizukak

@iizukak I discussed with @primenumber about this directly. This feature to change tolerance is acceptable for him. But an warning should be shown when the user changes the tolerance because we might miss a bug with a bigger tolerance setting.

tk26eng avatar Mar 23 '20 06:03 tk26eng

But an warning should be shown when the user changes the tolerance because we might miss a bug with a bigger tolerance setting.

Sound good.

iizukak avatar Mar 24 '20 00:03 iizukak