blueoil icon indicating copy to clipboard operation
blueoil copied to clipboard

Cannot run inference again witout reboot after FPGA was stuck

Open hadusam opened this issue 4 years ago • 2 comments

When running different inferences at the same time, FPGA will be stuck and inferences are stopped. And after that, I cannot run any inference which needs FPGA, these will be stuck too. I know that the Blueoil does not support parallel inferences. I only would like to know how to reset the FPGA state without a reboot. Is there a way to reset this state?

How to reproduce this situation

It is easy, run a following command on different terminal at the same time.

python run.py -i ../../inference_test_data/raw_image.png -c ../models/meta.yaml -m ../models/lib/libdlk_fpga.so --trial 100

It will be stopped on INFO:__main__:Function _run part. And after that happened, we cannot run inference unless rebooting.

hadusam avatar May 12 '20 13:05 hadusam

@hadusam Sorry, I don't know the solution.

@tk26eng Do you know something?

iizukak avatar May 12 '20 13:05 iizukak

@iizukak @hadusam

Maybe there's no method to recover it after FPGA gets stucked. But avoiding it is more important than recovering it. We might need to implement the way to avoid multiple inference at same time at library level. That is like singleton feature.

tk26eng avatar May 15 '20 14:05 tk26eng