global singleton streamHolder problems

[coolbei]

for now, streamHolder is a global singleton. And all the primitives share one mkldnn stream. It has 2 problems:

1. not thread-safe. Even when a net is created and used in only one thread. In other words, Two different net in two thread would not be thread safe.
2. every forward/backward, the mkldnn primitives submit itself into the stream. so the stream become bigger and bigger, the performace degrade. But if one primitive has its own stream, it can just do rerun.

Im wondering if there is any plan to fix the global stream problem ?

[matt-ny]

Yes, I think it’s the same as what we experienced in this issue - https://github.com/intel/caffe/issues/230

One can use MKL2017 instead of MKLDNN, to work around it, but a thread safe MKLDNN would be better.