Chenhao Wu
Chenhao Wu
same issue here
I was thinking about using batch inference with ml too, but I get this wired question that when running in batch inference mode, the actual time used is far more...
I have seen issues in other framework reporting about the high memory usage, i guess it might help a lot if this issue solved
also, can you share the scripts you used in testing tokio & other framework?