Yifan Zhang
Results
3
comments of
Yifan Zhang
I think the problem may be due to the version of vLLM or something.
The guidance library might have some caching mechanism for multiple queries of the same context, we suggest you to run it on A100-80GB.
You may try using alternative libraries, though the prompt may need adjustment for compatibility with different models and libraries, and the results may vary.