rorycaputo

Non-descipt "'BatchLogitsProcessor' object is not iterable" error while running llama.cpp

1

The model works when I use llama-cpp-python locally, but when I try hitting my Inference Process I'm getting error logs and no use-able response. Wondering if anyone can help me...