smolraccoon

Results 1 comments of smolraccoon

Empty output when running Q4_K_M quantization of Llama-3-8B-Instruct with llama-cpp-python

@JHH11 If you're running code similar to what I posted above, try deleting the `response_format` block entirely - that fixed it for me, though I still have no idea why