southkorea2013

> FWIW changing the function calling example code (from https://github.com/abetlen/llama-cpp-python/blob/main/examples/notebooks/Functions.ipynb ) from a role of "tool" to that of "function": > > messages.append( { "tool_call_id": tool_call.id, #"role": "tool", "role": "function",...

> > both of them cannot get the results as expected > > Either approach is quite reasonable and should work. What happens when you try? I suspect it is...

> > both of them cannot get the results as expected > > Either approach is quite reasonable and should work. What happens when you try? For example I use...

Hi Prince, My testing Machine is: M3 Max 128G ram. Thanks, Nan

> I would recommend: > > 1. Trying 8bit or 4bit quants. > 2. Trying the 2B version. > 3. Or lowering the resolution further to 512 or 224 Ok,...