Fox Buchele comments

Repositories
Issues
Comments

Results 3 comments of


                                            Fox Buchele

I use python's llama-cpp package to run the code. There is a cuda environment and the contents of llama.cpp (compiled), but I still cannot use the GPU.

A few weeks ago they changed the instructions (and the environment variable) needed to use CUDA. It used to be '-DLLAMA_CUBLAS=ON' and now it's '-DLLAMA_CUDA=on'. Try uninstallying llama-cpp-python, adding the...

Not selecting the tokens with the highest probabilities with temperature 0

I stumbled upon this issue while debugging a separate issue in a different repository. After updating several packages, including Guidance (which depends on llama-cpp-python) I noticed my Mistral Instruct responses...

Not passing the arguments for the tool use.

Does it behave any differently if you add the @guidance decorator to your tools? Everything else looks fine to me on a first glance.