Module 'guidance' has no attribute 'llms'

[cairocoder]

When I run below: npm start -- --local -o unminified.js app.min.js I got an error: AttributeError: module 'guidance' has no attribute 'llms'. Did you mean: 'llm'?

Here's the full terminal output:

Traceback (most recent call last):
  File "/Users/cairocoder/Sites/localhost/humanify/local-inference/inference-server.py", line 3, in <module>
    from rename import rename
  File "/Users/cairocoder/Sites/localhost/humanify/local-inference/rename.py", line 2, in <module>
    from guidance_config import guidance
  File "/Users/cairocoder/Sites/localhost/humanify/local-inference/guidance_config.py", line 4, in <module>
    guidance.llm = guidance.llms.Transformers(mname, device="mps", caching=False, temperature=0.5)
                   ^^^^^^^^^^^^^
AttributeError: module 'guidance' has no attribute 'llms'. Did you mean: 'llm'?

[aaronegger]

I have the same issue

[aaronegger]

Try changing it to this:

import guidance from guidance import models

mname = "meta-llama/Llama-2-7b-chat-hf" guidance.llm = models.Transformers(mname, device="mps", caching=False, temperature=0.5)

[jehna]

Fixed at v2, no Python needed anymore

[AGenchev]

which version of guidance you have ? I guess it has developed and changed its API.

[jehna]

I actually dropped guidance (and python) in v2, I'm now using gbnf grammars to force a specific output instead of using Guidance.