guidance

guidance copied to clipboard

I'm on Python 3.11 ``` pip3 install guidance==0.0.42 ``` Then a simple script with ``` import guidance ``` Gets error ``` Traceback (most recent call last): File "scrubbed...", line 1,...

evanmays

Comparison with Langchain?

4

I skimmed the documentation and notebooks, and I found some similarities between guidance and langchain. Can you please explain the reasons one should use guidance over langchain? More importantly, some...

ibehnam

add quantization parameters for transformers models

6

This adds the `torch_dtype` and `load_in_8bit` quantization parameters for `transformers` models that allow loading larger models with lower VRAM requirements. The Llama and MPT subclasses load the model directly, so...

jquesnelle

Add timeout parameter in requests post, with its value set to 10. This is to handle the situation when the server has not issued a response for 10 seconds, i.e....

satarupaguha11

Bitsandbytes

3

Can there be a way to load a transformer from huggingface in bits and bytes. That could make the model loading easier. I might add this after work but it...

grantCelley

HuggingFace -> Hugging Face

eltociear

Hello, Like #11 , so I suppose the resolution will be similar. Creating this for future reference for other searches. Pure intuition, Streamlit spawns worker threads to run the Python...

andfanilo

Pattern guides/regex patterns don't seem to have quite the impact one would expect ```python from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Neko-Institute-of-Science/LLaMA-7B-HF") model = AutoModelForCausalLM.from_pretrained("Neko-Institute-of-Science/LLaMA-7B-HF") llama = guidance.llms.Transformers(model=model, tokenizer=tokenizer, device=5)...

dwahdany

Running guidance within Gradio

2

Hi, I'm trying to run guidance within a Gradio app. I'm not sure about the async framework behind Gradio but it seems to be AnyIO and is using some worker...

HaukurPall

Cannot initialize AzureOpenAI llm

5

A minor bug is causing the AzureOpenAI llm `__init__` to fail. Steps to reproduce: ```python import guidance guidance.llms.AzureOpenAI("gpt-3.5-turbo") ``` Error message: ``` --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Cell...

kblissett