dspy Better Langchain integration

Better Langchain integration

Open BlueKiji77 opened this issue 4 months ago • 16 comments

For a few days now I have been trying to use these example to develop RAG solution with DSPy using models downloaded from Huggingface but the integration has GPT4 hardcoded.

I modified LangChainPredict to support a model wrapped by HuggingfacePipeline but I keep hitting blocks requiring to sift through the codebase and modify functions.

Do you have any advice/solution on how to definitevily integrate Langchain and DSPy using any model loaded from Huggingface?

Mar 01 '24 19:03 BlueKiji77

Hi @BlueKiji77 you can use HFClientTGI to support HuggingFace models and configure this to be your lm through dspy.settings(lm = ...)

Mar 01 '24 19:03 arnavsinghvi11

Hello arnavsinghvi11, I am aware of HFClientTGI but for my case I need to be able to use models locally loaded in memory. That is why I have been trying to get HuggingfacePipeline working here.

Mar 01 '24 19:03 BlueKiji77

Hi @BlueKiji77 , you can try using HFModel but this is not recommended. Feel free to push a PR to support HuggingFacePipeline in DSPy!

Mar 05 '24 22:03 arnavsinghvi11

Hi, I've also been trying to run the example code from the example, and I get stuck at this part

zeroshot_chain = (RunnablePassthrough.assign(context=retrieve) | LangChainPredict(prompt, llm) | StrOutputParser())

I get the following error AttributeError: 'FieldInfo' object has no attribute 'finalize'

It seems to point to dspy/predict/langchain.py line 80.

Has anyone had success in running that example notebook?

Mar 23 '24 06:03 antonvs88

mark, i run LLMs locally too, hitting the same issue

Mar 24 '24 06:03 lucifer2288

Is it possible to use Hugging Face Inference API for LLMs similar to HuggingFaceEndpoint LLM class in langchain? 🤔

cc: @okhat @lucifer2288 @antonvs88 @arnavsinghvi11 @BlueKiji77

Mar 29 '24 13:03 97k

Same error here!

This is my env:

[tool.poetry.dependencies]
python = "~3.11"
dspy-ai = "^2.4.0"
langchain = "^0.1.14"
langchain-openai = "^0.1.1"

Apr 03 '24 02:04 jjovalle99

I am getting the same error: AttributeError: 'FieldInfo' object has no attribute 'finalize'. Anyone solved this issue?

EDIT: using version 2.1.1 solved this issue

Apr 03 '24 20:04 kyutcho

Same issue trying to run this code (langchain/dspy) with ollama locally.

Also there are some dependencies on OAI GPT-4 in LangchainPredict ... that I would get rid of.

def _build_signature(self, template):
        gpt4T = dspy.OpenAI(model='gpt-4-1106-preview', max_tokens=4000, model_type='chat')

        with dspy.context(lm=gpt4T): parts = dspy.Predict(Template2Signature)(template=template)

Apr 06 '24 12:04 jeanbapt

Trying to run the example in Google Colab I'm getting the same error

Apr 11 '24 21:04 herbgruutz

I am getting the same error: AttributeError: 'FieldInfo' object has no attribute 'finalize'. Anyone solved this issue?

EDIT: using version 2.1.1 solved this issue

share you langchain dspy and openai version...