Local hugging face model to search over docs

[VedAustin]

Hi .. I am currently into problems where I call the LLM to search over the local docs, I get this warning which never seems to stop

Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.
...

Here is my simple code:

loader = TextLoader('state_of_the_union.txt')
documents = loader.load()

text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=100)
docs = text_splitter.split_documents(documents)

from langchain.llms import HuggingFacePipeline
from langchain.chains.question_answering import load_qa_chain

llm = HuggingFacePipeline.from_model_id(model_id='stabilityai/stablelm-base-alpha-7b', task='text-generation', device=0, model_kwargs={"temperature":0, "max_length": 1024})

query = "What did the President say about immigration?"
chain = load_qa_chain(llm, chain_type="map_reduce")
chain.run(input_documents=docs, question=query)

Currently on 1 A100 with 80GB memory.

[filippo82]

A quick search on Google shows that this is an issue related to HuggingFace and not with LangChain. See this post on StackOverflow as an example.