Empty API token in HuggingFaceEndpoint unless explicitly passed to Constructor

[gil-frenkel-marpai]

System Info

System: Linux Kernel: Linux version 5.15.0-1026-aws (buildd@lcy02-amd64-088) ( gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 ) Python version: 3.8.16 LangChain version: 0.0.157

Who can help?

@hwchase17

Information

• [ ] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [X] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

Upon constructing the HuggingFaceEndpoint, when the API token is set as an environment variable (not passing to the constructor), the attribute huggingface_api_token will always be empty, causing a missing token in the request headers:

from dotenv import load_dotenv
load_dotenv()

llm = HuggingFaceEndpoint(
    endpoint_url="togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1",
    task="text-generation"
)

print(llm.huggingfacehub_api_token)
print("What is the capital of France?")

Output:

None
Traceback (most recent call last):
  File "research/red_pajama.py", line 58, in <module>
    print(llm("What is the capital of France?"))
  File "/home/ubuntu/miniconda3/envs/marpai_llm/lib/python3.8/site-packages/langchain/llms/base.py", line 281, in __call__
    self.generate([prompt], stop=stop, callbacks=callbacks)
  File "/home/ubuntu/miniconda3/envs/marpai_llm/lib/python3.8/site-packages/langchain/llms/base.py", line 176, in generate
    raise e
  File "/home/ubuntu/miniconda3/envs/marpai_llm/lib/python3.8/site-packages/langchain/llms/base.py", line 170, in generate
    self._generate(prompts, stop=stop, run_manager=run_manager)
  File "/home/ubuntu/miniconda3/envs/marpai_llm/lib/python3.8/site-packages/langchain/llms/base.py", line 377, in _generate
    self._call(prompt, stop=stop, run_manager=run_manager)
  File "/home/ubuntu/miniconda3/envs/marpai_llm/lib/python3.8/site-packages/langchain/llms/huggingface_endpoint.py", line 133, in _call
    raise ValueError(
ValueError: Error raised by inference API: Authorization header is correct, but the token seems invalid

ValueError exception raised here: https://github.com/hwchase17/langchain/blob/65c95f9fb2b86cf3281f2f3939b37e71f048f741/langchain/llms/huggingface_endpoint.py#L118

Expected behavior

huggingFaceEndpoint should send a valid POST request to inference API.

[PawelFaron]

Thanks for reporting this, fixed.

[Smartking1]

How can I fix this issue

[shreshtha48]

My error is something similar to this `query="How many people did ted bundy kill?"

response=qa.run(query) print(response)`

how do i fix this

[erlebach]

Any news on this issue? I work on a Mac with M2 check and have a similar error. The last line of my stack trace is:

Error raised by inference API: Authorization header is correct, but the token seems invalid

[AnujaDilrukshiHerath]

I work on a Mac with M1 check and have a similar error i'm getting. The last line of my stack trace is:

Error raised by inference API: Authorization header is correct, but the token seems invalid

[dosubot[bot]]

Hi, @gil-frenkel-marpai,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, the issue you reported regarding the HuggingFaceEndpoint sending empty API tokens in the request headers has been resolved by PawelFaron. Other users also reported similar errors and asked for updates on the issue.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to LangChain! If you have any further questions or concerns, feel free to reach out.

I'm here to help, Dosu