model_id remains set to 'gpt2' when creating HuggingFacePipeline from pipeline

[marcovirgolin]

On langchain==0.0.147

I get

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, 
                    model_kwargs=llm_kwargs, device=device)
hf = HuggingFacePipeline(pipeline=pipe)
print(hf.model_id)

always gives gpt2, irrespective of what model is.

[praneetreddy017]

Just looked into the huggingface_pipeline.py file I don't think there is an init function which takes care of a pipeline being passed in as such Unless, there is something else in the backend that I am not able to understand.

@hwchase17, could you have a look. If you think this is something that we can fix, I can try my hand at it

[malaccan]

yeah am getting the same, hf.model_id always defaults to "gpt2" whenever pipeline=pipe is passed in that way. looks like HuggingFacePipeline.model_id isn't correctly set on init.

simple workaround is to just add the model_id string: hf = HuggingFacePipeline(pipeline=pipe, model_id="yourmodelname")

[DesmondChoy]

Still seeing this issue as of langchain 0.0.262. Passed StableBeluga-13B into a huggingface pipeline, and then ran

hf = HuggingFacePipeline(pipeline=pipe)

but when I inspected hf, it still gave me gpt2:

HuggingFacePipeline(cache=None, verbose=False, callbacks=None, callback_manager=None, tags=None, metadata=None, pipeline=<transformers.pipelines.text_generation.TextGenerationPipeline object at 0x7fb88c703370>, model_id='gpt2', model_kwargs=None, pipeline_kwargs=None)

[myndtrust]

I am seeing the same issue with Llama-2-7b-hf. Is the correct model being used and its just the "model_id" string is hard coded in Langchain?

[praveenv]

Still seeing this issue as of Langchain 0.0.279 This is a problem, since using the HuggingFacePipeline.from_model_id approach enforces that the device value is always set (default is -1)

This interferes with the use of device_map = "auto" when trying to load the model on multiple GPUs. HuggingFace gives a warning that "both device and device_map are set, and device will be prioritized over device_map"

Does anyone have a workaround?

[dosubot[bot]]

Hi, @marcovirgolin

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, you reported an issue with the HuggingFacePipeline where the model_id remains set to 'gpt2' regardless of the specified model. Several users have confirmed experiencing the same issue and have suggested workarounds, such as manually setting the model_id string. The problem persists in the latest Langchain version, causing interference with the use of device_map = "auto" when trying to load the model on multiple GPUs. Users are seeking a workaround for this issue.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!