can't run llama-2-7b-hf even though I'm using use_auth_token

[brando90]

Error:

-- Get HuggingFace LLaMA index LLM
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:07<00:00,  3.56s/it]
Traceback (most recent call last):
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 261, in hf_raise_for_status
    response.raise_for_status()
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-hf/resolve/main/tokenizer_config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/transformers/utils/hub.py", line 417, in cached_file
    resolved_file = hf_hub_download(
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1195, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1541, in get_hf_file_metadata
    hf_raise_for_status(r)
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 293, in hf_raise_for_status
    raise RepositoryNotFoundError(message, response) from e
huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-64b70d44-24ec86d03e68830022d37425;109c9108-722e-401e-b2de-552f182609a6)

Repository Not Found for url: https://huggingface.co/meta-llama/Llama-2-7b-hf/resolve/main/tokenizer_config.json.
Please make sure you specified the correct `repo_id` and `repo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/lfs/ampere1/0/brando9/massive-autoformalization-maf/maf-src/data_utils/informal_textbook_2_informal_data_frame.py", line 261, in <module>
    textbook_txt_2_maf_informal_data_frame()
  File "/lfs/ampere1/0/brando9/massive-autoformalization-maf/maf-src/data_utils/informal_textbook_2_informal_data_frame.py", line 162, in textbook_txt_2_maf_informal_data_frame
    llm = HuggingFaceLLM(
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/llama_index/llms/huggingface.py", line 64, in __init__
    self.tokenizer = tokenizer or AutoTokenizer.from_pretrained(
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 643, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 487, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/lfs/ampere1/0/brando9/miniconda/envs/maf/lib/python3.10/site-packages/transformers/utils/hub.py", line 433, in cached_file
    raise EnvironmentError(
OSError: meta-llama/Llama-2-7b-hf is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.

[Daryl149]

Try my version otherwise, just converted it, public repo: https://huggingface.co/daryl149/llama-2-7b-chat-hf

[brando90]

seems that I just need to wait for the official HF permission not only metas?

[jasonsheinkopf]

I have been granted access Gated model You have been granted access to this model But get the same error.

I created a new 'read' access token to use.

Do I need to use a specific access token or can I just create one?

[l294265421]

the same problem

[JaktensTid]

I have the same problem when I just try to clone repo from hugging face using git clone

[GitMeAI]

Having similar issues.

OSError: llama-2-7b.ggmlv3.q2_K.bin is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo with use_auth_token or log in with huggingface-cli login and pass use_auth_token=True.

[romilgoel]

I was also getting the same issue. It worked for me. Here are the steps that i followed :

1. Get approval from Meta
2. Get approval from HF
3. Create a read token from here : https://huggingface.co/settings/tokens
4. pip install transformers
5. execute huggingface-cli login and provide read token
6. Execute your code. It should work fine.

[drorata]

I'm trying to follow this tutorial and I failed at the:

tokenizer = AutoTokenizer.from_pretrained(model)

step. I ran huggingface-cli login in the shell and then tried to run the code from the tutorial (either as a script or interactively in a notebook). In both cases I get the error:

OSError: meta-llama/Llama-2-7b-chat-hf is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.

I guess I'm missing the (1) step in @romilgoel's answer. Can you give some hints how to do it?

N.B.

FWIW, I opened https://huggingface.co/meta-llama/Llama-2-7b-chat-hf and there was a button to click. I'm now waiting :)

N.B. 2

Yep, that's probably what I was missing. I ran into another problem (ValueError: Could not load model meta-llama/Llama-2-7b-chat-hf with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.llama.modeling_llama.LlamaForCausalLM'>).), but that's a different story probably.

[inesdmu]

Hi, I am having a similar problem:

%pip install transformers
%pip install accelerate
!pip install huggingface-hub==0.14.1

!huggingface-cli login --token "my_token"

from transformers import AutoTokenizer
import transformers
import torch

model = "meta-llama/Llama-2-7b-chat-hf"

tokenizer = AutoTokenizer.from_pretrained(model)

I am however getting the following error 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json

I got both the Meta and the HF accesses granted, this token corresponds to the account having the access granted.

Any idea where this could come from?

[jianyinglangaws]

I got a similar error too. I got approval from Meta and hugging face and provided token access through huggingface-cli login.

OSError: meta-llama/Llama-2-7b-hf is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.

[bnicholl]

I am also getting the same error with approval from Meta and hugging face.

[Jasonli1997]

Yeah I'm also getting the same error with approval from Meta and using the access token from Huggingface and setting use_auth_token = True

[logancyang]

Also getting the same 401 error with approval from both meta and hf. Went thru these steps still no luck

I was also getting the same issue. It worked for me. Here are the steps that i followed :

1. Get approval from Meta
2. Get approval from HF
3. Create a read token from here : https://huggingface.co/settings/tokens
4. pip install transformers
5. execute huggingface-cli login and provide read token
6. Execute your code. It should work fine.

[dylanxia2017]

I was also getting the same issue. It worked for me. Here are the steps that i followed :

1. Get approval from Meta
2. Get approval from HF
3. Create a read token from here : https://huggingface.co/settings/tokens
4. pip install transformers
5. execute huggingface-cli login and provide read token
6. Execute your code. It should work fine.

this doesn't work on my case

[Jasonli1997]

Yeah I'm also getting the same error with approval from Meta and using the access token from Huggingface and setting use_auth_token = True

I was able to get everything running after downloading the Huggingface repo with git-lfs

[SimasJan]

Try using a different provider.

example:

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("togethercomputer/LLaMA-2-7B-32K")
model = AutoModelForCausalLM.from_pretrained("togethercomputer/LLaMA-2-7B-32K")

[alinemati-uwm]

Try using a different provider.

example:

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("togethercomputer/LLaMA-2-7B-32K")
model = AutoModelForCausalLM.from_pretrained("togethercomputer/LLaMA-2-7B-32K")

checked. not working.

[MaratZakirov]

I was also getting the same issue. It worked for me. Here are the steps that i followed :

1. Get approval from Meta
2. Get approval from HF
3. Create a read token from here : https://huggingface.co/settings/tokens
4. pip install transformers
5. execute huggingface-cli login and provide read token
6. Execute your code. It should work fine.

Can download all the files but code still fails

[drorata]

@MaratZakirov Check out this thread

[dakshbhatnagar]

I was also getting the same issue. It worked for me. Here are the steps that i followed :

1. Get approval from Meta
2. Get approval from HF
3. Create a read token from here : https://huggingface.co/settings/tokens
4. pip install transformers
5. execute huggingface-cli login and provide read token
6. Execute your code. It should work fine.

How to get approval @romilgoel from HF. Facebook I guess also haven't shared the access with me. I get the below error

meta-llama/Llama-2-7b-chat-hf is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo with use_auth_tokenor log in withhuggingface-cli loginand passuse_auth_token=True.

[chaudharynitin]

I am also getting below error and unable to fix

Repository Not Found for url: https://huggingface.co/api/models/llama-2-7b-chat.ggmlv3.q4_0.bin/revision/main. Please make sure you specified the correct repo_id and repo_type. If you are trying to access a private or gated repo, make sure you are authenticated.

[jocelin]

for this error

ValueError: Could not load model meta-llama/Llama-2-7b-chat-hf with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.llama.modeling_llama.LlamaForCausalLM'>).)

I have been able to resolve it using below script

model_name = "meta-llama/Llama-2-7b-chat-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.float16,
    device_map="auto",
)

However, running with AutoModelForCausalLM directly will lead to another issue around Xformers (xformers is not installed correcly). https://github.com/huggingface/transformers/issues/24903

To fix xformers issue, apply the changes in https://github.com/huggingface/transformers/pull/24960 would work. I changed the Pipfile to load transformers from git (since the PR is not yet released) and can get llama2 working after

transformers = { git = "https://github.com/huggingface/transformers.git@main" }

[margish100]

Error

10:59:35

404 Client Error. (Request ID: Root=1-64d3243f-4b364ff52499ba15050bc73c)

Repository Not Found for url: https://huggingface.co/api/models/llama-2-7b-chat.ggmlv3.q8_0.bin/revision/main. Please make sure you specified the correct repo_id and repo_type.Error

10:59:35

404 Client Error. (Request ID: Root=1-64d3243f-4b364ff52499ba15050bc73c)

Repository Not Found for url: https://huggingface.co/api/models/llama-2-7b-chat.ggmlv3.q8_0.bin/revision/main. Please make sure you specified the correct repo_id and repo_type. If you are trying to access a private or gated repo, make sure you are authenticated. If you are trying to access a private or gated repo, make sure you are authenticated.

Auth token i have set but still i got same error and how to get access from HF and meta

[MustafaAlahmid]

this worked for me :

change model name in adapter_config.json to "NousResearch/Llama-2-7b-hf" to use non gated llama2 models

[puneethegde]

Run this !huggingface-cli login

Login using your token then run this

!pip install huggingface_hub

Solved for me!

[jiafuzha]

set use_auth_token to your actual token. it worked for me.

[realliyifei]

Try my version otherwise, just converted it, public repo: huggingface.co/daryl149/llama-2-7b-chat-hf

I am trying your model as the workaround.

@Daryl149 Is your daryl149/llama-2-7b-hf exactly the same as meta-llama/Llama-2-7b ? (in which ‘hf’ stands for hugging face?)

[gkcng]

Same as jiafuzha, both logging in via huggingface-cli login or setting use_auth_token works for me, after approval from both Meta and HF, then created a HF Token.

pretrained_name_or_path='meta-llama/Llama-2-7b-hf' 

model = transformers.AutoModelForCausalLM.from_pretrained(
    pretrained_name_or_path,
    trust_remote_code="true",
    torch_dtype=fp_type,
    device_map= None,
    # token=HF_TOKEN,
    use_auth_token=HF_TOKEN
)

tokenizer = AutoTokenizer.from_pretrained(
    pretrained_name_or_path, 
    trust_remote_code="true", 
    padding_side="left",
    # token=HF_TOKEN, 
    use_auth_token=HF_TOKEN
)

NB: The only annoying thing was getting warning messages saying use_auth_token is deprecated and to use token instead, but when I do both calls errored out.

[tcapelle]

Lol I was using: meta-llama/Llama-2-7B-hf instead of meta-llama/Llama-2-7b-hf...

[karan842]

I think we have to request to the Meta to use this model.