ggml-org
support `--hf-token` param in addition of `--hf-repo`
Motivation
Models with non standard licence requires an opt-in on HF and the download requires a read token if the model is gated.
Proposal
- Add an
--hf-tokenparam incommonand add it in the url built with--hf-repoand--hf-fileinllama_load_model_from_hf.
Hey, can I get assigned to solve this problem? I will be glad to contribute to it.
@TheBongStack reading the issue again, I think it is easier to pass the token until loadmodel_from_url and add it as Authorization Bearer header.
Hi @TheBongStack, are you still working on this issue ?
Hey, Yes I am still working on this issue and have come up with an incomplete solution. I apologize as it may take some because I am new to open source contributing and its going to be my first PR submission to a large project.
I am just confused should I just create and return the built string value or make the HTTPS request.
No worries, take your time.
You need to concatenate hf_token in the url just where the hf_repo is used:
https://github.com/ggerganov/llama.cpp/blob/master/common/common.cpp#L2183
The token must be before the hostname prefixed with an @
@TheBongStack reading the issue again, I think it is easier to pass the token until loadmodel_from_url and add it as Authorization Bearer header.
I added an if block to accept --hf-token or -hft parameter and added the passed argument to params.hf_token Now, when I follow up the function calls, I see load_model_from_url function call with a curl object and url and path arguments.
The thing I am most confused about is in llama_load_from_url where do I set the authorization headers before making the call to accept the params.hf_token variable. Also I think we have to pass the params object too toacces it in the function
I yes I got it, a token cannot be passed as a user:password@host in curl url. Then modify the load_model_from_url to take a token as parameter
Hey, I added an pull request solving this issue can you review it please?