llama icon indicating copy to clipboard operation
llama copied to clipboard

download.sh closes without downloading anything.

Open Vermath opened this issue 1 year ago • 37 comments

I've tried poddering through a few of the tips here, but the typical tips of 'make sure the URL is correct' etc haven't had any impact.

It runs, I input the URL, I specify a model, it zips through a few lines of code, then closes the bash terminal without doing anything.

Any assistance would be helpful. Thanks.

Vermath avatar Jul 18 '23 20:07 Vermath

Same issue for me. Getting 403 forbidden when it tries to download the model files and then finishes saying it could not find any MD5 checksums.

HTTP request sent, awaiting response... 403 Forbidden 2023-07-18 13:47:06 ERROR 403: Forbidden.

Checking checksums md5sum: checklist.chk: no properly formatted MD5 checksum lines found

gamerscomplete avatar Jul 18 '23 20:07 gamerscomplete

same 403 forbidden

serkandyck avatar Jul 18 '23 20:07 serkandyck

same issue. Got the email at 14:10pm CT today.

HTTP request sent, awaiting response... 403 Forbidden 2023-07-18 20:36:32 ERROR 403: Forbidden.

Checking checksums md5sum: checklist.chk: no properly formatted MD5 checksum lines found

Any help pls?

bqcao avatar Jul 18 '23 20:07 bqcao

First 7 consolidated.pth of 70B-chat downloaded perfectly. 8th failed with 403. Can't download any models now. This was the first download with my URL. Requesting a second download URL. Let's see if that works...

iibw avatar Jul 18 '23 21:07 iibw

Same issue here. I am still waiting for the new requested url...

joshzyj avatar Jul 18 '23 21:07 joshzyj

same issue

orngeatom avatar Jul 18 '23 21:07 orngeatom

Open a command line, clone the repo, and cd into it. Then: $ chmod 755 download.sh $ ./download.sh Now, when passing the URL to the download script, make sure you're pasting an URL that begins with https://download.llamameta.net and not with https://l.facebook.com. If you copy the link address from the e-mail, you will get a https://l.facebook.com address, that's not what we want. We want the https://download.llamameta.net address, as displayed in the message body itself. Better copy-paste it as plain text through a text editor first, before passing it on to the download script. I hope this helps.

diogoff avatar Jul 18 '23 21:07 diogoff

Had the same issue. Went back to the Meta Llama page and filled in the request again. Email took about 30 seconds to come through. URL works this time.

Steps taken:

  • Delete existing llama folder
  • Reclone from git
  • Re-request email
  • Click on URL in email. This takes you to a webpage
  • The webpage is at a URL beginning download.llamameta.net/ and the header text reads: This XML file does not appear to have any style information associated with it. The document tree is shown below.
  • It displays as Error XML mapping in the space below.
  • Copy the URL from the browser URL bar and enter that in terminal when requested.
  • Success.

NB - The first time around I had right clicked and copied the URL directly from the email. On a second attempt with the same URL, I had clicked it first, and was taken to a webpage. This webpage was DIFFERENT from the one described above. Would be interested to know if people who follow the exact same steps I applied on my second attempt find success.

Edit: My experience mirrors the post by diogoff directly above this post. Only minor diff is that I used chmod +x download.sh to give execute permissions.

Yelnam avatar Jul 18 '23 21:07 Yelnam

The URL in the email beginning with https://download.llamameta.net.., if you right-click on it and copy the link then you'll get a URL beginning with https://l.facebook.com/..., this won't work.

You need to select the text of the URL and copy the text, not the link - it should begin with https://download.llamameta.net.

This worked for me (so far, still downloading).

sanity avatar Jul 18 '23 21:07 sanity

Keep getting HTTP 403 randomly.. after some retries then I hit the limit. :/

ssteo avatar Jul 18 '23 21:07 ssteo

Try using your email instead!

JOKERKING88 avatar Jul 18 '23 21:07 JOKERKING88

same thing here

smy20011 avatar Jul 18 '23 21:07 smy20011

I was getting HTTP 403 but then I sent a request access again from https://ai.meta.com/resources/models-and-libraries/llama-downloads/ now I'm downloading. Here are the steps I followed:

  1. git clone https://github.com/facebookresearch/llama
  2. cd llama
  3. chmod 755 download.sh
  4. ./download.sh

After that make sure you copied the link correctly. If you right-click on it it'll copy wrong link. Make sure you select the text of the URL and copy the text. It should begin with https://download.llamameta.net.

mustafamerttunali avatar Jul 18 '23 21:07 mustafamerttunali

Hey, I'm getting the same 403 error as many people. I've properly copied the key and all. it starts with "https://download.llamameta.net/*?Policy=eyJTdGF(....)"

dseddah avatar Jul 18 '23 22:07 dseddah

re-requesting access with another email solved it for me

asselinpaul avatar Jul 18 '23 22:07 asselinpaul

Same for me , 403 forbidden ... guess i'll try re-requesting?

I tried removing the prepended fb domain from the url but still no luck

this seems like an odd way to do it instead of just giving a key that expires ( i can understand why they don't just use email)

thewilltejeda avatar Jul 18 '23 22:07 thewilltejeda

I'm also getting 403. I verified that I'm using the correct "download.llamameta.net" url. This distribution method seems very fragile -- it was a similar fiasco last time with the original llama release.

joshvfleming avatar Jul 18 '23 22:07 joshvfleming

@thewilltejeda / @joshvfleming Try to a. remove the llama directory b. reclone the repo c. resent the request to https://ai.meta.com/resources/models-and-libraries/llama-downloads/ again to get the new link d. cd llama e. chmod 755 download.sh f. copy the url from the newest email to the ./download.sh

If you try to download all models with some undownloaded models. We could set one model to download, it is worked for me. 圖片

MIBlue119 avatar Jul 18 '23 22:07 MIBlue119

did all that everyone suggested and I am still getting this error: Downloading LICENSE and Acceptable Usage Policy ./download.sh: line 17: wget: command not found ./download.sh: line 18: wget: command not found Downloading tokenizer ./download.sh: line 21: wget: command not found ./download.sh: line 22: wget: command not found md5sum: tokenizer_checklist.chk: No such file or directory Downloading llama-2-13b-chat ./download.sh: line 52: wget: command not found ./download.sh: line 52: wget: command not found ./download.sh: line 55: wget: command not found ./download.sh: line 56: wget: command not found Checking checksums md5sum: checklist.chk: No such file or directory

this was after pip installing wget and attempting

mhussar avatar Jul 18 '23 22:07 mhussar

./download.sh: line 52: wget: command not found

@mhussar Bro, you need to install wget, lmao. This issue is for the Forbidden error.

Edit: I can also confirm that re-requesting the link solved the problem. They probably just got overwhelmed with requests and invalidated a bunch of old tokens, asking people to re-request them.

Rocamonde avatar Jul 18 '23 22:07 Rocamonde

I followed @MIBlue119's approach and it worked for me.

efraisse avatar Jul 18 '23 22:07 efraisse

Open a command line, clone the repo, and cd into it. Then: $ chmod 755 download.sh $ ./download.sh Now, when passing the URL to the download script, make sure you're pasting an URL that begins with https://download.llamameta.net and not with https://l.facebook.com. If you copy the link address from the e-mail, you will get a https://l.facebook.com address, that's not what we want. We want the https://download.llamameta.net address, as displayed in the message body itself. Better copy-paste it as plain text through a text editor first, before passing it on to the download script. I hope this helps.

IMHO, this should be dropped at the top of the README.md the short term. Also add that if your link is burnt due to 403's, to re-request one from MetaAI.

🙏

supermario-ai avatar Jul 18 '23 22:07 supermario-ai

Yep, seems to work after re-requesting the link.

joshvfleming avatar Jul 18 '23 22:07 joshvfleming

I used the exact same setup I had before and re-requested and the second link that was sent worked as intended. The problem seems to soley be on facebooks authorization system. If your download doesn't work just keep requesting a new one until it works.

gamerscomplete avatar Jul 18 '23 22:07 gamerscomplete

@Rocamonde yeah, I already installed wget and other dependencies before I posted this. That’s why I bothered to post this because it didn’t make sense that I was getting the wget error after confirming it was already installed successfully. But thanks for the mature reply.

mhussar avatar Jul 18 '23 23:07 mhussar

@mhussar well, you clearly installed it incorrectly. Either way, the problem still has nothing to do with this particular issue (or with the repo for that matter). You probably need to fix your PATH or open a new terminal window, depending on what OS you're using. Check out https://stackoverflow.com/search?q=wget+command+not+found

Rocamonde avatar Jul 18 '23 23:07 Rocamonde

@mhussar

this was after pip installing wget

In any case "pip installing wget" is not what you should do. If you're running linux you need to google how to install wget in your distribution (using apt, pacman, or whatever your distribution uses). If you're using macOS, you will probably need to install homebrew first, and then brew install wget. If you're using windows... well I guess you have to use WSL, realistically.

gattilorenz avatar Jul 19 '23 12:07 gattilorenz

Hi everyone, we fixed some issues with the download links yesterday - resetting your local directory and requesting a new link is the best way to start over unless you've already verified checksums and know exactly which files you need. We've added a notice at the top of the README to encourage people to try this - good suggestion @svpermari0!

chrismarra avatar Jul 19 '23 13:07 chrismarra

What worked for me on Windows 11, 64 bit machine.

  1. Cloned the repo
  2. Added wget.exe and md5sums.exe in C:\Program Files\Git\mingw64\bin Download md5sums Download wget If you download a zip file, extract it and use the exe file in it. The names should be wget.exe and md5sums.exe. So rename if it is downloaded as wget64.exe or something else.
  3. Opened git bash and this is what I did:
senthilvikram@iamgroot MINGW64 ~
$ cd OneDrive/Desktop/AI/test/llama/

senthilvikram@iamgroot MINGW64 ~/OneDrive/Desktop/AI/test/llama (main)
$ chmod 755 download.sh

senthilvikram@iamgroot MINGW64 ~/OneDrive/Desktop/AI/test/llama (main)
$ ./download.sh
Enter the URL from email: copy pasted the download link from the email after pasting it in notepad.

Enter the list of models to download without spaces (7B,13B,70B,7B-chat,13B-chat,70B-chat), or press Enter for all: 7B
Downloading LICENSE and Acceptable Usage Policy
--2023-08-10 12:26:15--  

. . . it starts downloading

SenthilVikram avatar Aug 10 '23 07:08 SenthilVikram

Anyone facing the below issue:

Enter the list of models to download without spaces (7B,13B,70B,7B-chat,13B-chat,70B-chat), or press Enter for all: 70B Downloading LICENSE and Acceptable Usage Policy Connecting to download.llamameta.net (52.85.234.29:443) saving to './LICENSE' LICENSE 100% || 7020 0:00:00 ETA './LICENSE' saved Connecting to download.llamameta.net (52.85.234.29:443) saving to './USE_POLICY.md' USE_POLICY.md 100% || 4766 0:00:00 ETA './USE_POLICY.md' saved Downloading tokenizer Connecting to download.llamameta.net (52.85.234.29:443) saving to './tokenizer.model' tokenizer.model 100% || 488k 0:00:00 ETA './tokenizer.model' saved Connecting to download.llamameta.net (52.85.234.29:443) saving to './tokenizer_checklist.chk' tokenizer_checklist. 100% || 50 0:00:00 ETA './tokenizer_checklist.chk' saved tokenizer.model: OK Downloading llama-2-70b seq: unknown option -- f BusyBox v1.37.0.git (2023-08-01 12:16:57 UTC)

Usage: seq [-w] [-s SEP] [FIRST [INC]] LAST

Print numbers from FIRST to LAST, in steps of INC. FIRST, INC default to 1.

    -w      Pad with leading zeros
    -s SEP  String separator

Connecting to download.llamameta.net (52.85.234.29:443) saving to './llama-2-70b/params.json' params.json 100% || 147 0:00:00 ETA './llama-2-70b/params.json' saved Connecting to download.llamameta.net (52.85.234.29:443) saving to './llama-2-70b/checklist.chk' checklist.chk 100% || 478 0:00:00 ETA './llama-2-70b/checklist.chk' saved Checking checksums md5sum: can't open 'consolidated.00.pth': No such file or directory consolidated.00.pth: FAILED md5sum: can't open 'consolidated.01.pth': No such file or directory consolidated.01.pth: FAILED md5sum: can't open 'consolidated.02.pth': No such file or directory consolidated.02.pth: FAILED md5sum: can't open 'consolidated.03.pth': No such file or directory consolidated.03.pth: FAILED md5sum: can't open 'consolidated.04.pth': No such file or directory consolidated.04.pth: FAILED md5sum: can't open 'consolidated.05.pth': No such file or directory consolidated.05.pth: FAILED md5sum: can't open 'consolidated.06.pth': No such file or directory consolidated.06.pth: FAILED md5sum: can't open 'consolidated.07.pth': No such file or directory consolidated.07.pth: FAILED params.json: OK md5sum: WARNING: 8 of 9 computed checksums did NOT match

Deepjyoti-ricky avatar Aug 16 '23 21:08 Deepjyoti-ricky