Update download.sh

[Inserian]

Changed wget to curl. Set -e to close if any downloads fail. -f with curl to close if downloads fail.

[neuhaus]

Some systems have curl, some have wget. Ideally the script would use whatever is available.

[Inserian]

Would be better if you check whether curl/wget is available on the system & then reference it later in the script, for example:

if [ -x "$(which wget)" ]; then
   download="wget"
else
  if [ -x "$(which curl)" ]; then
   download="curl -f"
  else
    echo "Error: both wget and curl are not installed."
  fi
fi

$download "https://example.com"

I like your idea better. I chose curl because its generally more compatible and faster. Will update later.

[DarkSynx]

#windows batch version

@echo off
setlocal enabledelayedexpansion

rem Copyright (c) Meta Platforms, Inc. et affiliés.
rem Ce logiciel peut être utilisé et distribué selon les termes de la licence publique générale GNU version 3.

set "PRESIGNED_URL="             rem remplacez par l'URL présignée reçue par e-mail
set "MODEL_SIZE=7B,13B,30B,65B"  rem modifiez cette liste avec les tailles de modèle que vous souhaitez télécharger
set "TARGET_FOLDER="             rem où tous les fichiers doivent être enregistrés

set "N_SHARD_DICT_7B=0"
set "N_SHARD_DICT_13B=1"
set "N_SHARD_DICT_30B=3"
set "N_SHARD_DICT_65B=7"

echo Téléchargement du tokenizer
powershell -Command "Invoke-WebRequest -Uri '%PRESIGNED_URL:/=*%/tokenizer.model%' -OutFile '%TARGET_FOLDER%/tokenizer.model'"
powershell -Command "Invoke-WebRequest -Uri '%PRESIGNED_URL:/=*%/tokenizer_checklist.chk%' -OutFile '%TARGET_FOLDER%/tokenizer_checklist.chk'"

echo Vérification des sommes de contrôle du tokenizer
cd "%TARGET_FOLDER%"
certutil -hashfile tokenizer.model MD5 | findstr /R /C:"^[A-Fa-f0-9]*$" > tokenizer_checklist_actual.chk
fc tokenizer_checklist.chk tokenizer_checklist_actual.chk > nul

if errorlevel 1 (
  echo La somme de contrôle de tokenizer.model est invalide
  exit /b 1
) else (
  echo La somme de contrôle de tokenizer.model est valide
)

for %%i in (%MODEL_SIZE%) do (
  echo Téléchargement de %%i
  mkdir "%TARGET_FOLDER%/%%i" 2> nul

  set "N_SHARD=!N_SHARD_DICT_%%i!"

  for /L %%s in (0, 1, !N_SHARD!) do (
    echo Téléchargement de %%i/consolidated.%%s.pth
    powershell -Command "Invoke-WebRequest -Uri '%PRESIGNED_URL:/=*%/%%i/consolidated.%%s.pth%' -OutFile '%TARGET_FOLDER%/%%i/consolidated.%%s.pth'"
  )

  powershell -Command "Invoke-WebRequest -Uri '%PRESIGNED_URL:/=*%/%%i/params.json%' -OutFile '%TARGET_FOLDER%/%%i/params.json'"
  powershell -Command "Invoke-WebRequest -Uri '%PRESIGNED_URL:/=*%/%%i/checklist.chk%' -OutFile '%TARGET_FOLDER%/%%i/checklist.chk'"

  echo Vérification des sommes de contrôle de %%i
  cd "%TARGET_FOLDER%/%%i"
  for /F "tokens=1,2 delims= " %%a in (checklist.chk) do (
    certutil -hashfile %%b MD5 | findstr /R /C:"^[A-Fa-f0-9]*$" > %%b_actual.chk
    fc %%a %%b_actual.chk > nul

    if errorlevel 1 (
      echo La somme de contrôle de %%b est invalide
      exit /b 1
    ) else (
      echo La somme de contrôle de %%b est valide
    )
  )
)

[ckxng]

picking this up, I am reviewing several of the outstanding PRs for download.sh

[ckxng]

This may need to have new changes merged in. This fails for me, as declare -A does not work on Mac OSX Ventura 13.5.1 with default settings, but the current main branch does not include declare -A. Merging main into this PR may help.