llm.c icon indicating copy to clipboard operation
llm.c copied to clipboard

dev/download_starter_pack.sh: adding SIGINT trap and current download…

Open Ricardicus opened this issue 1 year ago • 2 comments

The download_starter_pack.sh felt like it got stuck and I did not know why. It only displays the files that got downloaded and not the ones in progress. I also couldn't stop it with CTRL-C properly.

This adds:

  • CTRL-C (SIGINT) stopping the script
  • A display that shows what files are being downloaded/have been downloaded (see below)

It can look like this (when three files have been downloaded, and there are three left to go)

Downloading gpt2_124M.bin...
Downloading gpt2_124M_bf16.bin...
Downloading gpt2_124M_debug_state.bin...
Downloaded gpt2_tokenizer.bin to /Users/user/llm.c/dev/../gpt2_tokenizer.bin   
Downloaded tiny_shakespeare_train.bin to /Users/user/llm.c/dev/data/tinyshakespeare/tiny_shakespeare_train.bin   
Downloaded tiny_shakespeare_val.bin to /Users/user/llm.c/dev/data/tinyshakespeare/tiny_shakespeare_val.bin   

Ricardicus avatar Jun 25 '24 20:06 Ricardicus

I think this looks pretty good. I removed the clear screen at the beginning now.

Is it a little bit over-engineered? yes. But does it look good? yes.

Ricardicus avatar Jun 26 '24 19:06 Ricardicus

I also added a little sanity check for the "curl" requirement. I ran this script in the Docker image "nvidia/cuda:12.4.1-devel-ubuntu22.04" from Dockerhub and apparently it does not come shipped with curl so the script failed and the output looked messy but it looks better now.

Ricardicus avatar Jun 26 '24 19:06 Ricardicus