alpaca-lora
alpaca-lora copied to clipboard
tloen
[WIP] Dockerfile
Description
This PR adds a Dockerfile to quickly run the finetune.py script without losing sanity
Usage
Build the container
docker build -t alpaca-lora .
Run it,
Training:
docker run --gpus=all --ipc=host --shm-size 64g \
-e MICRO_BATCH_SIZE=5 \
-e BATCH_SIZE=256 \
-e GRADIENT_ACCUMULATION_STEPS=64 \
-e EPOCHS=5 \
-e LEARNING_RATE=0.001 \
-e CUTOFF_LEN=512 \
-e LORA_R=16 \
-e LORA_ALPHA=32 \
-e LORA_DROPOUT=0.1 \
-e VAL_SET_SIZE=5000 \
-e TARGET_MODULES="q_proj,v_proj,a_proj" \
-e DATA_PATH="/path/to/data.json" \
-e OUTPUT_DIR="/path/to/output" \
-v ${HOME}/.cache:/root/.cache \
-v ${PWD}/lora-alpaca:/workspace/lora-alpaca \
--rm -it alpaca-lora training.py
Generation
docker run --gpus=all --ipc=host --shm-size 64g \
-v ${HOME}/.cache:/root/.cache \
-v ${PWD}/lora-alpaca:/workspace/lora-alpaca \
-p 7860:7860 \
--rm -it alpaca-lora generate.py
Then natigate to https://localhost:7860
all good!
TODOs
- [x] add more env variables to control training
A dockerfile is really needed to clear the env-related issues. Thanks for initiating this. I have tried running this on my machine, but I'm getting this error after running docker.
@dittops thanks for trying it out, I don't see the issue. Unfortunately, till tomorrow I am travelling. Just to be sure, are you using the code from the main branch?
I'm getting this error after running docker.
Would you have more than 1 gpu by chance? Can you try exposing only one of them to Docker?
@AngainorDev Thank you. That fixed the problem. The issue is with trying to use all gpus.
Good stuff, I was dockerizing this myself, but you beat me too finishing it. I will check this out and see if there are any issues.
Changed finetune.py to use env variables, handy if you want to quickly change some finetuning parameters
Updated the doc accordingly
The docker image now only exposes python3.10, thus we can also run the generate script by
docker run --gpus=all --ipc=host --shm-size 64g \
-v ${HOME}/.cache:/root/.cache \
-v ${PWD}/lora-alpaca:/workspace/lora-alpaca \
-p 7860:7860 \
--rm -it alpaca-lora generate.py
@tloen I think this is in a good state now, fell free to add any comment :)
I'll try to get this tested in the next day or two. Good work, and thanks for hanging in with all the interface changes!
