dolly
dolly copied to clipboard
databrickslabs
Running the code without databricks
Hi in the train_dolly.py, a lot of MAGIC commands were there which is used in databricks notebooks. Do we need to run those commands separately if we are not using the databricks framework.
If yes, can you also suggest how/what to modify this to run on a single/double GPU as I have access to those.
Yes, you need to run those commands. They're largely just shell commands, and you can skip (for example) the tensorboard integration.
@matthayes I wonder if we could replace the .py files that are notebooks with .ipynb, as Databricks will readily understand those too.
Do I need to make some changes to run on single/double GPUs instead of the Apache cluster as mentioned in the readme?
Start a 12.2 LTS ML (includes Apache Spark 3.3.2, GPU, Scala 2.12) single-node cluster with node type having 8 A100 GPUs (e.g. Standard_ND96asr_v4 or p4d.24xlarge).
Ohh I misunderstood this.
Hey, I have transformed the databricks notebook into shell commands at https://github.com/youkaichao/dolly#train-the-model . Maybe it can help :)
Hey, I have transformed the databricks notebook into shell commands at https://github.com/youkaichao/dolly#train-the-model . Maybe it can help :)
great works! this make a pr would be better.
Hey, I have transformed the databricks notebook into shell commands at youkaichao/dolly#train-the-model . Maybe it can help :)
great works! this make a pr would be better.
Not sure if the authors like it. It seems the databricks notebook is their internal format.
Hey, I have transformed the databricks notebook into shell commands at https://github.com/youkaichao/dolly#train-the-model . Maybe it can help :)
Great work, btw , small typo : the following shell cmd in your "Train the model" section needs to be corrected-
export deepspeed_config = `pwd`/config/ds_z3_bf16_config.json
to (no spaces around the equal sign)
export deepspeed_config=pwd`/config/ds_z3_bf16_config.json
Yes, you need to run those commands. They're largely just shell commands, and you can skip (for example) the tensorboard integration.
@matthayes I wonder if we could replace the .py files that are notebooks with .ipynb, as Databricks will readily understand those too.
"Wishful thinking" question: Would this model run on a single GPU (Nvidia GTX 1080, 8GB RAM ) home machine with 32 GB RAM and AMD Ryzen 600X 6-Core Processor 4.20 GHz?
I'm sure everyone finds that just fine - it's open source, it's just an example you can tweak, there is no "internal Databricks format", it's just .py or .ipynb files. The idea was to give you a notebook but it's just a slug of python and shell code, however you want to run it.
This was targeted at 8 x A100 40GB so that it runs in more like 30 mins. Someone on another thread tweaked deepspeed for 32GB V100s. I think 8GB is too little, or would take far too long with enough deepspeed tweaking if it does. Per the blog, you can just e-mail [email protected] for weights - not public here because they are based on data with a non-commercial license. Or I see people have already just posted then on Hugging Face, you can grab those without training if you're sure the non-commercial data source is OK. Or just run it in the cloud and grab the weights, it's not expensive.
I wonder if we could replace the .py files that are notebooks with .ipynb, as Databricks will readily understand those too.
I prefer to keep it as .py files as it's easier to view and edit in an IDE.
Yes. You may just need to adapt the code in the .py file a little bit as it's notebook code, which can execute things like shell commands too. Pull out the apt-get install commands and so on at the top and execute them first separately (or otherwise ensure those CUDA packages are available). The rest should be just Python code you can run.
@srowen Can you please share the step-by-step guid to running it on CPU machine? My machine config is: OS: Ubuntu 18.04.6 LTS RAM: 58 GB CPU(s): 16 Python version: Python 3.8.16
