bigscience
bigscience copied to clipboard
bigscience-workshop
Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
Why is the value of Zero-State 0 when deepspeed is enabled in the Bloom training script? Can the Bloom model be trained and the loss curve is aligned when deepspeed...
Hello, the evaluation script of bloom-7b1 is found in the repo, is evaluation/results/tr11/scripts/run_trevalharness_7b1.slurm, but the training script of bloom-7b1 is not found. Can you share the bloom-7b1 training script? Thank...
* Big Science version: latest * Python version: 3.8.8 * Operating System: Ubuntu 20.04.5 LTS ### Description How to get train-splits.txt and valid-splits.txt at Line39 in [train/tr11-176B-ml/tr11-176B-ml.slurm](https://github.com/bigscience-workshop/bigscience/blob/master/train/tr11-176B-ml/tr11-176B-ml.slurm#L39 ). Thx. `TRAIN_DATA_PATH=$MEGATRON_DEEPSPEED_REPO/data/train-splits.txt`...
Can you please provide the files for the bias evaluation on the crowspairs dataset? The results are given in section 4.9 of the paper, but I do not see the...
we want to continue fine tuning a bloomz-7b1 model, where can we get the model checkpoints like 176B :
I noticed you evaluated the opt-175B model, how did it convert to a Megatron-Deepspeed checkpoint? I can not find a 175B huggingface transformers checkpoint. Also, I can not successfully convert...
What are the minimum requirements regarding RAM and GPU memories for performing only inferences over the [Bloom](https://huggingface.co/bigscience/bloom) model?
Metadata
Owner
Metadata
Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.