OpenChatKit
OpenChatKit copied to clipboard
togethercomputer
Anyone know how to issue this exception?  I have tried use_new_zipfile_serialization=False, but it doesn't work: 
Ubuntu Ubuntu 22.04.2 LTS After downloading the model and now trying to convert: ``` (OpenChatKit) georgi@georgi-hackintosh:~/Documents/GitHub/OpenChatKit$ python3.10 tools/convert_to_hf_gptneox.py --ckpt-path model_ckpts/GPT-Neo-XT-Chat-Base-20B/checkpoint_5 --save-path huggingface_models/GPT-NeoXT-Chat-Base-20B --n-stages 8 --n-layer-per-stage 6 loading stage 0 Traceback...
**Describe the bug** The bash script to train the model does not work because of a Cupy error: ``` (OpenChatKit-Test) user@pc:~/OpenChatKit$ bash training/finetune_GPT-NeoXT-Chat-Base-20B.sh Traceback (most recent call last): File "/home/user/OpenChatKit/training/dist_clm_train.py",...
**Describe the bug** I've downloaded the corpus and the model weights, I ran the command `bash training/finetune_GPT-NeoXT-Chat-Base-20B.sh` and I got the following: https://gist.github.com/riatzukiza/0930307fc90bf940103364be2d3db5c1 **To Reproduce** Steps to reproduce the behavior:...
I think the explanation of train and fine-tune process is much few, can Can you show some specific examples of ipynb documentation reference? Many thanks!
Does it run on single nVidia RTX A4000 or do I need two or more?
Previous conversion code assumes the model is neox-20b, and some config is hard-coded, i.e. 44 layers. (also added a fp16 option.) example: ```sh python tools/convert_to_hf_gptneox.py \ --config-name EleutherAI/pythia-6.9b-deduped \ --ckpt-path...
Hello What is minimum specification to launch (but not train) it on local machine with normal speed? Thank you
# sh training/finetune_Pythia-Chat-Base-7B.sh Namespace(use_cuda=True, cuda_id=0, cuda_num=1, debug_mem=True, dist_backend='cupy_nccl', dp_backend='nccl', dist_url='tcp://127.0.0.1:7033', world_size= train_data=['./glue_dataset/data/QQP/train.tsv'], valid_data=['./glue_dataset/data/QQP/test.tsv'], tokenizer_type='BertWordPieceLowerCase', vocab_file='', train_log_backend='print', project_name='together', batch_size=32, micro_batch_size=1, lr=1e-05, num_iters=10, fp16=True, loss_scale=0, initial_loss_slreduce', gradient_accumulate_step=1, model_name='/data/app/OpenChatKit/training/../pretrained/Pythia-6.9B-deduped/EleutherAI_pythia-6.9b-deduped/', toketype='gptneox', checkpoint_path='/data/app/OpenChatKit/training/../model_ckpts/Pythia-Chat-Base-7B', task_name='/data/app/OpenChatKit/training/../data/OI_checkpoint=True, seed=42,...
