OpenChatKit icon indicating copy to clipboard operation
OpenChatKit copied to clipboard

`convert_to_hf_gptneox` adds support to other neox models

Open LorrinWWW opened this issue 1 year ago • 2 comments

Previous conversion code assumes the model is neox-20b, and some config is hard-coded, i.e. 44 layers. (also added a fp16 option.)

example:

python tools/convert_to_hf_gptneox.py \
    --config-name EleutherAI/pythia-6.9b-deduped \
    --ckpt-path ${PATH_CKPTS}/checkpoint_100 --save-path pythia-6.9b-finetuned \
    --n-stages 4 --n-layer-per-stage 8 --fp16

LorrinWWW avatar Mar 21 '23 06:03 LorrinWWW

Would use (plus help with issues,) but for the lack of support for regular GPT models.

ETERNALBLUEbullrun avatar Mar 22 '23 16:03 ETERNALBLUEbullrun

We need to update the README as well. @LorrinWWW, can you do that?

csris avatar Mar 22 '23 17:03 csris

@csris Sure I will do that

LorrinWWW avatar Mar 23 '23 05:03 LorrinWWW

Done

LorrinWWW avatar Mar 23 '23 06:03 LorrinWWW

谢谢,我已收到您的来信。Thank you!I have received your letter.ありがとう!電子メールを受信しました!

814007450 avatar Mar 23 '23 06:03 814007450