Charles Srisuwananukorn

That's a typo in the README. I'll put a fix up in a moment. It should be: ``` mkdir huggingface_models \ && python tools/convert_to_hf_gptneox.py \ --ckpt-path model_ckpts/GPT-Neo-XT-Chat-Base-20B/checkpoint_5 \ --save-path huggingface_models/GPT-NeoXT-Chat-Base-20B...

The README has _another_ typo. Run this from the root of the repo: ``` mkdir huggingface_models \ && python tools/convert_to_hf_gptneox.py \ --ckpt-path model_ckpts/GPT-Neo-XT-Chat-Base-20B/checkpoint_5 \ --save-path huggingface_models/GPT-NeoXT-Chat-Base-20B \ --n-stages 8 \...

Also, make sure to update the path to the checkpoint (the `--ckpt-path` flag) to point at your desired checkpoint.

@LorrinWWW can give better advice than I can. But I'll do my best: * The `training/finetune_GPT-NeoXT-Chat-Base-20B.sh` script saves checkpoints to the `model_ckpts/GPT-NeoXT-Chat-Base-20B` directory during training. * The script, by default,...

@davismartens, would you like to join our Discord server? Here's an invite link: https://discord.gg/9Rk6sSeWEG.

Python 3.10 actually introduced grouping context managers with parentheses. See [the documentation for the with statement](https://docs.python.org/3.10/reference/compound_stmts.html#the-with-statement). What version of Python are you using?

This repo definitely requires python3 even with this patch.

Thank you for the PR!

@zhangce, what do you think?

Thanks for the feature request. This is a great idea. Will put it on the roadmap.