One
One
Thanks! We're now actively fixing it. BTW we're using OpenAI API protocol to communicate with openchat inference server. If you're curious, please refer to https://github.com/imoneoi/openchat/issues/78#issuecomment-1813878280
Are you running the command in the root directory of the repo? The working directory should be the root directory.
Are you using A100 80GB or 40GB?
Which dataset are you using?
With 8xA100 80GB + OpenChat 3.2 dataset, we can finish training in 10 hours. The 6K GPT4 (v1) can finish in 1 hour.
I finished training on this dataset in 10hrs.Try to tune hyperparameters such as batch size, ZeRO stage 2/3, CPU offload, etc.?
A100 40GB may be slower because of smaller batch / CPU offloading etc.
@vikrantrathore Thanks for your detailed answer! BTW, to use the provided openchat server with tensor parallel over multiple GPUs, you can set the tensor parallel argument, e.g. ``` # N...
The OpenChat server is compatible with OpenAI API protocol. You can use `openai` package with `OPENAI_API_BASE` environment variable set to the server address of OpenChat.
It's created by the following script: https://github.com/imoneoi/openchat/blob/master/ochat/scripts/hf_add_tokens.py We'll document it in the README.