Adrian-1234

On a Multi GPU A100 system: $ cat CONFIG_FILE.py from colossalai.amp import AMP_TYPE SEQ_LENGTH = 512 BATCH_SIZE = 8 NUM_EPOCHS = 10 WARMUP_EPOCHS = 1 parallel = dict( tensor=dict(mode="1d", size=4),...

Setting --max-tokens produces an error no matter what the value

1

$python inference/bot.py --model togethercomputer/Pythia-Chat-Base-7B --max-tokens 128 Loading togethercomputer/Pythia-Chat-Base-7B to cuda:0... Loading checkpoint shards: 100%|????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????| 2/2 [00:06>> hi Traceback (most recent call last): File "Togethercomputer-GPT-NeoXT-Chat-Base-20B/OpenChatKit-main/inference/bot.py", line 269, in main() File "Togethercomputer-GPT-NeoXT-Chat-Base-20B/OpenChatKit-main/inference/bot.py",...