JIA

RuntimeError: expected scalar type Half but found Float

2

When I run `CUDA_VISIBLE_DEVICES=0,1 torchrun --standalone --nnodes=1 --nproc-per-node=2 cli_demo_sat.py --from_pretrained cogcom-base-17b --local_tokenizer tokenizer --english --fp16 (--quant 4)`, and test certain pictures, 50% of them will lead to exception, so I...

Questions about Hardware requirement

2

Excuse me, but when the model inference on 1 * RTX4090, running `python cli_demo_sat.py --from_pretrained cogcom-base-17b --local_tokenizer tokenizer --english --quant 4`, the output will be CUDA out of memory. I...