InternEvo icon indicating copy to clipboard operation
InternEvo copied to clipboard

Published 20 hours ago • verified publisher icon InternLM

add use_fp32_logits flag

Open KimmiShi opened this issue 10 months ago • 0 comments
trafficstars

use bf16 logits for loss :

loss = dict(
    label_smoothing=0, op_type='flash_vocab_parallel'
)
use_fp32_logits = False

by default use_fp32_logits is True, no BC-break.

KimmiShi avatar Dec 20 '24 05:12 KimmiShi