FuseAI
FuseAI copied to clipboard
can use Qwen1.5-7B-Chat ?
exec readme bash Pairwise Knowledge Fusion
FuseLLM/FuseChat/train/trainer.py", line 121, in compute_loss
if self.args.distill_loss_type == "ce": loss_lm = cross_entropy(input=outputs["logits"].view(-1, vocab_size), target=target_dist.view(-1, vocab_size), reduction="none").view(batch_size, -1) # (bs, seq_len)
RuntimeError: shape '[-1, 151936]' is invalid for input of size 77642752