edwardelric1202

使用IWSLT17中-英数据集，在训练过程中BLEU持续升高，没有收敛的迹象，但模型在测试集上的泛化能力很差

1

使用的是IWSLT17中-英数据集，模型为Transformer，在训练过程中BLEU值一直在升高没有收敛，请问这是什么原因，与超参数的设置有关吗？ `INFO:tensorflow:BLEU at step 10000: 0.110296` `INFO:tensorflow:BLEU at step 20000: 0.144964` `INFO:tensorflow:BLEU at step 30000: 0.178070` `INFO:tensorflow:BLEU at step 40000: 0.198967` `INFO:tensorflow:BLEU at step 50000: 0.222250` `INFO:tensorflow:BLEU at step 60000:...

Is it possible to continue training from the last lora checkpoint?

1

Hi, I wonder that if I can continue finetune this model from the last lora checkpoint if I interrupted training process?

NCCL RuntimeError

1

After run successfully and passed several minutes, it occured this error: **RuntimeError: NCCL communicator was aborted on rank 2. Original reason for failure was: [Rank 2] Watchdog caught collective operation...

Hi, I used this code to train a dqn in Breakout-v5, but found the reward in training just reach 3.5-4, could you please give some advice of training? I wonder...

Loss computation in finetune

2

Hi, I wonder how does the training loss computed when we finetune starcoder by lora? Is the loss computation includes prompt tokens and generated tokens, or just the generated tokens?...

Hi, I found it is wield that **the eos token** in config.json of codegen2-7B model is set to 2, but in codegen I think it was 50256, is it for...