patrickvonplaten
BERT2BERT for CNN/Dailymail jupyter notebook Error in instantiate trainer
These block code lines in BERT2BERT for CNN/Dailymail jupyter notebook
set training arguments - these params are not really tuned, feel free to change
training_args = Seq2SeqTrainingArguments( output_dir="./", evaluation_strategy="steps", per_device_train_batch_size=batch_size, per_device_eval_batch_size=batch_size, predict_with_generate=True, logging_steps=2, # set to 1000 for full training save_steps=16, # set to 500 for full training eval_steps=4, # set to 8000 for full training warmup_steps=1, # set to 2000 for full training max_steps=16, # delete for full training overwrite_output_dir=True, save_total_limit=3, fp16=True, )
instantiate trainer
trainer = Seq2SeqTrainer( model=bert2bert, tokenizer=tokenizer, args=training_args, compute_metrics=compute_metrics, train_dataset=train_data, eval_dataset=val_data, ) trainer.train()
set training arguments - these params are not really tuned, feel free to change
training_args = Seq2SeqTrainingArguments( output_dir="./", evaluation_strategy="steps", per_device_train_batch_size=batch_size, per_device_eval_batch_size=batch_size, predict_with_generate=True, logging_steps=2, # set to 1000 for full training save_steps=16, # set to 500 for full training eval_steps=4, # set to 8000 for full training warmup_steps=1, # set to 2000 for full training max_steps=16, # delete for full training overwrite_output_dir=True, save_total_limit=3, fp16=True, )
TypeError: BertModel.forward() got an unexpected keyword argument 'num_items_in_batch' Even I use generative AI to response but it couldn't fix this much
