Zixuan Zhang comments

Results 5 comments of


                                            Zixuan Zhang

Longformer EncoderDecoder (LED)-Large model finetuning for summarization results in </s><s><s><s><s><s><s><s><s><s><s>... output

I had the same issue. `allenai/led-base-16384` works well but `allenai/led-large-16384` and `allenai/PRIMERA` simply generates `""` after about a few hundreds steps of training.

Longformer EncoderDecoder (LED)-Large model finetuning for summarization results in </s><s><s><s><s><s><s><s><s><s><s>... output

I assume that it is an error in the `generate` method, since the training loss curves for the `base` and `large` models look really similar and both of them are...

Problem loading pretrained model

Hi, sorry for the late reply. This error usually comes out when the model path is incorrect. Can you double-check to make sure the path to the model is correct?...

Training details

Hi, sorry for the late reply. We provided our model checkpoints trained on ACE-05E and also released the config file to make it easier to reproduce the results. We will...

train.py OOM on TPUv3-8

> ``` > 1 Successfully loaded and sharded model parameters! > 2 0%| | 0/155 [00:00