liuyang148 issues

Repositories
Issues
Comments

Results 2 issues of


                                            liuyang148

What's the difference of bigbr_base and bigbr_base_tf2 at the gs://bigbird-transformer/pretrain ?

I found there are two bigbr_base pretrain weights at [Google Cloud Storage Bucket](https://console.cloud.google.com/storage/browser/bigbird-transformer), what is the difference? And I have checked that their word embeddings are different by [this script](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/inspect_checkpoint.py),...

reproduce arxiv classification task

We try to reproduce arxiv task with `f1 92` as shown in the paper, we are using default hyperparameters defined in `bigbird/classifier/base_size.sh`, pretrained checkpoint [here](https://storage.googleapis.com/bigbird-transformer/pretrain/bigbr_base), but with `batch size =...