transformers-bart-pretrain

transformers-bart-pretrain copied to clipboard

First, thanks for the pretraining script! I've only really worked in PyTorch, so far, so I'm really not familiar with how things are done in TF. Could you provide an...

jbmaxwell

Error ocuured on pretraining with transformers>=4.12 like below ``` /nas/home/psj8252/personal/transformers-bart-training/venv/lib/python3.7/site-packages/tensorflow/python/distribute/one_device_strategy.py:188 run [24/426] return super(OneDeviceStrategy, self).run(fn, args, kwargs, options) /nas/home/psj8252/personal/transformers-bart-training/venv/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:1259 run return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs) /nas/home/psj8252/personal/transformers-bart-training/venv/lib/python3.7/site-packages/tensorflow/python/distribute/distribute_lib.py:2730 call_for_each_replica return self._call_for_each_replica(fn, args, kwargs)...

cosmoquester