xbert

Implementation of pre-training model loading architecture of bert and its variants with tensorflow2

Description

This is based on the Transformer architecture implemented by tf2.keras, which can quickly load the pre-trained bert model for downstream finetune training. So welcome to star and I will continue to update in the future.

Install

Temporary support：

pip install git+https://github.com/xuyingjie521/xbert.git

Features

Features that have been implemented so far：

• Load pre-training weights of bert/roberta for finetune.
• Support tf2.keras.

Pre-trained models be loaded

• Google original bert: https://github.com/google-research/bert
• Harbin Institute of Technology version roberta: https://github.com/ymcui/Chinese-BERT-wwm
• Brightmart version of roberta: https://github.com/brightmart/roberta_zh