Onebula

提供对HuggingFace的Transformers的支持

3

HuggingFace的Transformers上提供了大量SOTA的NLP预训练模型，包括很多先进的tiny模型。 可以作为fastNLP的embeddings导入（类似已有的BERT），但是目前没有办法直接使用，可否提供相应的API呢？ 写了一段代码尝试了Albert的导入，下游任务使用了BertForSentenceMatching： ``` from torch import nn from transformers import BertTokenizer, AlbertModel class AlbertEmbedding(nn.Module): def __init__(self, tokenizer, model): super(AlbertEmbedding, self).__init__() self.tokenizer = tokenizer self.model = model self.model.include_cls_sep = True...

0.5.5版本与0.6.0版本的bertembedding在自定义vocab后的参数量大小不同

1

代码完全一致，都是自定义了vocab。 ``` with open('checkpoints/vocab_2020-10-09-23-11-21.pickle', 'rb') as f: vocab = pickle.load(f) tst_df = pd.read_csv('data/xxx.csv') tst_data = DataSet(tst_df.to_dict(orient='list')) tst_data.apply(lambda x: list(transform(x['event1'], f2h=False, fb=False)) + ['[SEP]'] + list(transform(x['event2'], f2h=False, fb=False)), new_field_name='words', is_input=True) tst_data.apply(lambda...

Update expire time for specific entrys

2

What I need is to add expire time for specific keys. Currently cacheout provides api "set" to create new entry or update existing entry. So a simple way is: ```...

Any experiments or plans on Chinese related tasks?

1

tasks like CLUE (https://www.cluebenchmarks.com/)

report to wandb能自动记录本项目里新增的参数么？例如stage、dataset、lora_rank、cutoff_len这些，暂时没看到有上报

3

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction RT ### Expected behavior _No response_ ### System Info _No response_ ### Others _No...

SPIN == DPO in self-iteration?

6

The following part in the paper explains the difference of SPIN and DPO. It claims that DPO improve the model using **instance** level information while SPIN are on the **distribution**...