UER-py icon indicating copy to clipboard operation
UER-py copied to clipboard

Requirement for the combination of classification + MLM target

Open Embedding opened this issue 3 years ago • 0 comments

It is often the case that a piece of text is attached with a label, e.g. the comments with the ratings on restaurant. It is beneficial to use the joint training of classification and MLM targets on this type of corpus. Could you add classification+MLM model in the project?

Maybe if we use the following the options to construct the above model:

--config_path models/bert/base_config.json --data_processor cls_mlm --target cls mlm

Embedding avatar Feb 24 '22 17:02 Embedding