BERT_REL with SciBert

[mariaBio]

HI,

Thank you for your code and for adding BioBert. Tried to use your code with SciBert. For this, I created a code 3 to specify the model type in the input (just as you introduced 2 for BioBert), and added the necessary specification everywhere (following BioBert example).

When training, there is the following error:

12/18/2020 03:02:57 PM [INFO]: Starting training process...

Traceback (most recent call last): File "main_task.py", line 52, in net = train_and_fit(args) File "/mnt/irisgpfs/users/mbiryukov/BertRel-UPD/BERT-Relation-Extraction/src/tasks/trainer.py", line 161, in train_and_fit e1_e2_start=e1_e2_start)
File "/opt/apps/resif/data/production/v1.2-20191021/default/software/devel/PyTorch/1.2.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in call result = self.forward(*input, **kwargs) File "/mnt/irisgpfs/users/mbiryukov/BertRel-UPD/BERT-Relation-Extraction/src/model/BERT/modeling_bert.py", line 759, in forward classification_logits = self.classification_layer(v1v2) File "/opt/apps/resif/data/production/v1.2-20191021/default/software/devel/PyTorch/1.2.0-fosscuda-2019a-Python-3.7.2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 591, in getattr type(self).name, name)) AttributeError: 'BertModel' object has no attribute 'classification_layer'

Could you please tell what should be modified (m.b. in model config or train?) to get it work?

Thank you!

[vabatista]

I got the same error trying to use another model other than Bert-base/large-ucased. This occurs because of the constructor of BertModel Class in src/model/BERT/modeling_bert.py.

There's a definition of classification layer only for bert-base|large-ucased.

[TharinduDR]

Did you find a solution for this? I am trying to use the BERT multilingual base, but facing a similar issue.

[plkmo]

Hi all, yes @vabatista is right, the constructor of BertModel has been modified here from the original, in order to implement this model.

If you want to use this for other models that does not use BertModel constructor here, you will have the copy the modification over to the base constructor of other models.