yhcc
yhcc
I will try to provide a solution, but I am not familiar with the whole part of CAI, so there must have mistakes in my reparation. I am still working...
While I was repairing this bug, I find there may be another bug, https://github.com/hpcaitech/ColossalAI/blob/4898ff8af45a013f13e5fdadf9b240b2d240b3ca/colossalai/utils/common.py#L125-L126 I think this function cannot take good care of the ColoTensor, which is actually a model...
I give a demo implementation in https://github.com/yhcc/ColossalAI/blob/main/colossalai/zero/sharded_optim/low_level_optim.py. To make it work with other part of ColossalAI, other modifications are also needed, such as do not use DDPGradientHandler to sync gradients.
Finally, we found out that this code should be running on gpu.
不好意思，一直没留意到。这个应该是数据集不一样导致的，以下是我的数据规格。你应该是使用ontonotes的v12，但过去的论文一般是使用的v4（参考https://github.com/yhcc/OntoNotes-5.0-NER） `` In total 3 datasets: dev has 8528 instances. test has 8262 instances. train has 59924 instances. ``
correspond to the best dev performance取这个。
这个应该是pretrain的biobert，是transformers那个包的自动下载。你可以把biobert下载到本地，然后将from_pretrained的参数修改为本地的路径。
应该是由于你在国内连不上github。可以切换成以下的地址 pip install git+http://gitee.com/fastnlp/fastNLP@dev 和 pip install git+http://gitee.com/fastnlp/fitlog
> 您好，我看了复旦大学开源的工具包应该以下地址，没有@dev > pip install git+http://gitee.com/fastnlp/fastNLP 和 pip install git+http://gitee.com/fastnlp/fitlog 需要@dev，dev分支上的fastNLP修复了一些我们使用过程中遇到的bug，一般用dev分支会更好一些。
按道理中文应该没有影响的（中文对char的bpe就等于不做任何处理吧？），应该不是由于bpe的问题，感觉可能还是数据处理的问题。