python train.py 报错，麻烦帮看下啥问题

[georgewangchn]

Some layers from the model checkpoint at pre_models were not used when initializing TFBertModel: ['mlm___cls', 'nsp___cls']

• This IS expected if you are initializing TFBertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing TFBertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). All the layers of TFBertModel were initialized from the model checkpoint at pre_models. If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertModel for predictions without further training. 2021-01-29 19:33:57.271548: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at einsum_op_impl.h:714 : Resource exhausted: OOM when allocating tensor with shape[64,12,218,218] and type float on /job:localhost/replica:0/task:0/device:CPU:0 by allocator cpu Traceback (most recent call last): File "train.py", line 6, in training_files="data", File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/train.py", line 109, in train loop, File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/utils/common.py", line 308, in run_in_loop result = loop.run_until_complete(f) File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/train.py", line 174, in train_async finetuning_epoch_fraction=finetuning_epoch_fraction, File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/train.py", line 353, in _train_async_internal finetuning_epoch_fraction=finetuning_epoch_fraction, File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/train.py", line 396, in _do_training finetuning_epoch_fraction=finetuning_epoch_fraction, File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/train.py", line 818, in _train_nlu_with_validated_data **additional_arguments, File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/train.py", line 116, in train interpreter = trainer.train(training_data, **kwargs) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/model.py", line 209, in train updates = component.train(working_data, self.config, **context) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/featurizers/dense_featurizer/lm_featurizer.py", line 814, in train batch_docs = self._get_docs_for_batch(batch_messages, attribute) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/featurizers/dense_featurizer/lm_featurizer.py", line 767, in _get_docs_for_batch batch_token_ids, batch_tokens, batch_examples, attribute, inference_mode File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/featurizers/dense_featurizer/lm_featurizer.py", line 685, in _get_model_features_for_batch batch_attention_mask, padded_token_ids File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/featurizers/dense_featurizer/lm_featurizer.py", line 537, in _compute_batch_sequence_features np.array(padded_token_ids), attention_mask=np.array(batch_attention_mask) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 985, in call outputs = call_fn(inputs, *args, **kwargs) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/transformers/models/bert/modeling_tf_bert.py", line 901, in call training=inputs["training"], File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 985, in call outputs = call_fn(inputs, *args, **kwargs) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/transformers/models/bert/modeling_tf_bert.py", line 694, in call training=inputs["training"], File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 985, in call outputs = call_fn(inputs, *args, **kwargs) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/transformers/models/bert/modeling_tf_bert.py", line 452, in call hidden_states, attention_mask, head_mask[i], output_attentions, training=training File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 985, in call outputs = call_fn(inputs, *args, **kwargs) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/transformers/models/bert/modeling_tf_bert.py", line 418, in call hidden_states, attention_mask, head_mask, output_attentions, training=training File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 985, in call outputs = call_fn(inputs, *args, **kwargs) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/transformers/models/bert/modeling_tf_bert.py", line 354, in call input_tensor, attention_mask, head_mask, output_attentions, training=training File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 985, in call outputs = call_fn(inputs, *args, **kwargs) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/transformers/models/bert/modeling_tf_bert.py", line 288, in call attention_scores = tf.einsum("aecd,abcd->acbe", key_layer, query_layer) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py", line 201, in wrapper return target(*args, **kwargs) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/ops/special_math_ops.py", line 684, in einsum return _einsum_v2(equation, *inputs, **kwargs) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/ops/special_math_ops.py", line 1113, in _einsum_v2 return gen_linalg_ops.einsum(inputs, resolved_equation) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/ops/gen_linalg_ops.py", line 1088, in einsum _ops.raise_from_not_ok_status(e, name) File "/root/anaconda3/envs/rasa/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 6843, in raise_from_not_ok_status six.raise_from(core._status_to_exception(e.code, message), None) File "", line 3, in raise_from tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[64,12,218,218] and type float on /job:localhost/replica:0/task:0/device:CPU:0 by allocator cpu [Op:Einsum]

[Dustyposa]

你用的模型文件是？