bert4keras icon indicating copy to clipboard operation
bert4keras copied to clipboard

怎么修改最后输出个数,变为从输入数据里提取文字索引组成句子

Open qiujunhan opened this issue 3 years ago • 0 comments

提问时请尽可能提供如下信息:

基本信息

  • 你使用的操作系统: win10
  • 你使用的Python版本: python3.8
  • 你使用的Tensorflow版本: 2.2.0
  • 你使用的Keras版本: 2.3.4
  • 你使用的bert4keras版本: 最新版
  • 你使用纯keras还是tf.keras: 纯keras
  • 你加载的预训练模型:chinese_L-12_H-768_A-12

核心代码

maxlen = 240 #输入的字库最大长度为240个字,我希望最终返回一个索引数组,表示选取了哪些字
keep_tokens = [i for i in range(maxlen)] # 修改了keep_tokens
model = build_transformer_model(
    config_path,
    checkpoint_path,
    application='unilm',
    keep_tokens=keep_tokens,  
)
#修改了token_ids,把后面的正确答案每个字改为输入序列的索引号
token_ids, segment_ids = tokenizer.encode(
                        content, str_, maxlen=maxlen
                    )
                    input_seq = token_ids[:-len(str_)-1]
                    str_ids = token_ids[-(len(str_)+1):-1]
                    str_ids = [ token_ids.index(i) for i in str_ids]
                    token_ids = input_seq+str_ids
                    batch_token_ids.append(token_ids)

输出信息

# 请在此处贴上你的调试输出
__________________________________________________________________________________________________
MLM-Norm (LayerNormalization)   (None, None, 768)    1536        MLM-Dense[0][0]                  
__________________________________________________________________________________________________
MLM-Bias (BiasAdd)              (None, None, 240)    240         Embedding-Token[1][0]            
__________________________________________________________________________________________________
MLM-Activation (Activation)     (None, None, 240)    0           MLM-Bias[0][0]                   
__________________________________________________________________________________________________
cross_entropy_1 (CrossEntropy)  (None, None, 240)    0           Input-Token[0][0]                
                                                                 Input-Segment[0][0]              
                                                                 MLM-Activation[0][0]             
==================================================================================================

Traceback (most recent call last):
  File "D:\program_temp\Anaconda3\lib\site-packages\keras\engine\training_generator.py", line 217, in fit_generator
    outs = model.train_on_batch(x, y,
  File "D:\program_temp\Anaconda3\lib\site-packages\keras\engine\training.py", line 1514, in train_on_batch
    outputs = self.train_function(ins)
  File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\keras\backend.py", line 3792, in __call__
    outputs = self._graph_fn(*converted_inputs)
  File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 1605, in __call__
    return self._call_impl(args, kwargs)
  File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 1645, in _call_impl
    return self._call_flat(args, self.captured_inputs, cancellation_manager)
  File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 1745, in _call_flat
    return self._build_call_outputs(self._inference_function.call(
  File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 593, in call
    outputs = execute.execute(
  File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\eager\execute.py", line 59, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError:  indices[15,1] = 421 is not in [0, 240)
	 [[node Embedding-Token/embedding_lookup (defined at \program_temp\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py:3007) ]] [Op:__inference_keras_scratch_graph_39117]
Function call stack:
keras_scratch_graph


自我尝试

尝试改回原来的token_ids,仍然不行,似乎不是数据集,那应该是模型修改有问题,请问怎么修改模型到240个输出

qiujunhan avatar Jan 21 '22 19:01 qiujunhan