bert4keras
bert4keras copied to clipboard
怎么修改最后输出个数,变为从输入数据里提取文字索引组成句子
提问时请尽可能提供如下信息:
基本信息
- 你使用的操作系统: win10
- 你使用的Python版本: python3.8
- 你使用的Tensorflow版本: 2.2.0
- 你使用的Keras版本: 2.3.4
- 你使用的bert4keras版本: 最新版
- 你使用纯keras还是tf.keras: 纯keras
- 你加载的预训练模型:chinese_L-12_H-768_A-12
核心代码
maxlen = 240 #输入的字库最大长度为240个字,我希望最终返回一个索引数组,表示选取了哪些字
keep_tokens = [i for i in range(maxlen)] # 修改了keep_tokens
model = build_transformer_model(
config_path,
checkpoint_path,
application='unilm',
keep_tokens=keep_tokens,
)
#修改了token_ids,把后面的正确答案每个字改为输入序列的索引号
token_ids, segment_ids = tokenizer.encode(
content, str_, maxlen=maxlen
)
input_seq = token_ids[:-len(str_)-1]
str_ids = token_ids[-(len(str_)+1):-1]
str_ids = [ token_ids.index(i) for i in str_ids]
token_ids = input_seq+str_ids
batch_token_ids.append(token_ids)
输出信息
# 请在此处贴上你的调试输出
__________________________________________________________________________________________________
MLM-Norm (LayerNormalization) (None, None, 768) 1536 MLM-Dense[0][0]
__________________________________________________________________________________________________
MLM-Bias (BiasAdd) (None, None, 240) 240 Embedding-Token[1][0]
__________________________________________________________________________________________________
MLM-Activation (Activation) (None, None, 240) 0 MLM-Bias[0][0]
__________________________________________________________________________________________________
cross_entropy_1 (CrossEntropy) (None, None, 240) 0 Input-Token[0][0]
Input-Segment[0][0]
MLM-Activation[0][0]
==================================================================================================
Traceback (most recent call last):
File "D:\program_temp\Anaconda3\lib\site-packages\keras\engine\training_generator.py", line 217, in fit_generator
outs = model.train_on_batch(x, y,
File "D:\program_temp\Anaconda3\lib\site-packages\keras\engine\training.py", line 1514, in train_on_batch
outputs = self.train_function(ins)
File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\keras\backend.py", line 3792, in __call__
outputs = self._graph_fn(*converted_inputs)
File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 1605, in __call__
return self._call_impl(args, kwargs)
File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 1645, in _call_impl
return self._call_flat(args, self.captured_inputs, cancellation_manager)
File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 1745, in _call_flat
return self._build_call_outputs(self._inference_function.call(
File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\eager\function.py", line 593, in call
outputs = execute.execute(
File "D:\program_temp\Anaconda3\lib\site-packages\tensorflow\python\eager\execute.py", line 59, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[15,1] = 421 is not in [0, 240)
[[node Embedding-Token/embedding_lookup (defined at \program_temp\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py:3007) ]] [Op:__inference_keras_scratch_graph_39117]
Function call stack:
keras_scratch_graph
自我尝试
尝试改回原来的token_ids,仍然不行,似乎不是数据集,那应该是模型修改有问题,请问怎么修改模型到240个输出