关于预训练pretrain出的bert模型，无法读取的问题

Open SimZhou opened this issue 3 years ago • 11 comments

提问时请尽可能提供如下信息：

基本信息

你使用的操作系统: Linux
你使用的Python版本: 3.7.10
你使用的Tensorflow版本: 1.14.0
你使用的Keras版本: 2.3.1
你使用的bert4keras版本: 0.10.6
你使用纯keras还是tf.keras: 预训练必须使用tf.keras，微调的时候使用了纯keras
你加载的预训练模型: 自己训练的。

核心代码

# 请在此处贴上你的核心代码。
# 请尽量只保留关键部分，不要无脑贴全部代码。

我正在做ner任务，使用了预训练中的两个脚本data_utils.py, pretraining.py 预训练了模型，预训练pretraining.py的配置如下：

# 语料路径和模型保存路径
# 如果是TPU训练，那么语料必须存放在Google Cloud Storage上面，
# 路径必须以gs://开头；如果是GPU训练，改为普通路径即可。
#model_saved_path = 'gs://xxxx/bert4keras/saved_model/bert_model.ckpt'
model_saved_path = './saved_model_0624/bert_model.ckpt'
corpus_paths = [ 
    './corpus_tfrecord/corpus.%s.tfrecord' % i for i in range(10)
]

# 其他配置
sequence_length = 512 
batch_size = 256 
#ckp_path = '/home/yhz25/.local/pretrained_models/chinese_roberta_wwm_large_ext_L-24_H-1024_A-16/'                                                                                                                                            
ckp_path = '/home/yhz25/.local/pretrained_models/chinese_wwm_ext_L-12_H-768_A-12/'
config_path = ckp_path + 'bert_config.json'
checkpoint_path = ckp_path + 'bert_model.ckpt'  # 如果从零训练，就设为None
#learning_rate = 0.00176
learning_rate = 1e-4
weight_decay_rate = 0.01
num_warmup_steps = 3125
num_train_steps = 125000
steps_per_epoch = 10000
grad_accum_steps = 16  # 大于1即表明使用梯度累积
epochs = num_train_steps * grad_accum_steps // steps_per_epoch
exclude_from_weight_decay = ['Norm', 'bias']
exclude_from_layer_adaptation = ['Norm', 'bias']
#tpu_address = 'grpc://xxx.xxx.xxx.xxx:8470'  # 如果用多GPU跑，直接设为None
tpu_address = None  # 如果用多GPU跑，直接设为None
which_optimizer = 'lamb'  # adam 或 lamb，均自带weight decay
lr_schedule = { 
    num_warmup_steps * grad_accum_steps: 1.0,
    num_train_steps * grad_accum_steps: 0.0,
}
floatx = K.floatx()

也就是我加载了chinese_wwm_ext_L-12_H-768_A-12，在其基础上进行增量预训练。然后，预训练training.log文件输出如下：

epoch,loss,mlm_acc_loss,mlm_loss_loss
0,0.04942777005136013,0.008477748,0.040950023
1,0.027718735705316067,0.011608544,0.01611019
2,0.02059221497476101,0.014212102,0.0063801124
3,0.018868646512925625,0.014359909,0.004508737
4,0.017998265926539896,0.014961219,0.003037046
5,0.01725778165310621,0.014966778,0.0022910023
6,0.016904429614543914,0.015371294,0.0015331351
7,0.016650540009140968,0.015199836,0.0014507021
8,0.017115043097734452,0.015355815,0.0017592277
9,0.016309582896530627,0.015342234,0.00096734974
10,0.016258064770698546,0.015522803,0.0007352621
11,0.016437076149880887,0.0154431,0.0009939758
12,0.016116816529631615,0.015390981,0.0007258361
13,0.016503105728328228,0.015465755,0.0010373499
14,0.016068534268438815,0.015357787,0.0007107466
15,0.016017625857889652,0.015493634,0.00052399136
16,0.016224851341545582,0.015508482,0.0007163685
17,0.01600364064127207,0.015509772,0.0004938696
18,0.01649426277279854,0.015705802,0.00078846083
19,0.01615502136349678,0.015646955,0.0005080676
20,0.01587021830379963,0.015412417,0.00045779996
21,0.016251139537990095,0.01551893,0.00073220825
22,0.015887844990193845,0.01538122,0.0005066253
23,0.016135430613160133,0.015552671,0.0005827606
24,0.015986557887494562,0.015725676,0.00026088062
25,0.016284882429242135,0.01588686,0.00039802233
26,0.01562889501005411,0.015183384,0.00044551183
27,0.01609939290434122,0.015773943,0.0003254493
28,0.015780764001607894,0.015261261,0.0005195037
29,0.015996115002036095,0.015700711,0.0002954049
30,0.016107061676681043,0.015698768,0.00040829292

看上去没啥问题预训练后获得的模型目录如下

> ls
bert_model.ckpt.data-00000-of-00002  bert_model.ckpt.data-00001-of-00002  bert_model.ckpt.index  bert_model.ckpt.meta  checkpoint

然后尝试使用ner训练脚本 task_sequence_labeling_ner_crf.py加载自己训练的预训练模型，然后报错我是这么配置加载的： config_path配置的是原开源模型目录chinese_wwm_ext_L-12_H-768_A-12的bert_config.json checkpoint_path配置的是新训练的模型目录中的bert_model.ckpt dict_path配置的是原开源模型目录chinese_wwm_ext_L-12_H-768_A-12中的vocab.txt

输出信息

# 请在此处贴上你的调试输出
Traceback (most recent call last):
  File "train.py", line 122, in <module>
    checkpoint_path,
  File "/mnt/lustre02/jiangsu/aispeech/home/yhz25/.conda/envs/tf114/lib/python3.7/site-packages/bert4keras-0.10.6-py3.7.egg/bert4keras/models.py", line 2448, in build_transformer_model
    transformer.load_weights_from_checkpoint(checkpoint_path)
  File "/mnt/lustre02/jiangsu/aispeech/home/yhz25/.conda/envs/tf114/lib/python3.7/site-packages/bert4keras-0.10.6-py3.7.egg/bert4keras/models.py", line 302, in load_weights_from_checkpoint
    raise e
  File "/mnt/lustre02/jiangsu/aispeech/home/yhz25/.conda/envs/tf114/lib/python3.7/site-packages/bert4keras-0.10.6-py3.7.egg/bert4keras/models.py", line 296, in load_weights_from_checkpoint
    values.append(self.load_variable(checkpoint, v))
  File "/mnt/lustre02/jiangsu/aispeech/home/yhz25/.conda/envs/tf114/lib/python3.7/site-packages/bert4keras-0.10.6-py3.7.egg/bert4keras/models.py", line 696, in load_variable
    variable = super(BERT, self).load_variable(checkpoint, name)
  File "/mnt/lustre02/jiangsu/aispeech/home/yhz25/.conda/envs/tf114/lib/python3.7/site-packages/bert4keras-0.10.6-py3.7.egg/bert4keras/models.py", line 267, in load_variable
    return tf.train.load_variable(checkpoint, name)
  File "/mnt/lustre02/jiangsu/aispeech/home/yhz25/.conda/envs/tf114/lib/python3.7/site-packages/tensorflow/python/training/checkpoint_utils.py", line 84, in load_variable
    return reader.get_tensor(name)
  File "/mnt/lustre02/jiangsu/aispeech/home/yhz25/.conda/envs/tf114/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 678, in get_tensor
    return CheckpointReader_GetTensor(self, compat.as_bytes(tensor_str))
tensorflow.python.framework.errors_impl.NotFoundError: Key bert/embeddings/word_embeddings not found in checkpoint

自我尝试

看样子问题应该出在load_weights_from_checkpoint没有读取到checkpoint中的embedding层参数。是否是因为pretraining过程不会保存embedding层的参数呢？

Jun 28 '21 06:06 SimZhou

save_weights保存的模型，load_weights绝对能加载，不存在什么无法读取的问题。

Jun 28 '21 07:06 bojone

至于你想用load_weights_from_checkpoint加载，自然要用save_weights_as_checkpoint保存。把逻辑搞清楚了，就不会有什么问题。

Jun 28 '21 07:06 bojone

save_weights保存的模型，load_weights绝对能加载，不存在什么无法读取的问题。

大神您好， save_weights()cpu下得到了1个文件，['best_model.weights']，load_weights(./best_model.weights)继续用于预测是没问题的。但是save_weights()gpu下得到四个文件['best_model.weights.data-00000-of-00002', 'best_model.weights.data-00001-of-00002', 'best_model.weights.index', 'checkpoint']，load_weights()时候该怎么写里面的内容，直接写load_weights(./best_model.weights)，报错：OSError: Unable to open file (unable to open file: name = './best_model.weights', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)。查询说3种可能：1.文件坏了，2.重装h5py，3.绝对路径。我都试过了，无法解决，特来请教！

Jul 05 '21 09:07 Eric-yuye

save_weights保存的模型，load_weights绝对能加载，不存在什么无法读取的问题。

大神您好， save_weights()cpu下得到了1个文件，['best_model.weights']，load_weights(./best_model.weights)继续用于预测是没问题的。但是save_weights()gpu下得到四个文件['best_model.weights.data-00000-of-00002', 'best_model.weights.data-00001-of-00002', 'best_model.weights.index', 'checkpoint']，load_weights()时候该怎么写里面的内容，直接写load_weights(./best_model.weights)，报错：OSError: Unable to open file (unable to open file: name = './best_model.weights', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)。查询说3种可能：1.文件坏了，2.重装h5py，3.绝对路径。我都试过了，无法解决，特来请教！

这个主要问题在于，预训练采用tf_keras=1，所以保存的是tensorflow checkpoint模型，就是这样格式的，而微调的过程使用的是原生keras，所以是h5格式的单文件。

Jul 06 '21 06:07 SimZhou

save_weights保存的模型，load_weights绝对能加载，不存在什么无法读取的问题。

目前的问题是，由于预训练使用的是tf，所以save_weights保存的格式是tf格式的weights（checkpoint）而微调的时候，使用的是keras，所以load_weights理应读取单个h5文件，所以load_weights也就无论如何都用不了了。（如果传入saved_model目录，则报错"Is a directory"，如果传入saved_model/bert_model.ckpt，则报文件不存在，因为bert_model.ckpt存的是checkpoint格式而不是单文件）

而您这边实现的load_weights_from_checkpoint，在读取的时候由于前者预训练过后保存的参数名和模型本身的参数名不一致，所以map失败了

Jul 07 '21 07:07 SimZhou

预训练保存的模型keys为：

['_CHECKPOINTABLE_OBJECT_GRAPH (DT_STRING) []',
 'layer_with_weights-0/embeddings/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [21128,768]',
 'layer_with_weights-0/embeddings/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [21128,768]',
 'layer_with_weights-0/embeddings/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [21128,768]',
 'layer_with_weights-0/embeddings/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [21128,768]',
 'layer_with_weights-1/embeddings/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [2,768]',
 'layer_with_weights-1/embeddings/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [2,768]',
 'layer_with_weights-1/embeddings/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [2,768]',
 'layer_with_weights-1/embeddings/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [2,768]',
 'layer_with_weights-10/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-10/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-10/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-10/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-10/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-10/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-10/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-10/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-10/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-10/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-10/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-10/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-10/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-10/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-10/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-10/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-11/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-11/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-11/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-11/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-11/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-11/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-11/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-11/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-12/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-12/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-13/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-13/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-13/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-13/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-13/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-13/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-13/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-13/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-14/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-14/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-14/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-14/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-14/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-14/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-14/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-14/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-14/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-14/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-14/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-14/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-14/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-14/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-14/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-14/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-15/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-15/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-15/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-15/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-15/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-15/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-15/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-15/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-16/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-16/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-17/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-17/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-17/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-17/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-17/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-17/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-17/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-17/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-18/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-18/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-18/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-18/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-18/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-18/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-18/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-18/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-18/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-18/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-18/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-18/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-18/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-18/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-18/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-18/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-19/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-19/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-19/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-19/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-19/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-19/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-19/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-19/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-2/embeddings/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [512,768]',
 'layer_with_weights-2/embeddings/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [512,768]',
 'layer_with_weights-2/embeddings/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [512,768]',
 'layer_with_weights-2/embeddings/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [512,768]',
 'layer_with_weights-20/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-20/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-20/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-21/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-21/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-21/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-21/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-21/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-21/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-21/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-21/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-22/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-22/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-22/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-22/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-22/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-22/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-22/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-22/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-22/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-22/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-22/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-22/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-22/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-22/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-22/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-22/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-23/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-23/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-23/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-23/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-23/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-23/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-23/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-23/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-24/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-24/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-25/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-25/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-25/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-25/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-25/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-25/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-25/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-25/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-26/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-26/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-26/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-26/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-26/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-26/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-26/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-26/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-26/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-26/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-26/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-26/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-26/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-26/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-26/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-26/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-27/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-27/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-27/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-27/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-27/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-27/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-27/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-27/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-28/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-28/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-29/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-29/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-29/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-29/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-29/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-29/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-29/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-29/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-3/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-3/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-3/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-3/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-3/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-3/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-3/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-3/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-30/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-30/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-30/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-30/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-30/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-30/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-30/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-30/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-30/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-30/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-30/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-30/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-30/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-30/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-30/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-30/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-31/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-31/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-31/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-31/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-31/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-31/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-31/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-31/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-32/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-32/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-33/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-33/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-33/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-33/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-33/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-33/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-33/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-33/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-34/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-34/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-34/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-34/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-34/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-34/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-34/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-34/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-34/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-34/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-34/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-34/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-34/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-34/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-34/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-34/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-35/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-35/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-35/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-35/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-35/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-35/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-35/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-35/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-36/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-36/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-37/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-37/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-37/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-37/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-37/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-37/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-37/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-37/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-38/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-38/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-38/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-38/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-38/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-38/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-38/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-38/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-38/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-38/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-38/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-38/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-38/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-38/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-38/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-38/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-39/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-39/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-39/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-39/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-39/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-39/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-39/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-39/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-4/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-4/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-40/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-40/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-41/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-41/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-41/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-41/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-41/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-41/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-41/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-41/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-42/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-42/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-42/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-42/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-42/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-42/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-42/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-42/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-42/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-42/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-42/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-42/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-42/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-42/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-42/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-42/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-43/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-43/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-43/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-43/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-43/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-43/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-43/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-43/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-44/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-44/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-45/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-45/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-45/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-45/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-45/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-45/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-45/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-45/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-46/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-46/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-46/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-46/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-46/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-46/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-46/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-46/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-46/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-46/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-46/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-46/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-46/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-46/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-46/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-46/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-47/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-47/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-47/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-47/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-47/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-47/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-47/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-47/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-48/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-48/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-49/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-49/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-49/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-49/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-49/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-49/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-49/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-49/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-5/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-5/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-5/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-5/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-5/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-5/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-5/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-5/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-50/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-50/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-50/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-50/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-50/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-50/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-50/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-50/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-50/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-50/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-50/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-50/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-50/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-50/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-50/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-50/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-51/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-51/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-51/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-51/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-51/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-51/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-51/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-51/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-52/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-52/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-52/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-52/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-52/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-52/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-52/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-52/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-53/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-53/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-53/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-53/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-53/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-53/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-53/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-53/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-54/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [21128]',
 'layer_with_weights-54/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [21128]',
 'layer_with_weights-54/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [21128]',
 'layer_with_weights-54/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [21128]',
 'layer_with_weights-6/i0_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-6/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-6/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-6/i0_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072]',
 'layer_with_weights-6/i0_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-6/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-6/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-6/i0_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,3072]',
 'layer_with_weights-6/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-6/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-6/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-6/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-6/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-6/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-6/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-6/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [3072,768]',
 'layer_with_weights-7/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-7/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-7/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-7/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-7/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-7/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-7/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-7/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/k_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/k_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/k_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/k_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/k_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/k_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/o_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/o_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/o_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/o_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/o_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/o_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/q_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/q_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/q_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/q_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/q_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/q_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/v_dense/bias/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/v_dense/bias/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/v_dense/bias/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/v_dense/bias/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-8/v_dense/kernel/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-8/v_dense/kernel/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768,768]',
 'layer_with_weights-9/beta/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-9/beta/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-9/beta/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-9/beta/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-9/gamma/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-9/gamma/.OPTIMIZER_SLOT/optimizer/ag/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-9/gamma/.OPTIMIZER_SLOT/optimizer/m/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'layer_with_weights-9/gamma/.OPTIMIZER_SLOT/optimizer/v/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) [768]',
 'optimizer/beta_1/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) []',
 'optimizer/beta_2/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) []',
 'optimizer/iter/.ATTRIBUTES/VARIABLE_VALUE (DT_INT64) []',
 'optimizer/learning_rate/.ATTRIBUTES/VARIABLE_VALUE (DT_FLOAT) []',
 '']

而bert应该需要的weights为：

['bert/embeddings/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/embeddings/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/embeddings/position_embeddings (DT_FLOAT) [512,768]',
 'bert/embeddings/token_type_embeddings (DT_FLOAT) [2,768]',
 'bert/embeddings/word_embeddings (DT_FLOAT) [21128,768]',
 'bert/encoder/layer_0/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_0/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_0/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_0/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_0/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_0/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_0/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_0/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_0/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_0/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_0/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_0/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_0/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_0/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_0/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_0/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_1/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_1/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_1/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_1/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_1/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_1/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_1/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_1/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_1/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_1/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_1/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_1/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_1/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_1/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_1/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_1/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_10/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_10/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_10/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_10/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_10/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_10/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_10/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_10/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_10/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_10/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_10/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_10/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_10/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_10/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_10/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_10/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_11/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_11/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_11/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_11/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_11/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_11/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_11/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_11/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_11/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_11/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_11/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_11/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_11/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_11/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_11/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_11/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_2/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_2/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_2/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_2/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_2/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_2/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_2/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_2/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_2/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_2/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_2/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_2/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_2/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_2/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_2/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_2/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_3/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_3/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_3/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_3/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_3/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_3/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_3/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_3/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_3/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_3/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_3/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_3/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_3/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_3/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_3/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_3/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_4/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_4/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_4/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_4/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_4/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_4/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_4/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_4/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_4/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_4/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_4/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_4/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_4/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_4/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_4/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_4/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_5/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_5/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_5/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_5/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_5/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_5/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_5/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_5/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_5/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_5/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_5/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_5/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_5/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_5/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_5/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_5/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_6/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_6/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_6/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_6/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_6/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_6/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_6/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_6/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_6/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_6/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_6/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_6/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_6/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_6/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_6/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_6/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_7/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_7/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_7/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_7/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_7/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_7/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_7/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_7/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_7/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_7/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_7/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_7/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_7/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_7/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_7/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_7/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_8/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_8/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_8/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_8/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_8/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_8/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_8/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_8/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_8/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_8/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_8/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_8/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_8/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_8/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_8/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_8/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/encoder/layer_9/attention/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_9/attention/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_9/attention/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_9/attention/output/dense/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_9/attention/self/key/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_9/attention/self/key/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_9/attention/self/query/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_9/attention/self/query/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_9/attention/self/value/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_9/attention/self/value/kernel (DT_FLOAT) [768,768]',
 'bert/encoder/layer_9/intermediate/dense/bias (DT_FLOAT) [3072]',
 'bert/encoder/layer_9/intermediate/dense/kernel (DT_FLOAT) [768,3072]',
 'bert/encoder/layer_9/output/LayerNorm/beta (DT_FLOAT) [768]',
 'bert/encoder/layer_9/output/LayerNorm/gamma (DT_FLOAT) [768]',
 'bert/encoder/layer_9/output/dense/bias (DT_FLOAT) [768]',
 'bert/encoder/layer_9/output/dense/kernel (DT_FLOAT) [3072,768]',
 'bert/pooler/dense/bias (DT_FLOAT) [768]',
 'bert/pooler/dense/kernel (DT_FLOAT) [768,768]',
 'cls/predictions/output_bias (DT_FLOAT) [21128]',
 'cls/predictions/transform/LayerNorm/beta (DT_FLOAT) [768]',
 'cls/predictions/transform/LayerNorm/gamma (DT_FLOAT) [768]',
 'cls/predictions/transform/dense/bias (DT_FLOAT) [768]',
 'cls/predictions/transform/dense/kernel (DT_FLOAT) [768,768]',
 'cls/seq_relationship/output_bias (DT_FLOAT) [2]',
 'cls/seq_relationship/output_weights (DT_FLOAT) [2,768]',
 'global_step (DT_INT64) []',
 '']

Jul 07 '21 07:07 SimZhou

你好，解决这个问题了吗

Aug 14 '21 10:08 zhongqiqianga

你好，我也出现这个问题，请问你解决这个问题了吗？

Sep 10 '21 09:09 VirgilG72

没

------------------ 原始邮件 ------------------ 发件人: "bojone/bert4keras" @.>; 发送时间: 2021年9月10日(星期五) 下午5:09 @.>; @.@.>; 主题: Re: [bojone/bert4keras] 关于预训练pretrain出的bert模型，无法读取的问题 (#365)

你好，我也出现这个问题，请问你解决这个问题了吗？

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

Oct 27 '21 09:10 Eric-yuye

同样的问题

Feb 11 '22 03:02 OKC13

先用train_model读取model，然后用save_weights_as_checkpoint保存，最后就能用build_transfoermer_model加载了，代码如下：

重新保存模型

bert, train_model, loss = build_transformer_model_with_lm()
train_model.load_weights(model_saved_path) ckpt_path = "./ckpt/model.ckpt" bert.save_weights_as_checkpoint(ckpt_path)

加载模型

model = build_transformer_model( config_path=config_path, checkpoint_path=checkpoint_path )

Apr 06 '22 03:04 cdj0311

bert4keras bert4keras copied to clipboard

关于预训练pretrain出的bert模型，无法读取的问题

基本信息

核心代码

输出信息

自我尝试

重新保存模型

加载模型

bert4keras
bert4keras copied to clipboard