tianchi_nl2sql
tianchi_nl2sql copied to clipboard
beader
代码复现环境报错
复现代码，在docker容器内运行或者新建环境，都会在bert_model = load_trained_model_from_checkpoint(paths.config, paths.checkpoint, seq_len=None) 这步报错：AttributeError: 'tuple' object has no attribute 'layer' 所有依赖和requirements.txt里要求的一样，尝试安装了tensorflow_gpu=1.14和1.13，都会报同样的错误，请问是什么情况？ 报错信息： `>>> from keras_bert import load_vocabulary, load_trained_model_from_checkpoint, Tokenizer, get_checkpoint_paths Using TensorFlow backend.
bert_model_path = '/opt/zhuiyi/tianchi_nl2sql-master/model/chinese_wwm_L-12_H-768_A-12' paths = get_checkpoint_paths(bert_model_path) bert_model = load_trained_model_from_checkpoint(paths.config, paths.checkpoint, seq_len=None) WARNING: Logging before flag parsing goes to stderr. W0225 03:19:48.417089 140073566517056 deprecation_wrapper.py:118] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
W0225 03:19:48.457948 140073566517056 deprecation_wrapper.py:118] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.
W0225 03:19:48.498272 140073566517056 deprecation.py:506] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/initializers.py:119: calling RandomUniform.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0225 03:19:49.240138 140073566517056 deprecation_wrapper.py:118] From /usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/api/_v1/estimator/init.py:10: The name tf.estimator.inputs is deprecated. Please use tf.compat.v1.estimator.inputs instead.
Traceback (most recent call last):
File "环境版本：Package Version
absl-py 0.7.1
asn1crypto 0.24.0
astor 0.8.0
attrs 19.1.0
backcall 0.1.0
bleach 3.1.0
cn2an 0.3.6
cryptography 2.1.4
cycler 0.10.0
decorator 4.4.0
defusedxml 0.6.0
entrypoints 0.3
enum34 1.1.6
gast 0.2.2
google-pasta 0.1.7
grpcio 1.21.1
h5py 2.9.0
idna 2.6
ipykernel 5.1.1
ipython 7.5.0
ipython-genutils 0.2.0
ipywidgets 7.4.2
jedi 0.13.3
Jinja2 2.10.1
jsonschema 3.0.1
jupyter 1.0.0
jupyter-client 5.2.4
jupyter-console 6.0.0
jupyter-core 4.4.0
jupyter-http-over-ws 0.0.6
Keras 2.2.4
Keras-Applications 1.0.8
keras-bert 0.68.1
keras-embed-sim 0.10.0
keras-layer-normalization 0.16.0
keras-multi-head 0.29.0
keras-pos-embd 0.13.0
keras-position-wise-feed-forward 0.8.0
Keras-Preprocessing 1.1.0
keras-self-attention 0.51.0
keras-transformer 0.40.0
keyring 10.6.0
keyrings.alt 3.0
kiwisolver 1.1.0
Markdown 3.1.1
MarkupSafe 1.1.1
matplotlib 3.1.0
mistune 0.8.4
nbconvert 5.5.0
nbformat 4.4.0
notebook 5.7.8
numpy 1.16.4
pandas 1.1.5
pandocfilters 1.4.2
parso 0.4.0
pexpect 4.7.0
pickleshare 0.7.5
pip 21.3.1
prometheus-client 0.7.0
prompt-toolkit 2.0.9
protobuf 3.8.0
ptyprocess 0.6.0
pycrypto 2.6.1
Pygments 2.4.2
PyGObject 3.26.1
pyparsing 2.4.0
pyrsistent 0.15.2
python-apt 1.6.4
python-dateutil 2.8.0
pytz 2021.3
pyxdg 0.25
PyYAML 6.0
pyzmq 18.0.1
qtconsole 4.5.1
scipy 1.5.4
SecretStorage 2.3.1
Send2Trash 1.5.0
setuptools 41.0.1
six 1.11.0
tb-nightly 1.14.0a20190614
termcolor 1.1.0
terminado 0.8.2
testpath 0.4.2
tf-estimator-nightly 1.14.0.dev2019061701
tf-nightly-gpu 1.14.1.dev20190617
thulac 0.2.0
tornado 6.0.2
tqdm 4.62.3
traitlets 4.3.2
wcwidth 0.1.7
webencodings 0.5.1
Werkzeug 0.15.4
wheel 0.30.0
widgetsnbextension 3.4.2
wrapt 1.11.1
zhon 1.1.5
新建虚拟环境的安装步骤：pip install tensorflow-gpu==1.14
pip install -r requirements.txt
`
安装环境我是少了什么步骤吗？
你尝试直接用我的镜像试试
docker pull beader/tensorflow:nightly-gpu-py3-jupyter
您好，直接使用镜像运行model1的代码，同样在载入模型load_trained_model_from_checkpoint这步出现了相同的报错：AttributeError: 'tuple' object has no attribute 'layer' 查阅了一下说是tf.keras和tf混用，可能是环境版本问题 tensorflow-gpu==1.14 keras==2.2.4 keras-bert==0.68.1
@acadaiaca 抱歉，这个项目已经很久了，而且我手边也没有设备，没法协助你，抱歉。你主要参考模型的一些大概思路，然后自己去implement会比较好。
@beader 非常感谢！目前问题已经解决，通过直接拷贝其他人能成功复现代码的环境包。该环境问题大概率是后来keras2.24经过改版之后造成与tf不兼容。
@beader 非常感谢！目前问题已经解决，通过直接拷贝其他人能成功复现代码的环境包。该环境问题大概率是后来keras2.24经过改版之后造成与tf不兼容。
您好，请问可以提供一下这个安装环境的requestment.txt文档么，谢谢您，[email protected]
!pip install PyYAML==5.3.1 !pip install tqdm==4.53.0 !pip install pandas==1.1.4 !pip install keras==2.2.4 !pip install keras-bert==0.68.1 !pip install cn2an==0.5.5 !pip install Keras-Applications==1.0.8 !pip install keras-bert==0.68.1 !pip install keras-embed-sim==0.8.0 !pip install keras-layer-normalization==0.14.0 !pip install keras-multi-head==0.27.0 !pip install keras-pos-embd==0.11.0 !pip install keras-position-wise-feed-forward==0.6.0 !pip install Keras-Preprocessing==1.1.2 !pip install keras-self-attention==0.46.0 !pip install keras-transformer==0.38.0
这个环境能跑通，docker用的是NVDIA官方的，名称是：nvcr.io/nvidia/tensorflow:20.10-tf1-py3
谢谢大佬
At 2022-03-10 01:44:11, "QingHao" @.***> wrote:
!pip install PyYAML==5.3.1 !pip install tqdm==4.53.0 !pip install pandas==1.1.4 !pip install keras==2.2.4 !pip install keras-bert==0.68.1 !pip install cn2an==0.5.5 !pip install Keras-Applications==1.0.8 !pip install keras-bert==0.68.1 !pip install keras-embed-sim==0.8.0 !pip install keras-layer-normalization==0.14.0 !pip install keras-multi-head==0.27.0 !pip install keras-pos-embd==0.11.0 !pip install keras-position-wise-feed-forward==0.6.0 !pip install Keras-Preprocessing==1.1.2 !pip install keras-self-attention==0.46.0 !pip install keras-transformer==0.38.0
这个环境能跑通，docker用的是NVDIA官方的，版本tf1.14，名称是：nvcr.io/nvidia/tensorflow:20.10-tf1-py3
— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.***>
我说一下自己的解决思路： 在整个项目中很多地方都出现了AttributeError，很多查询后都说是混用了keras和tf.keras的包，但作者的代码里并没有混用 在我多次定位排查后发现是keras包内部的一些问题，在这些报错信息的trace back中可以找到一些文件，其import部分包含了一些首字母大写的名称的模块，这些模块点进去后可以发现其引用只有两行： from tensorflow import keras from tensorflow.keras import ***（可能记得不是很清楚，但确实是引用了tf.keras） 将这些部分改成引用纯keras模块，大概有六七个此类文件可以更改，改完之后解决报错 如果有新的类似报错还是去trace里找上述类型的文件去看import部分，不过在更改之前还是建议百度一下，一些错误不一定也是这个原因，此方法只解决混用导致的报错 希望可以帮到碰到此类问题的朋友
pip install PyYAML==5.3.1 !pip install tqdm==4.53.0 !pip install pandas==1.1.4 !pip install keras==2.2.4 !pip install keras-bert==0.68.1 !pip install cn2an==0.5.5 !pip install Keras-Applications==1.0.8 !pip install keras-bert==0.68.1 !pip install keras-embed-sim==0.8.0 !pip install keras-layer-normalization==0.14.0 !pip install keras-multi-head==0.27.0 !pip install keras-pos-embd==0.11.0 !pip install keras-position-wise-feed-forward==0.6.0 !pip install Keras-Preprocessing==1.1.2 !pip install keras-self-attention==0.46.0 !pip install keras-transformer==0.38.0
如果没有使用docker的条件，可以用conda create --name nl2sql tensorflow-gpu==1.15, 然后安装上述依赖，亲测可以跑通
