Chinese-NER icon indicating copy to clipboard operation
Chinese-NER copied to clipboard

AssertionError when train the model

Open bzqweiyi opened this issue 5 years ago • 8 comments

I have got the issue when train the model come across the AssertionError. assert os.path.isfile(ext_emb_path) Could you pls help to solve this problem. Many thx.

bzqweiyi avatar Mar 14 '19 14:03 bzqweiyi

flags.DEFINE_boolean("pre_emb", True, "Wither use pre-trained embedding") set this line (which exists in main.py line 33) to be False to disable pretrained embedding if you don't have one

GeneZC avatar Mar 14 '19 14:03 GeneZC

flags.DEFINE_boolean("pre_emb", True, "Wither use pre-trained embedding") set this line (which exists in main.py line 33) to be False to disable pretrained embedding if you don't have one

Many thx. This issue has been solved. While another issue occur. Have you ever seen this issue before? This open source project is important for me.Pls help. And the traceback is: Traceback (most recent call last): File "main.py", line 225, in tf.app.run(main) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run _sys.exit(main(argv)) File "main.py", line 219, in main train() File "main.py", line 171, in train model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger) File "/Users/apple/Documents/02 Code/tensorflow_Prj/Chinese-NER/utils.py", line 172, in create_model model = Model_class(config) File "/Users/apple/Documents/02 Code/tensorflow_Prj/Chinese-NER/model.py", line 48, in init temp = tf.sign(tf.reduce_sum(used, reduction_indices=2)) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func return func(*args, **kwargs) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1345, in reduce_sum name=name)) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 8389, in _sum name=name) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func return func(*args, **kwargs) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op op_def=op_def) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1792, in init control_input_ops) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1631, in _create_c_op raise ValueError(str(e)) ValueError: Invalid reduction dimension 2 for input with 2 dimensions. for 'Sum' (op: 'Sum') with input shapes: [?,?], [] and with computed input tensors: input[1] = <2>.

bzqweiyi avatar Mar 15 '19 03:03 bzqweiyi

在 2019年3月14日,下午10:29,WhereIsMyHead [email protected] 写道:

flags.DEFINE_boolean("pre_emb", True, "Wither use pre-trained embedding") set this line to be False to disable pretrained embedding if you don't have one

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/GeneZC/Chinese-NER/issues/2#issuecomment-472885420, or mute the thread https://github.com/notifications/unsubscribe-auth/AN_avAsFRjEThGCzdkWwjHLc24rYxyFwks5vWlzZgaJpZM4b0OBp.

Dear Gene,

Thanks for your help. while run the Chinese NER project have other issues I can solve some of them. but for the tensorflow  reduction dimension I have no any idea.

I am not sure whether you ever come across with the issue.

Best Regards, James

Traceback (most recent call last): File "main.py", line 225, in tf.app.run(main) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run _sys.exit(main(argv)) File "main.py", line 221, in main evaluate_line() File "main.py", line 200, in evaluate_line model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger) File "/Users/apple/Documents/02 Code/tensorflow_Prj/Chinese-NER/utils.py", line 172, in create_model model = Model_class(config) File "/Users/apple/Documents/02 Code/tensorflow_Prj/Chinese-NER/model.py", line 47, in init temp = tf.sign(tf.reduce_sum(used, reduction_indices=2)) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func return func(*args, **kwargs) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1345, in reduce_sum name=name)) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 8389, in _sum name=name) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func return func(*args, **kwargs) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op op_def=op_def) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1792, in init control_input_ops) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1631, in _create_c_op raise ValueError(str(e)) ValueError: Invalid reduction dimension 2 for input with 2 dimensions. for 'Sum' (op: 'Sum') with input shapes: [?,?], [] and with computed input tensors: input[1] = <2>.

bzqweiyi avatar Mar 15 '19 03:03 bzqweiyi

flags.DEFINE_boolean("pre_emb", True, "Wither use pre-trained embedding") set this line (which exists in main.py line 33) to be False to disable pretrained embedding if you don't have one

Many thx. This issue has been solved. While another issue occur. Have you ever seen this issue before? This open source project is important for me.Pls help. And the traceback is: Traceback (most recent call last): File "main.py", line 225, in tf.app.run(main) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run _sys.exit(main(argv)) File "main.py", line 219, in main train() File "main.py", line 171, in train model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger) File "/Users/apple/Documents/02 Code/tensorflow_Prj/Chinese-NER/utils.py", line 172, in create_model model = Model_class(config) File "/Users/apple/Documents/02 Code/tensorflow_Prj/Chinese-NER/model.py", line 48, in init temp = tf.sign(tf.reduce_sum(used, reduction_indices=2)) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func return func(*args, **kwargs) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1345, in reduce_sum name=name)) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 8389, in _sum name=name) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func return func(*args, **kwargs) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op op_def=op_def) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1792, in init control_input_ops) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1631, in _create_c_op raise ValueError(str(e)) ValueError: Invalid reduction dimension 2 for input with 2 dimensions. for 'Sum' (op: 'Sum') with input shapes: [?,?], [] and with computed input tensors: input[1] = <2>.

the line 34 for model.py should be shape=[None, None, None],

GeneZC avatar Mar 15 '19 05:03 GeneZC

Sorry for the bugs, it's been a while that I've written the code. I've just updated the code and you could continue to comment here if you find anything else wrong.

GeneZC avatar Mar 15 '19 05:03 GeneZC

Dear Gene,

Thank you very much. I have download the model.py and find that you have changed little. After that, run again the code. These code still have some issues. Would you pls run the code in your site, confirm the version is ok, and then you can release to the repo. I will run the demo first, then study this NER method. And finally we want to change the ner module for our project. It’s very kind of you to do this.

Regards, James

Traceback (most recent call last): File "main.py", line 225, in tf.app.run(main) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run _sys.exit(main(argv)) File "main.py", line 219, in main train() File "main.py", line 171, in train model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger) File "/Users/apple/Documents/02 Code/tensorflow_Prj/Chinese-NER/utils.py", line 172, in create_model model = Model_class(config) File "/Users/apple/Documents/02 Code/tensorflow_Prj/Chinese-NER/model.py", line 61, in init lstm_inputs = self.char_cnn_layer(dropout_inputs) File "/Users/apple/Documents/02 Code/tensorflow_Prj/Chinese-NER/model.py", line 128, in char_cnn_layer self.b = tf.Variable(tf.constant(0.1, shape=self.char_dim), name="b") File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 208, in constant value, dtype=dtype, shape=shape, verify_shape=verify_shape)) File "/Users/apple/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 485, in make_tensor_proto shape = [int(dim) for dim in shape] TypeError: 'int' object is not iterable

bzqweiyi avatar Mar 15 '19 06:03 bzqweiyi

ok, I will check the code as soon as possible.

GeneZC avatar Mar 15 '19 08:03 GeneZC

@bzqweiyi The code is runnable now. The problems lie in that the version incompatibility of Tensorflow. Now I've released a cleaner version.

GeneZC avatar Mar 15 '19 10:03 GeneZC