CasRel
CasRel copied to clipboard
weizhepei
IndexError: Read less bytes than requested
I get the following error while running the run.py file
!python run.py --train=True --dataset=NYT
Using TensorFlow backend.
2020-07-03 14:29:11.236282: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-07-03 14:29:11.241179: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2200000000 Hz
2020-07-03 14:29:11.241354: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1e66a00 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-03 14:29:11.241383: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2020-07-03 14:29:11.243340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-07-03 14:29:11.248062: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2020-07-03 14:29:11.248099: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: e54567bf18c0
2020-07-03 14:29:11.248114: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: e54567bf18c0
2020-07-03 14:29:11.248169: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 418.67.0
2020-07-03 14:29:11.248200: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 418.67.0
2020-07-03 14:29:11.248212: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 418.67.0
train_data len: 56195
dev_data len: 4999
test_data len: 1297
Traceback (most recent call last):
File "run.py", line 40, in <module>
subject_model, object_model, hbt_model = E2EModel(bert_config_path, bert_checkpoint_path, LR, num_rels)
File "/content/CasRel/model.py", line 15, in E2EModel
bert_model = load_trained_model_from_checkpoint(bert_config_path, bert_checkpoint_path, seq_len=None)
File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 170, in load_trained_model_from_checkpoint
load_model_weights_from_checkpoint(model, config, checkpoint_file, training=training)
File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 114, in load_model_weights_from_checkpoint
loader('bert/encoder/layer_%d/output/dense/kernel' % i),
File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 18, in _loader
return tf.train.load_variable(checkpoint_file, name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py", line 85, in load_variable
return reader.get_tensor(name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/py_checkpoint_reader.py", line 70, in get_tensor
self, compat.as_bytes(tensor_str))
IndexError: Read less bytes than requested
Please do let me know if there is any solution to this problem. Thanks in advance! :D
I get the following error while running the
run.pyfile
!python run.py --train=True --dataset=NYTUsing TensorFlow backend. 2020-07-03 14:29:11.236282: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-07-03 14:29:11.241179: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2200000000 Hz 2020-07-03 14:29:11.241354: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1e66a00 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-07-03 14:29:11.241383: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-07-03 14:29:11.243340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2020-07-03 14:29:11.248062: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected 2020-07-03 14:29:11.248099: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: e54567bf18c0 2020-07-03 14:29:11.248114: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: e54567bf18c0 2020-07-03 14:29:11.248169: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 418.67.0 2020-07-03 14:29:11.248200: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 418.67.0 2020-07-03 14:29:11.248212: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 418.67.0 train_data len: 56195 dev_data len: 4999 test_data len: 1297 Traceback (most recent call last): File "run.py", line 40, in <module> subject_model, object_model, hbt_model = E2EModel(bert_config_path, bert_checkpoint_path, LR, num_rels) File "/content/CasRel/model.py", line 15, in E2EModel bert_model = load_trained_model_from_checkpoint(bert_config_path, bert_checkpoint_path, seq_len=None) File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 170, in load_trained_model_from_checkpoint load_model_weights_from_checkpoint(model, config, checkpoint_file, training=training) File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 114, in load_model_weights_from_checkpoint loader('bert/encoder/layer_%d/output/dense/kernel' % i), File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 18, in _loader return tf.train.load_variable(checkpoint_file, name) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py", line 85, in load_variable return reader.get_tensor(name) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/py_checkpoint_reader.py", line 70, in get_tensor self, compat.as_bytes(tensor_str)) IndexError: Read less bytes than requestedPlease do let me know if there is any solution to this problem. Thanks in advance! :D
i have find how to change the code just annotation the os.device the the first issue
I had a similar issue (list index out of range) after running below:
python run.py --train=True --dataset=NYT
Using TensorFlow backend. WARNING:tensorflow:From run.py:11: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.
WARNING:tensorflow:From run.py:13: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
2020-09-08 20:36:29.369876: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-09-08 20:36:29.375942: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2399980000 Hz
2020-09-08 20:36:29.377320: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x555558c6f550 executing computations on platform Host. Devices:
2020-09-08 20:36:29.377352: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0):
WARNING:tensorflow:From /gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.
Traceback (most recent call last):
File "run.py", line 40, in
I had a similar issue (list index out of range) after running below:
python run.py --train=True --dataset=NYT
Using TensorFlow backend. WARNING:tensorflow:From run.py:11: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.
WARNING:tensorflow:From run.py:13: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
2020-09-08 20:36:29.369876: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-09-08 20:36:29.375942: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2399980000 Hz 2020-09-08 20:36:29.377320: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x555558c6f550 executing computations on platform Host. Devices: 2020-09-08 20:36:29.377352: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , train_data len: 56195 dev_data len: 4999 test_data len: 5000 WARNING:tensorflow:From /gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
WARNING:tensorflow:From /gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.
Traceback (most recent call last): File "run.py", line 40, in subject_model, object_model, hbt_model = E2EModel(bert_config_path, bert_checkpoint_path, LR, num_rels) File "/gstore/home/kimc26/CasRel/model.py", line 15, in E2EModel bert_model = load_trained_model_from_checkpoint(bert_config_path, bert_checkpoint_path, seq_len=None) File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/loader.py", line 169, in load_trained_model_from_checkpoint **kwargs) File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/loader.py", line 58, in build_model_from_config **kwargs) File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/bert.py", line 84, in get_model dropout_rate=dropout_rate, File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/layers/embedding.py", line 37, in get_embedding )(inputs[0]), File "/gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/engine/base_layer.py", line 497, in call File "/gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/engine/base_layer.py", line 565, in _add_inbound_node IndexError: list index out of range
#find the code os.device and delete it and maker sure you use the gpu ,i just see CPU used in
Hi all, this seems like a preprocessing issue. You can check the discussion here to see if it helps. Please ping me here if there is still a problem with running the code.
Hi Guys, If anyone still have this issue, IndexError: Read less bytes than requested.. This is what worked for me,
My google drive disk ran out of memory and it was throwing me this error when I was trying to load model that I have previously saved in the google drive.
I used different google drive account with decent memory disk, to save and load model, and I had no issue.
Hope this helps!.
Thanks Kalpana
I had a similar issue (tensorflow.python.framework.errors_impl.OutOfRangeError: Read less bytes than requested) after running,Please do let me know if there is any solution to this problem.
Traceback (most recent call last):
File "run.py", line 40, in
If you are working in Google Colab, please try running with different account account, as the current one might have ran out of memory.
I get the following error while running the
run.pyfile!python run.py --train=True --dataset=NYTUsing TensorFlow backend. 2020-07-03 14:29:11.236282: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-07-03 14:29:11.241179: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2200000000 Hz 2020-07-03 14:29:11.241354: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1e66a00 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-07-03 14:29:11.241383: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-07-03 14:29:11.243340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2020-07-03 14:29:11.248062: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected 2020-07-03 14:29:11.248099: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: e54567bf18c0 2020-07-03 14:29:11.248114: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: e54567bf18c0 2020-07-03 14:29:11.248169: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 418.67.0 2020-07-03 14:29:11.248200: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 418.67.0 2020-07-03 14:29:11.248212: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 418.67.0 train_data len: 56195 dev_data len: 4999 test_data len: 1297 Traceback (most recent call last): File "run.py", line 40, in <module> subject_model, object_model, hbt_model = E2EModel(bert_config_path, bert_checkpoint_path, LR, num_rels) File "/content/CasRel/model.py", line 15, in E2EModel bert_model = load_trained_model_from_checkpoint(bert_config_path, bert_checkpoint_path, seq_len=None) File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 170, in load_trained_model_from_checkpoint load_model_weights_from_checkpoint(model, config, checkpoint_file, training=training) File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 114, in load_model_weights_from_checkpoint loader('bert/encoder/layer_%d/output/dense/kernel' % i), File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 18, in _loader return tf.train.load_variable(checkpoint_file, name) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py", line 85, in load_variable return reader.get_tensor(name) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/py_checkpoint_reader.py", line 70, in get_tensor self, compat.as_bytes(tensor_str)) IndexError: Read less bytes than requestedPlease do let me know if there is any solution to this problem. Thanks in advance! :D
i have find how to change the code just annotation the os.device the the first issue
hello,i can't find the os.device,could you tell me where it is,thanks
