mac-network icon indicating copy to clipboard operation
mac-network copied to clipboard

Evaluation error

Open sunchipsster1 opened this issue 5 years ago • 0 comments

Hello! Thank you so much for putting up this beautiful work!

After training on the args.txt configuration, I proceeded to evaluate as per the instructions. I obtained the following error. Do you know what is going on?

2020-06-26 00:53:56.788567: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-06-26 00:53:56.795400: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2599845000 Hz 2020-06-26 00:53:56.795894: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xecc6dc0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-06-26 00:53:56.795930: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-06-26 00:53:56.801185: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2020-06-26 00:53:56.923729: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xecf9b60 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2020-06-26 00:53:56.923796: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Tesla P40, Compute Capability 6.1 2020-06-26 00:53:56.925923: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: Tesla P40 major: 6 minor: 1 memoryClockRate(GHz): 1.531 pciBusID: 0000:84:00.0 2020-06-26 00:53:56.927409: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /share/apps/python3/3.7.3/intel/lib:/share/apps/gcc/6.3.0/lib64:/share/apps/gcc/6.3.0/lib:/share/apps/mpc/1.0.3/gnu/lib:/share/apps/mpfr/3.1.5/gnu/lib:/share/apps/gmp/6.1.2/gnu/lib:/share/apps/intel/19.0.1/mkl/lib/intel64:/share/apps/intel/19.0.1/lib/intel64:/share/apps/centos/7/usr/lib64:/opt/slurm/lib64 2020-06-26 00:53:56.928514: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /share/apps/python3/3.7.3/intel/lib:/share/apps/gcc/6.3.0/lib64:/share/apps/gcc/6.3.0/lib:/share/apps/mpc/1.0.3/gnu/lib:/share/apps/mpfr/3.1.5/gnu/lib:/share/apps/gmp/6.1.2/gnu/lib:/share/apps/intel/19.0.1/mkl/lib/intel64:/share/apps/intel/19.0.1/lib/intel64:/share/apps/centos/7/usr/lib64:/opt/slurm/lib64 2020-06-26 00:53:56.929673: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /share/apps/python3/3.7.3/intel/lib:/share/apps/gcc/6.3.0/lib64:/share/apps/gcc/6.3.0/lib:/share/apps/mpc/1.0.3/gnu/lib:/share/apps/mpfr/3.1.5/gnu/lib:/share/apps/gmp/6.1.2/gnu/lib:/share/apps/intel/19.0.1/mkl/lib/intel64:/share/apps/intel/19.0.1/lib/intel64:/share/apps/centos/7/usr/lib64:/opt/slurm/lib64 2020-06-26 00:53:56.930791: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /share/apps/python3/3.7.3/intel/lib:/share/apps/gcc/6.3.0/lib64:/share/apps/gcc/6.3.0/lib:/share/apps/mpc/1.0.3/gnu/lib:/share/apps/mpfr/3.1.5/gnu/lib:/share/apps/gmp/6.1.2/gnu/lib:/share/apps/intel/19.0.1/mkl/lib/intel64:/share/apps/intel/19.0.1/lib/intel64:/share/apps/centos/7/usr/lib64:/opt/slurm/lib64 2020-06-26 00:53:56.932004: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /share/apps/python3/3.7.3/intel/lib:/share/apps/gcc/6.3.0/lib64:/share/apps/gcc/6.3.0/lib:/share/apps/mpc/1.0.3/gnu/lib:/share/apps/mpfr/3.1.5/gnu/lib:/share/apps/gmp/6.1.2/gnu/lib:/share/apps/intel/19.0.1/mkl/lib/intel64:/share/apps/intel/19.0.1/lib/intel64:/share/apps/centos/7/usr/lib64:/opt/slurm/lib64 2020-06-26 00:53:56.933170: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /share/apps/python3/3.7.3/intel/lib:/share/apps/gcc/6.3.0/lib64:/share/apps/gcc/6.3.0/lib:/share/apps/mpc/1.0.3/gnu/lib:/share/apps/mpfr/3.1.5/gnu/lib:/share/apps/gmp/6.1.2/gnu/lib:/share/apps/intel/19.0.1/mkl/lib/intel64:/share/apps/intel/19.0.1/lib/intel64:/share/apps/centos/7/usr/lib64:/opt/slurm/lib64 2020-06-26 00:53:56.933611: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /share/apps/python3/3.7.3/intel/lib:/share/apps/gcc/6.3.0/lib64:/share/apps/gcc/6.3.0/lib:/share/apps/mpc/1.0.3/gnu/lib:/share/apps/mpfr/3.1.5/gnu/lib:/share/apps/gmp/6.1.2/gnu/lib:/share/apps/intel/19.0.1/mkl/lib/intel64:/share/apps/intel/19.0.1/lib/intel64:/share/apps/centos/7/usr/lib64:/opt/slurm/lib64 2020-06-26 00:53:56.933650: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform. Skipping registering GPU devices... 2020-06-26 00:53:56.933691: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-06-26 00:53:56.933715: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 2020-06-26 00:53:56.933736: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N [1mPreprocess data...[0m [1mLoading data...[0m took 76.19 seconds [1mLoading word vectors...[0m 0 {'<PAD>': 0, '<UNK>': 1, '<START>': 2, '<END>': 3, 'are': 4, 'there': 5, 'more': 6, 'big': 7, 'green': 8, 'things': 9, 'than': 10, 'large': 11, 'purple': 12, 'shiny': 13, 'cubes': 14, 'how': 15, 'many': 16, 'other': 17, 'of': 18, 'the': 19, 'same': 20, 'shape': 21, 'as': 22, 'tiny': 23, 'cyan': 24, 'matte': 25, 'object': 26, 'is': 27, 'color': 28, 'sphere': 29, 'cube': 30, 'what': 31, 'material': 32, 'that': 33, 'right': 34, 'brown': 35, 'cylinder': 36, 'and': 37, 'left': 38, 'gray': 39, 'on': 40, 'side': 41, 'small': 42, 'rubber': 43, 'behind': 44, 'thing': 45, 'to': 46, 'metallic': 47, 'size': 48, 'any': 49, 'have': 50, 'block': 51, 'blue': 52, 'yellow': 53, 'a': 54, ';': 55, 'it': 56, 'ball': 57, 'its': 58, 'in': 59, 'front': 60, 'does': 61, 'number': 62, 'red': 63, 'spheres': 64, 'made': 65, 'metal': 66, 'cylinders': 67, 'both': 68, 'balls': 69, 'or': 70, 'blocks': 71, 'objects': 72, 'visible': 73, 'another': 74, 'has': 75, 'greater': 76, 'fewer': 77, 'less': 78, 'either': 79, 'anything': 80, 'else': 81, 'do': 82, 'an': 83, 'equal': 84} 85 {'yes': 0, '2': 1, 'no': 2, 'rubber': 3, 'large': 4, '0': 5, 'sphere': 6, 'gray': 7, 'cube': 8, 'blue': 9, 'brown': 10, '1': 11, 'yellow': 12, 'purple': 13, 'cylinder': 14, 'small': 15, 'green': 16, 'metal': 17, '3': 18, '4': 19, 'cyan': 20, '6': 21, 'red': 22, '5': 23, '8': 24, '7': 25, '9': 26, '10': 27} 28 {'<PAD>': 0, '<UNK>': 1, '<START>': 2, '<END>': 3, 'are': 4, 'there': 5, 'more': 6, 'big': 7, 'green': 8, 'things': 9, 'than': 10, 'large': 11, 'purple': 12, 'shiny': 13, 'cubes': 14, 'yes': 15, 'how': 16, 'many': 17, 'other': 18, 'of': 19, 'the': 20, 'same': 21, 'shape': 22, 'as': 23, 'tiny': 24, 'cyan': 25, 'matte': 26, 'object': 27, '2': 28, 'is': 29, 'color': 30, 'sphere': 31, 'cube': 32, 'no': 33, 'what': 34, 'material': 35, 'that': 36, 'right': 37, 'brown': 38, 'cylinder': 39, 'and': 40, 'left': 41, 'rubber': 42, 'gray': 43, 'on': 44, 'side': 45, 'small': 46, 'behind': 47, 'thing': 48, '0': 49, 'to': 50, 'metallic': 51, 'size': 52, 'any': 53, 'have': 54, 'block': 55, 'blue': 56, 'yellow': 57, 'a': 58, ';': 59, 'it': 60, 'ball': 61, 'its': 62, 'in': 63, 'front': 64, 'does': 65, 'number': 66, 'red': 67, 'spheres': 68, 'made': 69, 'metal': 70, 'cylinders': 71, '1': 72, 'both': 73, 'balls': 74, 'or': 75, 'blocks': 76, 'objects': 77, 'visible': 78, 'another': 79, 'has': 80, 'greater': 81, 'fewer': 82, 'less': 83, '3': 84, '4': 85, 'either': 86, 'anything': 87, 'else': 88, 'do': 89, '6': 90, 'an': 91, 'equal': 92, '5': 93, '8': 94, '7': 95, '9': 96, '10': 97} 98 took 0.00 seconds [1mVectorizing data...[0m took 13.70 seconds took [1m[34m89.90[0m seconds [1mBuilding model...[0m took [1m[34m17.59[0m seconds Traceback (most recent call last): File "main.py", line 802, in main() File "main.py", line 691, in main epoch = loadWeights(sess, saver, init) File "main.py", line 190, in loadWeights config.restoreEpoch, config.lr = lastLoggedEpoch() File "main.py", line 62, in lastLoggedEpoch epoch = int(lastLine[0]) ValueError: invalid literal for int() with base 10: 'epoch'

Thank you very much in advance for the help! :) Take care :)

sunchipsster1 avatar Jun 26 '20 04:06 sunchipsster1