moonlight icon indicating copy to clipboard operation
moonlight copied to clipboard
verified publisher icon tensorflow

Update Travis to try to unbreak the unexplained memory issues.

Open ringw opened this issue 4 years ago • 0 comments

Try to unbreak Travis where we are getting free() crashes at head.

Old build: https://travis-ci.org/github/tensorflow/moonlight/builds/759368001

Stack trace:

oonlight/structure:structure_test:
Running tests under Python 3.6.3: /home/travis/virtualenv/python3.6.3/bin/python
[ RUN      ] StructureTest.testCompute
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/structure/structure_test.py:31: The name tf.resource_loader.get_data_files_path is deprecated. Please use tf.compat.v1.resource_loader.get_data_files_path instead.
W0217 17:35:17.408096 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/structure/structure_test.py:31: The name tf.resource_loader.get_data_files_path is deprecated. Please use tf.compat.v1.resource_loader.get_data_files_path instead.
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/structure/structure_test.py:33: The name tf.read_file is deprecated. Please use tf.io.read_file instead.
W0217 17:35:17.408478 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/structure/structure_test.py:33: The name tf.read_file is deprecated. Please use tf.io.read_file instead.
/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/tensorflow_core/python/framework/tensor_util.py:521: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor_proto.tensor_content = nparray.tostring()
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/run_length.py:55: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W0217 17:35:17.437835 140371440740160 deprecation.py:323] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/run_length.py:55: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/run_length.py:76: The name tf.unsorted_segment_max is deprecated. Please use tf.math.unsorted_segment_max instead.
W0217 17:35:17.486910 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/run_length.py:76: The name tf.unsorted_segment_max is deprecated. Please use tf.math.unsorted_segment_max instead.
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/run_length.py:85: The name tf.bincount is deprecated. Please use tf.math.bincount instead.
W0217 17:35:17.494965 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/run_length.py:85: The name tf.bincount is deprecated. Please use tf.math.bincount instead.
WARNING:tensorflow:From /home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/tensorflow_core/python/ops/array_ops.py:1475: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W0217 17:35:17.524388 140371440740160 deprecation.py:323] From /home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/tensorflow_core/python/ops/array_ops.py:1475: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/segments.py:159: The name tf.segment_mean is deprecated. Please use tf.math.segment_mean instead.
W0217 17:35:17.553140 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/segments.py:159: The name tf.segment_mean is deprecated. Please use tf.math.segment_mean instead.
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/segments.py:164: The name tf.segment_sum is deprecated. Please use tf.math.segment_sum instead.
W0217 17:35:17.557595 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/segments.py:164: The name tf.segment_sum is deprecated. Please use tf.math.segment_sum instead.
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/staves/staffline_distance.py:136: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W0217 17:35:17.679880 140371440740160 deprecation.py:323] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/staves/staffline_distance.py:136: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/vision/hough.py:53: The name tf.ceil is deprecated. Please use tf.math.ceil instead.
W0217 17:35:19.980039 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/vision/hough.py:53: The name tf.ceil is deprecated. Please use tf.math.ceil instead.
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/staves/hough.py:216: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.
    
W0217 17:35:20.235045 140371440740160 deprecation.py:323] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/staves/hough.py:216: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.
    
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/segments.py:157: The name tf.segment_min is deprecated. Please use tf.math.segment_min instead.
W0217 17:35:20.449665 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/segments.py:157: The name tf.segment_min is deprecated. Please use tf.math.segment_min instead.
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/segments.py:89: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
W0217 17:35:20.519207 140371440740160 deprecation.py:323] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/segments.py:89: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/segments.py:95: The name tf.floordiv is deprecated. Please use tf.math.floordiv instead.
W0217 17:35:20.522479 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/util/segments.py:95: The name tf.floordiv is deprecated. Please use tf.math.floordiv instead.
WARNING:tensorflow:From /opt/python/3.6.3/lib/python3.6/contextlib.py:60: TensorFlowTestCase.test_session (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `self.session()` or `self.cached_session()` instead.
W0217 17:35:21.027812 140371440740160 deprecation.py:323] From /opt/python/3.6.3/lib/python3.6/contextlib.py:60: TensorFlowTestCase.test_session (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `self.session()` or `self.cached_session()` instead.
2021-02-17 17:35:21.028494: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2021-02-17 17:35:21.033070: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2800240000 Hz
2021-02-17 17:35:21.033231: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x86908e0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-02-17 17:35:21.033245: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/structure/__init__.py:132: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.
W0217 17:35:21.697415 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/structure/__init__.py:132: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.
WARNING:tensorflow:From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/staves/base.py:45: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
W0217 17:35:33.211079 140371440740160 module_wrapper.py:139] From /home/travis/.cache/bazel/_bazel_travis/a57dff875537c2d2b17575b95ebd3f63/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/moonlight/structure/structure_test.runfiles/__main__/moonlight/staves/base.py:45: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
[       OK ] StructureTest.testCompute
[ RUN      ] StructureTest.test_session
[  SKIPPED ] StructureTest.test_session
----------------------------------------------------------------------
Ran 2 tests in 15.855s
OK (skipped=1)
*** Error in `/home/travis/virtualenv/python3.6.3/bin/python': free(): invalid pointer: 0x00000000025f2b70 ***
Fatal Python error: Aborted
Current thread 0x00007faac5dca740 (most recent call first):
*** Received signal 6 ***
*** BEGIN MANGLED STACK TRACE ***
/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/tensorflow_core/python/../libtensorflow_framework.so.1(+0xfe037d)[0x7faaa6db637d]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7faac5476330]
/lib/x86_64-linux-gnu/libpthread.so.0(raise+0x2b)[0x7faac54761fb]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7faac5476330]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x7faac50d3c37]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7faac50d7028]
/lib/x86_64-linux-gnu/libc.so.6(+0x732a4)[0x7faac51102a4]
/lib/x86_64-linux-gnu/libc.so.6(+0x7f82e)[0x7faac511c82e]
/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/numpy/core/_multiarray_umath.cpython-36m-x86_64-linux-gnu.so(+0x258980)[0x7faac3b18980]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(+0xb8e6f)[0x7faac573ce6f]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(+0xbebd9)[0x7faac5742bd9]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(+0x1bbf22)[0x7faac583ff22]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(_PyGC_CollectNoFail+0x31)[0x7faac5840bb1]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(PyImport_Cleanup+0x106)[0x7faac580c866]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(Py_FinalizeEx+0x7a)[0x7faac581efca]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(Py_Exit+0x8)[0x7faac581fcd8]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(+0x19ec41)[0x7faac5822c41]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(PyErr_PrintEx+0x17d)[0x7faac5822fdd]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(PyRun_SimpleFileExFlags+0x455)[0x7faac5824415]
/opt/python/3.6.3/lib/libpython3.6m.so.1.0(Py_Main+0xec3)[0x7faac583f193]
/home/travis/virtualenv/python3.6.3/bin/python(main+0x16e)[0x400ade]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7faac50bef45]
/home/travis/virtualenv/python3.6.3/bin/python[0x400ba2]
*** END MANGLED STACK TRACE ***
Fatal Python error: Segmentation fault
Current thread 0x00007faac5dca740 (most recent call first):
*** Received signal 11 ***
*** BEGIN MANGLED STACK TRACE ***
/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/tensorflow_core/python/../libtensorflow_framework.so.1(+0xfe037d)[0x7faaa6db637d]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7faac5476330]
/lib/x86_64-linux-gnu/libpthread.so.0(raise+0x2b)[0x7faac54761fb]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7faac5476330]
/lib/x86_64-linux-gnu/libc.so.6(+0x809f9)[0x7faac511d9f9]
/lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x60)[0x7faac511fae0]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_Znwm+0x1d)[0x7faaa5928dad]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59)[0x7faaa5984249]
/lib/x86_64-linux-gnu/libc.so.6(+0x1802a6)[0x7faac521d2a6]
*** END MANGLED STACK TRACE ***

ringw avatar Feb 17 '21 18:02 ringw