returnn icon indicating copy to clipboard operation
returnn copied to clipboard
verified publisher icon rwth-i6

test_ConvLayer_empty_out fails: InvalidArgumentError: Incompatible shapes: [7] vs. [1,1,5]

Open albertz opened this issue 2 months ago • 1 comments

CI run log tf-tests (3.8, 2.10.0, TEST=TFNetworkLayer).

Python env: python is /opt/hostedtoolcache/Python/3.8.18/x64/bin/python Python 3.8.18
NumPy: 1.24.4
TensorFlow: v2.10.0-rc3-6-g359c3cdfc5f 2.10.0 /home/runner/.local/lib/python3.8/site-packages/tensorflow/__init__.py

Relevant log:

___________________________ test_ConvLayer_empty_out ___________________________
Traceback (most recent call last):
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1378, in BaseSession._do_call
...
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1454, in BaseSession._call_tf_sessionrun
...
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [7] vs. [1,1,5]
	 [[{{node output/add_1}}]]

...
  File "/home/runner/work/returnn/returnn/tests/test_TFNetworkLayer.py", line 8775, in test_ConvLayer_empty_out
    line: net.construct_from_dict(
              {
                  # Use filter_size 2 and T=1 to get 0 size out.
                  # Using filter_size 3 would result in negative size according to the formula.
                  # Actually I would have expected that TF also deals with this but this is not the case.
                  "output": {"class": "conv", "n_out": 7, "filter_size": [2], "padding": "valid", "from": "data"},
              }
          )
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 831, in construct_from_dict
    line: self.construct_layer(net_dict, name, get_layer=get_layer)
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1219, in construct_layer
    line: return add_layer(name=name_with_prefix, layer_class=layer_class, **layer_desc)
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1416, in add_layer
    line: layer = self._create_layer(name=name, layer_class=layer_class, **layer_desc)
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1310, in _create_layer
    line: layer = layer_class(**layer_desc)
  File "/home/runner/work/returnn/returnn/returnn/tf/layers/basic.py", line 6531, in module.__init__
    line: y += b

More full log:

___________________________ test_ConvLayer_empty_out ___________________________
Traceback (most recent call last):
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1378, in BaseSession._do_call
...
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1454, in BaseSession._call_tf_sessionrun
...
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [7] vs. [1,1,5]
	 [[{{node output/add_1}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 341, in CallInfo.from_call
...
  File "/home/runner/work/returnn/returnn/tests/test_TFNetworkLayer.py", line 8786, in test_ConvLayer_empty_out
    line: out, seq_lens = session.run(
              [out_.placeholder, out_.size_placeholder[0]], feed_dict=make_feed_dict(net.extern_data, n_time=1, n_batch=1)
          )
    locals:
      session = <local> <tensorflow.python.client.session.Session object at 0x7f0e3cf2bd30>
      out_ = <local> Tensor{'output_output', [B,T|'time:var:extern_data:data+-1'[B],F|F'output:channel'(7)]}
      out_.placeholder = <local> <tf.Tensor 'output/add_1:0' shape=(?, ?, 7) dtype=float32>
      out_.size_placeholder = <local> {0: <tf.Tensor 'output/Maximum:0' shape=(?,) dtype=int32>}, len = 1
      net = <local> <TFNetwork '' train=False>
      net.extern_data = <local> <ExternData data={'data': Tensor{'data', [B,T|'time:var:extern_data:data'[B],F|F'feature:data'(5)]}}>
...
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

Detected at node 'output/add_1' defined at (most recent call last):
    File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
      line: return _run_code(code, main_globals, None,
                             "__main__", mod_spec)
    File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 87, in _run_code
      line: exec(code, run_globals)
    File "/home/runner/.local/lib/python3.8/site-packages/pytest/__main__.py", line 9, in <module>
      line: raise SystemExit(pytest.console_main())
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 201, in console_main
      line: code = main()
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 175, in main
      line: ret: ExitCode | int = config.hook.pytest_cmdline_main(config=config)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
      line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
      line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
      line: res = hook_impl.function(*args)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 330, in pytest_cmdline_main
      line: return wrap_session(config, _main)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 283, in wrap_session
      line: session.exitstatus = doit(config, session) or 0
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 337, in _main
      line: config.hook.pytest_runtestloop(session=session)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
      line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
      line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
      line: res = hook_impl.function(*args)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 362, in pytest_runtestloop
      line: item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
      line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
      line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
      line: res = hook_impl.function(*args)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 113, in pytest_runtest_protocol
      line: runtestprotocol(item, nextitem=nextitem)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 132, in runtestprotocol
      line: reports.append(call_and_report(item, "call", log))
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 241, in call_and_report
      line: call = CallInfo.from_call(
                lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
            )
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
      line: result: TResult | None = func()
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
      line: lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
      line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
      line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
      line: res = hook_impl.function(*args)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
      line: item.runtest()
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/python.py", line 1627, in runtest
      line: self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
      line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
      line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
      line: res = hook_impl.function(*args)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
      line: result = testfunction(**testargs)
    File "/home/runner/work/returnn/returnn/tests/test_TFNetworkLayer.py", line 8775, in test_ConvLayer_empty_out
      line: net.construct_from_dict(
                {
                    # Use filter_size 2 and T=1 to get 0 size out.
                    # Using filter_size 3 would result in negative size according to the formula.
                    # Actually I would have expected that TF also deals with this but this is not the case.
                    "output": {"class": "conv", "n_out": 7, "filter_size": [2], "padding": "valid", "from": "data"},
                }
            )
    File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 831, in construct_from_dict
      line: self.construct_layer(net_dict, name, get_layer=get_layer)
    File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1219, in construct_layer
      line: return add_layer(name=name_with_prefix, layer_class=layer_class, **layer_desc)
    File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1416, in add_layer
      line: layer = self._create_layer(name=name, layer_class=layer_class, **layer_desc)
    File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1310, in _create_layer
      line: layer = layer_class(**layer_desc)
    File "/home/runner/work/returnn/returnn/returnn/tf/layers/basic.py", line 6531, in module.__init__
      line: y += b
Node: 'output/add_1'
Incompatible shapes: [7] vs. [1,1,5]
	 [[{{node output/add_1}}]]

Original stack trace for 'output/add_1':
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    line: return _run_code(code, main_globals, None,
                           "__main__", mod_spec)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 87, in _run_code
    line: exec(code, run_globals)
  File "/home/runner/.local/lib/python3.8/site-packages/pytest/__main__.py", line 9, in <module>
    line: raise SystemExit(pytest.console_main())
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 201, in console_main
    line: code = main()
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 175, in main
    line: ret: ExitCode | int = config.hook.pytest_cmdline_main(config=config)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    line: res = hook_impl.function(*args)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 330, in pytest_cmdline_main
    line: return wrap_session(config, _main)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 283, in wrap_session
    line: session.exitstatus = doit(config, session) or 0
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 337, in _main
    line: config.hook.pytest_runtestloop(session=session)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    line: res = hook_impl.function(*args)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 362, in pytest_runtestloop
    line: item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    line: res = hook_impl.function(*args)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 113, in pytest_runtest_protocol
    line: runtestprotocol(item, nextitem=nextitem)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 132, in runtestprotocol
    line: reports.append(call_and_report(item, "call", log))
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 241, in call_and_report
    line: call = CallInfo.from_call(
              lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
          )
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
    line: result: TResult | None = func()
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
    line: lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    line: res = hook_impl.function(*args)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
    line: item.runtest()
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/python.py", line 1627, in runtest
    line: self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    line: res = hook_impl.function(*args)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
    line: result = testfunction(**testargs)
  File "/home/runner/work/returnn/returnn/tests/test_TFNetworkLayer.py", line 8775, in test_ConvLayer_empty_out
    line: net.construct_from_dict(
              {
                  # Use filter_size 2 and T=1 to get 0 size out.
                  # Using filter_size 3 would result in negative size according to the formula.
                  # Actually I would have expected that TF also deals with this but this is not the case.
                  "output": {"class": "conv", "n_out": 7, "filter_size": [2], "padding": "valid", "from": "data"},
              }
          )
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 831, in construct_from_dict
    line: self.construct_layer(net_dict, name, get_layer=get_layer)
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1219, in construct_layer
    line: return add_layer(name=name_with_prefix, layer_class=layer_class, **layer_desc)
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1416, in add_layer
    line: layer = self._create_layer(name=name, layer_class=layer_class, **layer_desc)
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1310, in _create_layer
    line: layer = layer_class(**layer_desc)
  File "/home/runner/work/returnn/returnn/returnn/tf/layers/basic.py", line 6531, in module.__init__
    line: y += b
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 150, in error_handler
    line: return fn(*args, **kwargs)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/ops/math_ops.py", line 1407, in binary_op_wrapper
    line: return func(x, y, name=name)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 150, in error_handler
    line: return fn(*args, **kwargs)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 1176, in op_dispatch_handler
    line: return dispatch_target(*args, **kwargs)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/ops/math_ops.py", line 1757, in _add_dispatch
    line: return gen_math_ops.add_v2(x, y, name=name)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/ops/gen_math_ops.py", line 475, in add_v2
    line: _, _, _op, _outputs = _op_def_library._apply_op_helper(
                "AddV2", x=x, y=y, name=name)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/framework/op_def_library.py", line 797, in _apply_op_helper
    line: op = g._create_op_internal(op_type_name, inputs, dtypes=None,
                                     name=scope, input_types=input_types,
                                     attrs=attr_protos, op_def=op_def)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3800, in _create_op_internal
    line: ret = Operation(
              node_def,
              self,
              inputs=inputs,
              output_types=dtypes,
              control_inputs=control_inputs,
              input_types=input_types,
              original_op=self._default_original_op,
              op_def=op_def)

----------------------------- Captured stdout call -----------------------------
layer /'data': [B,T|'time:var:extern_data:data'[B],F|F'feature:data'(5)] float32
layer /'output': [B,T|'time:var:extern_data:data+-1'[B],F|F'output:channel'(7)] float32
Tensor{'output_output', [B,T|'time:var:extern_data:data+-1'[B],F|F'output:channel'(7)]}
Tensor{'data', [B,T|'time:var:extern_data:data'[B],F|F'feature:data'(5)]} <tf.Tensor 'extern_data/placeholders/data/data:0' shape=(?, ?, 5) dtype=float32>: shape [1, 1, 5]
=============================== warnings summary ===============================
../../../.local/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:326
  /home/runner/.local/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:326: DeprecationWarning: `np.bool8` is a deprecated alias for `np.bool_`.  (Deprecated NumPy 1.24)
    np.bool8: (False, True),

returnn/tf/network.py:1998
  /home/runner/work/returnn/returnn/returnn/tf/network.py:1998: SyntaxWarning: "is" with a literal. Did you mean "=="?
    if loss is 0:

returnn/tf/util/basic.py:2219
  /home/runner/work/returnn/returnn/returnn/tf/util/basic.py:2219: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    if dim is not 1:

returnn/tf/util/basic.py:6229
  /home/runner/work/returnn/returnn/returnn/tf/util/basic.py:6229: SyntaxWarning: "is" with a literal. Did you mean "=="?
    if start is 0 and stop is None:

returnn/tf/layers/base.py:1713
  /home/runner/work/returnn/returnn/returnn/tf/layers/base.py:1713: SyntaxWarning: "is" with a literal. Did you mean "=="?
    if c is 0:

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/test_TFNetworkLayer.py::test_ConvLayer_empty_out - tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

Detected at node 'output/add_1' defined at (most recent call last):
    File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
      line: return _run_code(code, main_globals, None,
                             "__main__", mod_spec)
    File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 87, in _run_code
      line: exec(code, run_globals)
    File "/home/runner/.local/lib/python3.8/site-packages/pytest/__main__.py", line 9, in <module>
      line: raise SystemExit(pytest.console_main())
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 201, in console_main
      line: code = main()
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 175, in main
      line: ret: ExitCode | int = config.hook.pytest_cmdline_main(config=config)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
      line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
      line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
      line: res = hook_impl.function(*args)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 330, in pytest_cmdline_main
      line: return wrap_session(config, _main)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 283, in wrap_session
      line: session.exitstatus = doit(config, session) or 0
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 337, in _main
      line: config.hook.pytest_runtestloop(session=session)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
      line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
      line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
      line: res = hook_impl.function(*args)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 362, in pytest_runtestloop
      line: item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
      line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
      line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
      line: res = hook_impl.function(*args)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 113, in pytest_runtest_protocol
      line: runtestprotocol(item, nextitem=nextitem)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 132, in runtestprotocol
      line: reports.append(call_and_report(item, "call", log))
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 241, in call_and_report
      line: call = CallInfo.from_call(
                lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
            )
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
      line: result: TResult | None = func()
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
      line: lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
      line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
      line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
      line: res = hook_impl.function(*args)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
      line: item.runtest()
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/python.py", line 1627, in runtest
      line: self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
      line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
      line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
      line: res = hook_impl.function(*args)
    File "/home/runner/.local/lib/python3.8/site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
      line: result = testfunction(**testargs)
    File "/home/runner/work/returnn/returnn/tests/test_TFNetworkLayer.py", line 8775, in test_ConvLayer_empty_out
      line: net.construct_from_dict(
                {
                    # Use filter_size 2 and T=1 to get 0 size out.
                    # Using filter_size 3 would result in negative size according to the formula.
                    # Actually I would have expected that TF also deals with this but this is not the case.
                    "output": {"class": "conv", "n_out": 7, "filter_size": [2], "padding": "valid", "from": "data"},
                }
            )
    File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 831, in construct_from_dict
      line: self.construct_layer(net_dict, name, get_layer=get_layer)
    File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1219, in construct_layer
      line: return add_layer(name=name_with_prefix, layer_class=layer_class, **layer_desc)
    File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1416, in add_layer
      line: layer = self._create_layer(name=name, layer_class=layer_class, **layer_desc)
    File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1310, in _create_layer
      line: layer = layer_class(**layer_desc)
    File "/home/runner/work/returnn/returnn/returnn/tf/layers/basic.py", line 6531, in module.__init__
      line: y += b
Node: 'output/add_1'
Incompatible shapes: [7] vs. [1,1,5]
	 [[{{node output/add_1}}]]

Original stack trace for 'output/add_1':
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    line: return _run_code(code, main_globals, None,
                           "__main__", mod_spec)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 87, in _run_code
    line: exec(code, run_globals)
  File "/home/runner/.local/lib/python3.8/site-packages/pytest/__main__.py", line 9, in <module>
    line: raise SystemExit(pytest.console_main())
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 201, in console_main
    line: code = main()
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 175, in main
    line: ret: ExitCode | int = config.hook.pytest_cmdline_main(config=config)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    line: res = hook_impl.function(*args)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 330, in pytest_cmdline_main
    line: return wrap_session(config, _main)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 283, in wrap_session
    line: session.exitstatus = doit(config, session) or 0
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 337, in _main
    line: config.hook.pytest_runtestloop(session=session)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    line: res = hook_impl.function(*args)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/main.py", line 362, in pytest_runtestloop
    line: item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    line: res = hook_impl.function(*args)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 113, in pytest_runtest_protocol
    line: runtestprotocol(item, nextitem=nextitem)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 132, in runtestprotocol
    line: reports.append(call_and_report(item, "call", log))
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 241, in call_and_report
    line: call = CallInfo.from_call(
              lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
          )
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call
    line: result: TResult | None = func()
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 242, in <lambda>
    line: lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    line: res = hook_impl.function(*args)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
    line: item.runtest()
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/python.py", line 1627, in runtest
    line: self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
    line: return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
    line: return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
    line: res = hook_impl.function(*args)
  File "/home/runner/.local/lib/python3.8/site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
    line: result = testfunction(**testargs)
  File "/home/runner/work/returnn/returnn/tests/test_TFNetworkLayer.py", line 8775, in test_ConvLayer_empty_out
    line: net.construct_from_dict(
              {
                  # Use filter_size 2 and T=1 to get 0 size out.
                  # Using filter_size 3 would result in negative size according to the formula.
                  # Actually I would have expected that TF also deals with this but this is not the case.
                  "output": {"class": "conv", "n_out": 7, "filter_size": [2], "padding": "valid", "from": "data"},
              }
          )
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 831, in construct_from_dict
    line: self.construct_layer(net_dict, name, get_layer=get_layer)
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1219, in construct_layer
    line: return add_layer(name=name_with_prefix, layer_class=layer_class, **layer_desc)
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1416, in add_layer
    line: layer = self._create_layer(name=name, layer_class=layer_class, **layer_desc)
  File "/home/runner/work/returnn/returnn/returnn/tf/network.py", line 1310, in _create_layer
    line: layer = layer_class(**layer_desc)
  File "/home/runner/work/returnn/returnn/returnn/tf/layers/basic.py", line 6531, in module.__init__
    line: y += b
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 150, in error_handler
    line: return fn(*args, **kwargs)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/ops/math_ops.py", line 1407, in binary_op_wrapper
    line: return func(x, y, name=name)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 150, in error_handler
    line: return fn(*args, **kwargs)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 1176, in op_dispatch_handler
    line: return dispatch_target(*args, **kwargs)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/ops/math_ops.py", line 1757, in _add_dispatch
    line: return gen_math_ops.add_v2(x, y, name=name)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/ops/gen_math_ops.py", line 475, in add_v2
    line: _, _, _op, _outputs = _op_def_library._apply_op_helper(
                "AddV2", x=x, y=y, name=name)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/framework/op_def_library.py", line 797, in _apply_op_helper
    line: op = g._create_op_internal(op_type_name, inputs, dtypes=None,
                                     name=scope, input_types=input_types,
                                     attrs=attr_protos, op_def=op_def)
  File "/home/runner/.local/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3800, in _create_op_internal
    line: ret = Operation(
              node_def,
              self,
              inputs=inputs,
              output_types=dtypes,
              control_inputs=control_inputs,
              input_types=input_types,
              original_op=self._default_original_op,
              op_def=op_def)

Not sure why I get this now. And whether this is just a random CI hickup (but it doesn't look like one), or sth else.

albertz avatar Oct 06 '25 10:10 albertz

Another instance of this error: https://github.com/rwth-i6/returnn/actions/runs/18658137586/job/53192158861

albertz avatar Oct 20 '25 16:10 albertz