numba icon indicating copy to clipboard operation
numba copied to clipboard
verified publisher icon numba

`test_target_extension.TestTargetHierarchySelection.test_overload_allocation` fails when run in isolation

Open gmarkall opened this issue 10 months ago • 4 comments

I suspect there is some relationship with #9954 - perhaps why things appeared to work without that change.

Running TestTargetHierarchySelection as a whole:

$ python -m numba.runtests numba.tests.test_target_extension.TestTargetHierarchySelection
.........
----------------------------------------------------------------------
Ran 9 tests in 0.386s

Running the one test in isolation:

$ python -m numba.runtests numba.tests.test_target_extension.TestTargetHierarchySelection.test_overload_allocation
F
======================================================================
FAIL: test_overload_allocation (numba.tests.test_target_extension.TestTargetHierarchySelection.test_overload_allocation)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/gmarkall/numbadev/numba/numba/tests/test_target_extension.py", line 709, in test_overload_allocation
    r = foo()
        ^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 443, in _compile_for_args
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 376, in _compile_for_args
    return_val = self.compile(tuple(argtypes))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/tests/test_target_extension.py", line 288, in compile
    return super().compile(sig)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 908, in compile
    cres = self._compiler.compile(args, return_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 80, in compile
    status, retval = self._compile_cached(args, return_type)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 94, in _compile_cached
    retval = self._compile_core(args, return_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 107, in _compile_core
    cres = compiler.compile_extra(self.targetdescr.typing_context,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 739, in compile_extra
    return pipeline.compile_extra(func)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 439, in compile_extra
    return self._compile_bytecode()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 505, in _compile_bytecode
    return self._compile_core()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 481, in _compile_core
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 473, in _compile_core
    pm.run(self.state)
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 363, in run
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 356, in run
    self._runPass(idx, pass_inst, state)
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_lock.py", line 35, in _acquire_compile_lock
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 311, in _runPass
    mutated |= check(pss.run_pass, internal_state)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 272, in check
    mangled = func(compiler_state)
              ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typed_passes.py", line 114, in run_pass
    typemap, return_type, calltypes, errs = type_inference_stage(
                                            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typed_passes.py", line 95, in type_inference_stage
    errs = infer.propagate(raise_errors=raise_errors)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typeinfer.py", line 1066, in propagate
    errors = self.constraints.propagate(self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typeinfer.py", line 160, in propagate
    constraint(typeinfer)
  File "/home/gmarkall/numbadev/numba/numba/core/typeinfer.py", line 566, in __call__
    self.resolve(typeinfer, typevars, fnty)
  File "/home/gmarkall/numbadev/numba/numba/core/typeinfer.py", line 589, in resolve
    sig = typeinfer.resolve_call(fnty, pos_args, kw_args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typeinfer.py", line 1560, in resolve_call
    return self.context.resolve_function_type(fnty, pos_args, kw_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/context.py", line 195, in resolve_function_type
    res = self._resolve_user_function_type(func, args, kws)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/context.py", line 247, in _resolve_user_function_type
    return func.get_call_type(self, args, kws)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/types/functions.py", line 311, in get_call_type
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/types/functions.py", line 308, in get_call_type
    sig = temp.apply(nolitargs, nolitkws)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 358, in apply
    sig = generic(args, kws)
          ^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 621, in generic
    disp, new_args = self._get_impl(args, kws)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 720, in _get_impl
    impl, args = self._build_impl(cache_key, args, kws)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 824, in _build_impl
    disp_type.get_call_type(self.context, args, kws)
  File "/home/gmarkall/numbadev/numba/numba/core/types/functions.py", line 538, in get_call_type
    self.dispatcher.get_call_template(args, kws)
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 319, in get_call_template
    self.compile(tuple(args))
  File "/home/gmarkall/numbadev/numba/numba/tests/test_target_extension.py", line 288, in compile
    return super().compile(sig)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 908, in compile
    cres = self._compiler.compile(args, return_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 80, in compile
    status, retval = self._compile_cached(args, return_type)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 94, in _compile_cached
    retval = self._compile_core(args, return_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 107, in _compile_core
    cres = compiler.compile_extra(self.targetdescr.typing_context,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 739, in compile_extra
    return pipeline.compile_extra(func)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 439, in compile_extra
    return self._compile_bytecode()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 505, in _compile_bytecode
    return self._compile_core()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 481, in _compile_core
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 473, in _compile_core
    pm.run(self.state)
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 363, in run
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 356, in run
    self._runPass(idx, pass_inst, state)
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_lock.py", line 35, in _acquire_compile_lock
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 311, in _runPass
    mutated |= check(pss.run_pass, internal_state)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 272, in check
    mangled = func(compiler_state)
              ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typed_passes.py", line 114, in run_pass
    typemap, return_type, calltypes, errs = type_inference_stage(
                                            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typed_passes.py", line 95, in type_inference_stage
    errs = infer.propagate(raise_errors=raise_errors)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typeinfer.py", line 1066, in propagate
    errors = self.constraints.propagate(self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typeinfer.py", line 160, in propagate
    constraint(typeinfer)
  File "/home/gmarkall/numbadev/numba/numba/core/typeinfer.py", line 566, in __call__
    self.resolve(typeinfer, typevars, fnty)
  File "/home/gmarkall/numbadev/numba/numba/core/typeinfer.py", line 589, in resolve
    sig = typeinfer.resolve_call(fnty, pos_args, kw_args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typeinfer.py", line 1560, in resolve_call
    return self.context.resolve_function_type(fnty, pos_args, kw_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/context.py", line 195, in resolve_function_type
    res = self._resolve_user_function_type(func, args, kws)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/context.py", line 247, in _resolve_user_function_type
    return func.get_call_type(self, args, kws)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/types/functions.py", line 311, in get_call_type
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/types/functions.py", line 308, in get_call_type
    sig = temp.apply(nolitargs, nolitkws)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 358, in apply
    sig = generic(args, kws)
          ^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 621, in generic
    disp, new_args = self._get_impl(args, kws)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 720, in _get_impl
    impl, args = self._build_impl(cache_key, args, kws)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 824, in _build_impl
    disp_type.get_call_type(self.context, args, kws)
  File "/home/gmarkall/numbadev/numba/numba/core/types/functions.py", line 538, in get_call_type
    self.dispatcher.get_call_template(args, kws)
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 319, in get_call_template
    self.compile(tuple(args))
  File "/home/gmarkall/numbadev/numba/numba/tests/test_target_extension.py", line 288, in compile
    return super().compile(sig)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 908, in compile
    cres = self._compiler.compile(args, return_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 80, in compile
    status, retval = self._compile_cached(args, return_type)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 94, in _compile_cached
    retval = self._compile_core(args, return_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 107, in _compile_core
    cres = compiler.compile_extra(self.targetdescr.typing_context,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 739, in compile_extra
    return pipeline.compile_extra(func)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 439, in compile_extra
    return self._compile_bytecode()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 505, in _compile_bytecode
    return self._compile_core()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 481, in _compile_core
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 473, in _compile_core
    pm.run(self.state)
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 363, in run
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 356, in run
    self._runPass(idx, pass_inst, state)
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_lock.py", line 35, in _acquire_compile_lock
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 311, in _runPass
    mutated |= check(pss.run_pass, internal_state)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 272, in check
    mangled = func(compiler_state)
              ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typed_passes.py", line 470, in run_pass
    lower.lower()
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 193, in lower
    self.lower_normal_function(self.fndesc)
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 232, in lower_normal_function
    entry_block_tail = self.lower_function_body()
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 262, in lower_function_body
    self.lower_block(block)
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 276, in lower_block
    self.lower_inst(inst)
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 462, in lower_inst
    val = self.lower_assign(ty, inst)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 674, in lower_assign
    return self.lower_expr(ty, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 1268, in lower_expr
    res = self.lower_call(resty, expr)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 939, in lower_call
    res = self._lower_call_normal(fnty, expr, signature)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 1239, in _lower_call_normal
    res = impl(self.builder, argvals, self.loc)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/base.py", line 1196, in __call__
    res = self._imp(self._context, builder, self._sig, args, loc=loc)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/base.py", line 1226, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 1113, in method_impl
    sig = self._get_signature(typing_context, fnty, sig.args, {})
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 1085, in _get_signature
    sig = fnty.get_call_type(typingctx, args, kws)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/types/functions.py", line 311, in get_call_type
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/types/functions.py", line 308, in get_call_type
    sig = temp.apply(nolitargs, nolitkws)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 358, in apply
    sig = generic(args, kws)
          ^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 621, in generic
    disp, new_args = self._get_impl(args, kws)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 720, in _get_impl
    impl, args = self._build_impl(cache_key, args, kws)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typing/templates.py", line 824, in _build_impl
    disp_type.get_call_type(self.context, args, kws)
  File "/home/gmarkall/numbadev/numba/numba/core/types/functions.py", line 538, in get_call_type
    self.dispatcher.get_call_template(args, kws)
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 319, in get_call_template
    self.compile(tuple(args))
  File "/home/gmarkall/numbadev/numba/numba/tests/test_target_extension.py", line 288, in compile
    return super().compile(sig)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 908, in compile
    cres = self._compiler.compile(args, return_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 80, in compile
    status, retval = self._compile_cached(args, return_type)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 94, in _compile_cached
    retval = self._compile_core(args, return_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/dispatcher.py", line 107, in _compile_core
    cres = compiler.compile_extra(self.targetdescr.typing_context,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 739, in compile_extra
    return pipeline.compile_extra(func)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 439, in compile_extra
    return self._compile_bytecode()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 505, in _compile_bytecode
    return self._compile_core()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 481, in _compile_core
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/compiler.py", line 473, in _compile_core
    pm.run(self.state)
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 363, in run
    raise e
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 356, in run
    self._runPass(idx, pass_inst, state)
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_lock.py", line 35, in _acquire_compile_lock
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 311, in _runPass
    mutated |= check(pss.run_pass, internal_state)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/compiler_machinery.py", line 272, in check
    mangled = func(compiler_state)
              ^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/typed_passes.py", line 470, in run_pass
    lower.lower()
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 193, in lower
    self.lower_normal_function(self.fndesc)
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 232, in lower_normal_function
    entry_block_tail = self.lower_function_body()
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 262, in lower_function_body
    self.lower_block(block)
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 276, in lower_block
    self.lower_inst(inst)
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 462, in lower_inst
    val = self.lower_assign(ty, inst)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 674, in lower_assign
    return self.lower_expr(ty, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 1268, in lower_expr
    res = self.lower_call(resty, expr)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 939, in lower_call
    res = self._lower_call_normal(fnty, expr, signature)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/lowering.py", line 1239, in _lower_call_normal
    res = impl(self.builder, argvals, self.loc)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/base.py", line 1196, in __call__
    res = self._imp(self._context, builder, self._sig, args, loc=loc)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/base.py", line 1226, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/np/arrayobj.py", line 4308, in codegen
    meminfo = context.nrt.meminfo_alloc_aligned(builder, allocsize, align)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/runtime/context.py", line 47, in wrap
    memptr = func(self, builder, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/runtime/context.py", line 160, in meminfo_alloc_aligned
    return self.meminfo_alloc_aligned_unchecked(builder, size, align)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmarkall/numbadev/numba/numba/core/runtime/context.py", line 183, in meminfo_alloc_aligned_unchecked
    assert align.type == u32, "align must be a uint32"
           ^^^^^^^^^^^^^^^^^
AssertionError: align must be a uint32

gmarkall avatar Mar 04 '25 11:03 gmarkall

Also:

https://github.com/numba/numba/blob/467136ab48871c36505ab54dda7eff49f1cc5eef/numba/tests/test_target_extension.py#L667-L668

gmarkall avatar Mar 04 '25 12:03 gmarkall

When the test is run in isolation, the wrong intrin_alloc is called. It is using

https://github.com/numba/numba/blob/467136ab48871c36505ab54dda7eff49f1cc5eef/numba/np/arrayobj.py#L4302-L4313

instead of

https://github.com/numba/numba/blob/467136ab48871c36505ab54dda7eff49f1cc5eef/numba/tests/test_target_extension.py#L660-L673

gmarkall avatar Mar 04 '25 12:03 gmarkall

I've attempted to debug this, it's quite a complicated issue. In words, what I think is happening is that, when run in isolation:

  1. The test is adding the types.Array._allocate method for the DPU target to the codegen registry.
  2. A context refresh is called from somewhere in the compilation stack (there's lots of places this might happen). The effect of this is to register the arrayobj version of types.Array._allocate into the codegen registry too by virtue of it being considered a "builtin".
  3. As a result of 2. the OverloadSelector has two versions of the _allocate method for the types.Array type in the order [the DPU definition, the arrayobj.py definition].
  4. As a result of 3. and a consequence of this code https://github.com/numba/numba/blob/e9b52344af4c1ab15153db5aabbe858355cc72d1/numba/core/base.py#L56-L64, the "last registered thing" will "win" selection for the same key, which is why at lowering time it's incorrectly finding the arrayobj.py variant and then failing the assert (because that variant doesn't have the uint32_t cast/it's for the wrong target).

Other observations:

  1. This was introduced by d184335, but it is not to blame. The effect this change had was to move the import of numba.np.arrayobj.make_array to a different place, the side effect being that the overload of type.Array._allocate is registered after the DPU variant.
  2. The registration code in the extension API for methods and attributes is wildly different to that of functions, it's hard to understand where this problem is coming from as there's a lot of side effect, and this path is less well visited (most use cases are function overloads). It's also made more complicated by there being a "trampoline" in lowering to go from the intrinsic doing the implementation of np.empty through to find the actual allocator function for the array buffer.
  3. Adding a target="CPU" kwarg to the decorator https://github.com/numba/numba/blob/e9b52344af4c1ab15153db5aabbe858355cc72d1/numba/np/arrayobj.py#L4286 makes the problem "go away" (it doesn't fix it, it just prevents the generic variant appearing in that .versions list in the OverloadSelector).
  4. Running another test before the failing one often "fixes" it, as the refresh that loads the "builtin" functions into the codegen registry is "lazy", the effect of running another test is that the codegen registry is populated and then subsequent refresh does not attempt to repopulate it. Somehow, and I'm not sure what happens to achieve this as so far I've only observed the effect, this results in there only being a single entry for types.Array._allocate in the OverloadSelector when it comes to running the test_overload_allocation test.

stuartarchibald avatar Mar 06 '25 13:03 stuartarchibald

The problem is weird because no amount of manual calling of refresh() is fixing the problem. But calling a empty dpjit function once does.

I'm wonder if the problems are all due to refresh() and delayed registrations, can we get rid of these.

sklam avatar Mar 14 '25 16:03 sklam