Overloaded generic procvar: useless instantiations that create type mismatch

[mratsim]

Here is a Christmas tree bug that touches upon generic instantiation, procvar, overloading and semcheck.

Scenario 1 - benign impact besides extra NimVM processing

Let's start with a simple example that establish the context. I'm working in tree datastructure and make generous usage of overloading (and generics). In my tree I also need to store a generic handler proc.

When passing the generic handler proc to initialise the tree, Nim will instantiate all overloads that match, even when the initialiser signature restrict to a single instantiation:

import typetraits

type
  MyProc[T] = proc(s: seq[T]): seq[T] {.nimcall.}

  Node[T] = object
    value: seq[T]
    handler: MyProc[T]

proc double[T: SomeNumber](x: T): T =
  static: echo "I'm in double[", x.type.name, "]"
  x + x

proc double[T](x: seq[T]): seq[T] =
  static: echo "I'm in double[", x.type.name, "]"
  result = newSeq[T](x.len)
  for i in 0 ..< x.len:
    result[i]= x[i] + x[i]

proc double[T](x: Node[T]): Node[T] =
  static: echo "I'm in double[", x.type.name, "]"
  result.value = x.value.double()

proc newNode[T](val: seq[T], handler: MyProc[T]): Node[T] =
  result.value = val
  result.handler = handler

let foo = newNode(@[1, 2, 3], double[int])
echo foo.handler.type.name

## Compile-time
# I'm in double[int]        # <---- useless instantiation
# I'm in double[seq[int]]
# I'm in double[tree[int]]  # <---- useless instantiation

## Run-time
# MyProc[system.int]

Scenario 2 - Prevents using overloaded procvar

While scenario 1 only cost is increased compilation-time due to extra instantiation and semantic checking, there is actually a side effect: your code even though correct will not compile if in any of those instantiations there is a call invalid for any of the input types.

Example:

import typetraits

type
  Container[T] = object
    value: T

  # CT for container
  RefWrapper[CT] = ref object
    raw: CT

  MyProc[CT] = proc(s: CT): CT {.nimcall.}

  Node[CT] = object
    payload: RefWrapper[CT]
    handler: MyProc[CT]

proc mul[T](a, b: Container[T]): Container[T] =
  result.value = a.value * b.value

proc mul[CT](a, b: RefWrapper[CT]): RefWrapper[CT] =
  new result
  result.raw = mul(a.raw, b.raw)

proc register_node[CT](payload: RefWrapper[CT], handler: MyProc[CT]) =
  static:
    echo payload.type.name
    echo handler.type.name

proc newRefWrapper[T](ct: Container[T]): RefWrapper[Container[T]] =
  new result
  result.raw = ct

let x = Container[int](value: 2)
let x_ref = newRefWrapper(x)

register_node(x_ref, mul[Container[int]])

# proc_inst2.nim(37, 25) template/generic instantiation of `mul` from here
# proc_inst2.nim(18, 26) Error: type mismatch: got <Container[system.int], Container[system.int]>
# but expected one of:
# proc `*`(x, y: float32): float32
#   first type mismatch at position: 1
#   required type: float32
#   but expression 'a.value' is of type: Container[system.int]

Workaround

Generic procvar should not be overloaded. It might be that not all procvar and no only generic ones have this issue

Proposed long-term solutions

1. If there is a type restriction for procvar, either because we have let: MyProc[int] = mul[Container[int]] or we pass to another proc, do not instantiate the useless ones.

2. Alternatively, delete the useless instantiations before semcheck starts.