proposal: Go 2: multi-type type parameters

[DeedleFake]

Update

As pointed out by @seankhliao, the pieces of this involving channels were already covered in #41148. However, I think that the generics-related code is still worse discussing. Therefore, this proposal is now only for the ... constraint detailed below. By limiting the proposal to this constraint, that also partially fixes the reflect issue, as

func Example[T ...](v T) {
  reflect.ValueOf(v)
}

would be illegal. A variable of a type constrained by ..., such as T ..., would not be allowed to be assigned to anything not also of that exact type T, meaning that even passing it to something that takes an any, such as reflect.ValueOf(), would be illegal.

However, this doesn't fix it completely. In order for this proposal to be useful, things like type Future[T ...] struct { val T } would have to be allowed, which would mean that reflect could still gain access to the 'variable' via the struct. I'm unsure of how to handle this, though I have a few ideas. One of them is touched on briefly in the original proposal below.

Original Proposal

Author background

• Would you consider yourself a novice, intermediate, or experienced Go programmer? Experienced.
• What other languages do you have experience with? Ruby, C, Java, Kotlin, Python, JavaScript.

Related proposals

• Has this idea, or one like it, been proposed before? Yes.
  • If so, how does this proposal differ? This is not a proposal for general tuples. Instead, it simply extends the idea of multiple returns to apply to more things than just functions.
• Does this affect error handling? Not directly, but possibly.
• Is this about generics? Yes.
  • If so, how does this relate to the accepted design and other generics proposals? It adds some new features.

Proposal

• What is the proposed change?

In Go, functions can return multiple values. This is handled directly, rather than via tuples, the way that a lot of other languages, such as Python and Rust. handle it. Because of this, some expressions in which it would be useful are not capable of yielding multiple 'returns' directly. For example, a user can declare a chan int, but not a chan (int, error). Due to the convention of returning error from functions directly as their own value, this can lead to some complication when trying to handle errors in other cases, such as when sent from a channel. The common solutions are to either use two channels, one for values and one for errors, or to define a top-level type that can hold the value for you. The first solution is better suited to specific situations, while the second solution mostly solves the problem, but can lead to namespace pollution and other annoyances, especially when the channel operation is happening across package boundaries.

This proposal is not a proposal for general tuples. Instead, this proposal is to extend the usage of multiple 'returns' to other places in which types are specified. These 'tuples' would not be possible to store in a single variable except for in one specific situation, but would instead require that they be destructured immediately, just like how multiple returns from a function works. Unfortunately, this can lead to some ambiguities, so these would have to introduce a small new syntax change to fix, but it would only apply in those caes.

c := make(chan (int, error)) // This would become legal.
v := <-c // This is _not_ legal. There is no real tuple type, so v would not have a valid type.
i, err := <-c // Also not legal due to ambiguity. Is err the bool indicating channel closure? Unclear.
(i, err) := <-c // Legal. This syntax is currently not legal, and it fixes the ambiguity.
(i, err), ok := <-c // Legal. It's now obvious which variable is the bool indicating closure.

// It might make sense to extend the usage to maps and slices, too, but unlike with channels I'm not sure how useful that would be.
c := make(map[string](int, error)) // Key would not be allowed to be multiple types due to lack of comparability.
c := make([](string, int), 0, 10)

// Multiple types would also be available for generics. This is where things get weird, since you can then kind of have variables with
// multiple types. Even limiting multiple types to any doesn't work because you can, for example, still take an address of a variable with
// an any-constrained type. A new pseudo-constraint, here represented by ..., would work similarly to any but would allow multiple types
// and disallow addressing and other complicating operations.
func Lazy[T ...](f func() T) func() T {
  var once sync.Once
  var val T // Would be illegal directly, but is fine here because of the constraints.
  return func() {
    once.Do(func() { val = f() }) // Just works.
    return val
  }
}

getDB := Lazy(OpenDB)
r := getDB() // Illegal, the same as for channels above.
db, err := getDB() // Legal.

The biggest complication that I can think of with these changes is how to handle them with reflect. I think the simplest approach is to introduce the idea directly into reflect.Type with some new methods that would allow you to pull out the types one by one, much like how it currently works for multiple returns to a function. This is potentially a backwards-compatibility issue, as existing code that didn't expect, for example, the element type of a channel to itself have multiple types would break when then given a channel that does have multiple types.

• Who does this proposal help, and why?

It would be quite useful for a number of recent proposals. For example, in #56102 the current thinking is to add both Lazy[T any] and Lazy2[T1, T2 any] just to allow people to use either functions that do or functions that do not return an error. This has happened in a number of other places as well. This proposal would allow that to be replaced Lazy[T ...] instead to handle not only those cases but any number of returns from the wrapped function. #56461 is another that would benefit.

• What would change in the language spec? Quite a few things. I'm not sure exactly what would be necessary to change.
• Please also describe the change informally, as in a class teaching Go. Instead of only functions being able to return multiple values, channels would be able to as well. Similarly, generics would get a special constraint allowing them to do so.
• Is this change backward compatible? Yes, except for the possible reflect issue detailed above.
• Orthogonality: how does this change interact or overlap with existing features? It extends an existing feature to more places.
• Is the goal of this change a performance improvement? No.

Costs

• Would this change make Go easier or harder to learn, and why? Harder. It adds some complication in terms of receiving from channels and fetching things from maps in particular, but it also just generally adds costs by adding some extra complexity to channels, maps, slices, and some usages of generics.
• What is the cost of this proposal? (Every language change has a cost). Some added complexity to certain types and usages of types.
• How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected? gopls and gofmt would definitely be affected because they would need to support the new syntax, but it should be relatively minimal.
• What is the compile time cost? Some, probably, but not much.
• What is the run time cost? None.
• Can you describe a possible implementation? No.
• Do you have a prototype? (This is not required.) No.

[seankhliao]

Is this meaningfully different from #41148 ?

[DeedleFake]

As far as channels specifically go, no I don't think so, but I think the generic usage is the most important part of this one. I was primarily inspired by #56102. I would be fine, given #41148, with removing everything except for the generic-specific part of this proposal.

Edit: I updated the original post to explain that the parts of this proposal that are not related to generics should be ignored.

[leaxoy]

If I understand correctly, this is called Variadic Generics in other languages?

[DeedleFake]

Yes, I believe so. See https://en.wikipedia.org/wiki/Variadic_template. The Go draft proposal for generics also lists

No variadic type parameters. There is no support for variadic type parameters, which would permit writing a single generic function that takes different numbers of both type parameters and regular parameters.

in the Omissions section.

At least to me, I think that the time that Go has now had generics for has shown that the way that the language handles multiple returns, and their usage in the error handling model in particular, makes the addition of variadic generics quite attractive.

[ianlancetaylor]

I think it could be pretty confusing to have a construct like var val T in your example code create multiple variables.

We also need to specify exactly what is permitted with T defined as [T ...]. It seems quite limiting. In C++ variadic function templates are quite a powerful construct, but to do that relies on features that Go does not have such as overloading and SFINAE. We probably don't want to be as powerful as C++, but we probably want to be able to do a little but more than just calling a function that returns multiple results.

[DeedleFake]

I've been thinking about what functionality would be required at a minimum for [T ...] to be useful. The list I have currently is

• a = b where a and b are both T.
• func (e E[T]) M() T, with actual usage working as though it had the same number of returns as types represented by T. In other words, if T was defined as (int, string, error), the user would have to do i, s, err := e.M() the same as if those were M()'s directly defined return types.
• func (e E[T]) M(v T). Both this and the previous one would apply equally to top-level functions, not just methods.
• type E struct { v T }. I think it makes sense to limit struct fields to unexported only. This would reduce the ability to assign directly from the field to something outside. ~~In other words, i, s, err := e.v wouldn't be possible except for inside the same package, though maybe that could be limited, too.~~ This would require other users to access the values via functions and methods, which would work the same as normal multi-return functions and methods do now.
• func NewE[T ...]() E[T].
• Zero values. This would just be the equivalent of all the zero values of the actual list of types.

Also, if a T was used in a list of other types, i.e. something like func E[T ...]() (T, error), it would treat it as though the types had been typed in regularly, meaning that E[(int, string)] would be equivalent to func E() (int, string, error).

In terms of access to struct fields, I don't think it would ever be an issue outside of reflect, which could do other things, such as just simply ignore those fields or treat them as autogenerated separate fields. I can't think of any scenario, barring reflect usage, that would allow access to the field without having T defined. You can't defined a function that takes an E directly, and since you need to define an E[T], an equivalent T must be already defined as well.

For reflect, it might be possible to treat the 'field' of the struct as being multiple fields with autogenerated, otherwise impossible names. Something like v[0], v[1], etc. in the example above.

Edit: Let me try to explain what I'm saying above about struct fields more clearly. Given

type Example[T ...] struct {
  v T
}

the following would be allowed

func Something[T ...](e E[T]) {
  a := e.v // a would be of type T.
  // ...
}

but the following would not

func Something(e E[(int, string)]) {
  i, str := e.v // Can't destructure from a variable. Has to be multiple returns from a function or method.
  // ...
}

Instead, you'd have to use a wrapper function

func Get[T ...](e E[T]) T {
  return e.v // Legal because it's just an assignment from T to T with no destructuring.
}

This is why I think it should be limited to unexported fields only. It wouldn't actually be a problem in terms of usage by other packages, but it could be confusing that in some unlikely situations, you wouldn't actually be able to access the values of the field directly and would instead have to write a wrapper function to decompose it. I think that avoiding the confusion outweighs the oddity of disallowing exporting of variadic generic fields.

[DeedleFake]

I think it could be pretty confusing to have a construct like var val T in your example code create multiple variables.

It never would, though. In all cases outside of function arguments are returns, and possible reflect via struct fields, it would act exactly like a single variable, For example,

type Optional[T ...] struct {
  v T // Looks like a single, normal struct field.
  ok bool
}

func NewOptional[T ...](v T, ok bool) Optional[T] {
  return Optional[T]{
    v: v, // Acts like a single variable here.
    ok: ok,
  }
}

func (opt Optional[T]) Or(v T) T {
  if opt.ok {
    return opt.v // Again, acts like a single variable.
  }
  return v
}

But then in usage, it could do something like this:

opt := NewOptional[(int, string)](3, "example", true) // At this end, it acts exactly like the T-typed stuff is more than one argument.
i, str := opt.Or(1, "something") // And the same here in the returns.

So inside of the definition, stuff of 'type' T looks and acts exactly as though it was a single variable with a single type, but outside, where it's actually used, it does exactly the opposite.

Edit: Another example, based on #56102:

package sync

// Adding in arguments just for fun.
func Lazy[A ...,R ...](f func(A) R) func(A) R {
  var once Once
  var r R
  return func(args A) R {
    once.Do(func() { r = f(args) })
    return r
  }
}

[zephyrtronium]

I can think of a number of functions of general forms like func Thunk[In ..., Out ...](func(In) Out, In) func(In) Out or func FoldMap[Init ..., Args ...](func(Init) Args, ...func(Args) Args) Args which would be useful. Rather, I get the sense that this feature would be particularly powerful for many kinds of generic higher-order functions. But I think just writing these signatures is enough to convince me that I don't want this in Go.

[DeedleFake]

The ... syntax is a placeholder. It reads just fine with a predefined identifier:

func Thunk[I various, O various](func(I) O) func(I) O

func FoldMap[I various, A various](func(I) A, ...func(A) A) A

There's nothing particularly strange about these functions signatures. They're not long or particularly complex.

[ianlancetaylor]

Just to check, would this permit writing a single Metric type for the example outlined at https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#metrics ? Thanks.

[DeedleFake]

In the current design, yes, but only the variant mentioned that uses a comparison function, not the one with the comparable constraint. For example,

type Metric[T ...] struct {
  mu sync.Mutex
  eq func(T, T) bool
  // ...
}

func NewMetric[T ...](eq func(T, T) bool) *Metric[T] {
  return &Metric[T]{
    eq: eq,
    // ...
  }
}

func (m *Metric[T]) Add(vals T) {
  m.mu.Lock()
  defer m.mu.Unlock()

  // Do whatever is necessary to add the data, calling m.eq to compare.
}

This does have some weirdness, though, including the fact that comparison function will wind up with two arguments for each type in T. For example, NewMetric[(int, string)](func(i1 int, s1 string, i2 int, s2 string) bool { ... }) would be correct.

It might be possible to allow constraints on variadics, but it would be quite limited. Essentially, it comes down to whether or not it makes sense to define operations on variadics. For example, comparability could be defined much like it is for a struct:

func Example[T ...comparable]() {
  var v1, v2 T
  v1 == v2 // This just makes sense.
}

In this case, a more broad variadic would be written as [T ...any] instead, which does have some nice parity. The issue I see is that I can't think of many other operations besides comparison that aren't available for any and make sense with variadics. Calling methods makes no sense, so that means that essentially every interface that doesn't have a type list would make no sense to constrain a variadic with. You could, but it wouldn't gain you anything. Similarly, basic operators like +, -, and so on wouldn't necessarily make sense, although they could certainly be allowed. For example, something like

func Add[T ...constraints.Integer](v1, v2 T) T {
  return v1 + v2
}

func main() {
  i1, i2 := Add[(int, float64)](1, 2, 3, 4)
}

makes sense, but I'm not sure if it makes sense to allow it or not.

Edit: The constrained version also provides the benefit of allowing structs to maintain comparability. For example, type Example[T ...comparable] struct { v T } would itself satisfy a comparable constraint.

Edit 2: I'm looking over the metrics example some more and I'm wondering if this is a good fit for variadic generics in the first place. It seems to make more sense to me to just define a type Metric[T comparable] struct { ... } and then use comparable struct types for each metric. That approach would result in quite poor ergonomics for something like #56102, but in this case I don't think it would make much difference. For example, instead of httpRequests.Add(1, “GET”, 404), it could use httpRequest.Add(1, requestResult{"GET", 404}).

Edit 3: I realized that I didn't demonstrate how the metric example would work with constraints. Here's the definition working for any number of types:

type Metric[T ...comparable] struct {
  mu sync.Mutex
  m map[key[T]]int
}

func (m *Metric[T]) Add(vals T) {
  m.mu.Lock()
  defer m.mu.Unlock()
  if m.m == nil {
    m.m = make(map[key[T]]int)
  }
  m.m[v]++
}

// Intermediary type necessary for use as a map key.
type key[T ...comparable] struct {
  v T
}

[ianlancetaylor]

Putting on hold for discussion of future generics changes.