go
go copied to clipboard
golang
Generics: cannot use common struct fields of a type set.
What version of Go are you using (go version)?
$ go version /tmp/golang-tip/bin/go version go version devel go1.18-986f8ea6b4 Tue Sep 21 00:59:42 2021 +0000 linux/amd64
Does this issue reproduce with the latest release?
No, it is a generics issue, therefore tested with a recent tip only.
What operating system and processor architecture are you using (go env)?
linux/amd64
go env Output
$ go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/home/me/.cache/go-build" GOENV="/home/me/.config/go/env" GOEXE="" GOEXPERIMENT="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GOMODCACHE="/home/me/src/go/pkg/mod" GONOPROXY="k8s.io/*" GONOSUMDB="" GOOS="linux" GOPATH="/home/me/src/go" GOPRIVATE="" GOPROXY="" GOROOT="/tmp/golang-tip" GOSUMDB="off" GOTMPDIR="" GOTOOLDIR="/tmp/golang-tip/pkg/tool/linux_amd64" GOVCS="" GOVERSION="devel go1.18-986f8ea6b4 Tue Sep 21 00:59:42 2021 +0000" GCCGO="gccgo" GOAMD64="v1" AR="ar" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="/home/me/src/gocrtp/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1474680903=/tmp/go-build -gno-record-gcc-switches"
What did you do?
I tried to compile this program (crtp.go) with generics:
package main
import "fmt"
type Point struct {
X, Y int
}
type Rect struct {
X, Y, W, H int
}
type Elli struct {
X, Y, W, H int
}
func GetX[P interface { Point | Rect | Elli }] (p P) int {
return p.X
}
func main() {
p := Point { 1, 2}
r := Rect {2, 3, 7, 8}
e := Elli {4, 5, 9, 10}
fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}
with tmp/golang-tip/bin/go build
What did you expect to see?
Program compiles, runs and outputs X: 1 2 4
What did you see instead?
./crtp.go:19:11: p.X undefined (type bound for P has no method X)
All three structs in the type bound have an identical X /field/, so I think this is wrong. Of course there is no method but I don't think that matters here. I feel I should be able to use the public field X of p since p can only be one of the three Point, Rect, or Elli.
@griesemer
Reading "Composite types in constraints" in https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md I get the impression that this ought to work; the example there says "this doesn't work because the field types for X don't all match" which implies that if the field types did match, as they do here, then it would work.
package main
import "fmt"
type hasX interface {
struct {
X, Y int
} | struct {
X, Y, H int
} | struct {
X, Y, W, H int
}
}
func GetX[P hasX](p *P) int {
return p.X
}
func main() {
p := struct {
X, Y int
}{1, 2}
r := struct {
X, Y, H int
}{2, 3, 8}
e := struct {
X, Y, W, H int
}{4, 5, 9, 10}
fmt.Printf("X: %d %d %d\n", GetX(&p), GetX(&r), GetX(&e))
}
We don't currently support field accesses of this kind even though the proposal says that this should/could work. We may not support this for Go1.18 as it doesn't seem like an essential feature. There's a trivial work-around that uses a method:
package main
import "fmt"
type Point struct {
X, Y int
}
func (p Point) GetX() int { return p.X }
type Rect struct {
X, Y, W, H int
}
func (r Rect) GetX() int { return r.X }
type Elli struct {
X, Y, W, H int
}
func (e Elli) GetX() int { return e.X }
func GetX[P interface { Point | Rect | Elli; GetX() int }] (p P) int {
return p.GetX()
}
func main() {
p := Point { 1, 2}
r := Rect {2, 3, 7, 8}
e := Elli {4, 5, 9, 10}
fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}
Of course, then you don't need generic code in the first place because you could just use dynamic method dispatch. And maybe you should.
The reason I tried this out is for #48499, wherr the OP directly wants to get fields or pointers to fields from a set of similar structs. In that use case, the overhead of an accessor function and dynamic method dispatch would be not acceptable.
So while it may not be essential, #48499, which is for use with Go language databases, goes to show that it is not academic, but actually would be very useful for existing code, in stead of the feature proposed in that issue. Furthermore it is more consistent and easier to learn to also allow it.
If there is not enough time left to implement this for 1.18, then please consider this for 1.19.
Open question here.
I feel this is a bit restrictive because we have to explicitly say which struct statisfy the constraint. Like for the interface we may want something that will allow us to tell what is the constraint not who can pass. For example :
package main
import "fmt"
type Point struct {
X, Y int
}
type Rect struct {
X, Y, W, H int
}
type Elli struct {
X, Y, W, H int
}
func GetX[P struct { X int }] (p P) int {
// here p is known to have only the field X anything more
return p.X
}
func main() {
p := Point { 1, 2}
r := Rect {2, 3, 7, 8}
e := Elli {4, 5, 9, 10}
fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}
With this example we can pass any struct that has an int field name X, our function is not coupled with some defined type.
func GetX[P struct { X int }] (p P) int {
possibly another way to describe this is..
type Xer struct {
X int
}
func GetX[P ~Xer](p P) int {
return p.X
}
Just stopping by to say that this is exactly something that I was looking for. Basically, how can I avoid explicitly defining the accessors (via an interface), especially in the case when I fully intend on allowing both Get & Set to occur.
I feel that gymnastics are a totally reasonable expectation when I either want only 1 of Get/Set, OR if the Get/Set needs extra explicit handling. In that case, an interface makes sense. Otherwise, it's just boilerplate.
I believe that this kind of generic structural typing would be extremely useful, and necessary for systems that have performance requirements that dynamic dispatch cannot handle.
That being said the above suggestions for type sets with struct members only being structural seems to be against the grain of the type set. If I were to write the follow:
type Named interface {
string | struct { Name string }
}
I expect the set of types to be the set that consists of string and the struct literal that Has a Name property of type string.
Not the set of string and any struct with property Name. Adding the ~ operator does help express perhaps that we want structural typing for the struct, but it is inconsistent with what came before in 1.18 where the meaning of the ~ operator is to mean any type's who's underlying type is what follows the operator.
Therefore, and this might be extremely unpopular, would it not make more sense to extend the constraint/interface syntax to allow for struct members?
type Xer interface {
X int
}
In the same way that type XGetter interface{ GetX() int } represents the set of types that implement the method GetX, Xer would be the set of types that have an accessor X ?
This way we don't need to touch what the tilde operator means, or how to interpret type unions? Otherwise I think we would be taking two steps backwards in regards to eventually supporting sum types (if ever).
EDIT:
To add a more concrete scenario suppose:
type Point1D { X int }
type Point2D { X, Y int }
type Point3D { X, Y, Z int }
// type constraint
type TwoDimensional interface { X, Y int }
// Works for any struct with X and Y of type int, including Point2D and Point3D, etc.
func SomePlanar2DOperation[T TwoDimensional](value T) { ... }
Many interesting ideas here. But, if the use case of my original examples could work somehow, I don't mind the implementation details.
I think my idea might need to be rewritten as a proposal. As far as whether common fields of a struct union should usable from the function body, I agree. But somebody who knows if it is implementable should comment.
I have a real scenario where I'm working on ETL jobs that work nicely with generics. It removes the necessity to do type checking with switch.
The only thing that prevents it to work is assigning a value of a Field to the struct. And since there is a lot of struct object and each has a lot of properties, adding setter / getter for each struct seems to defeat the whole purpose.
It is strange that if all the structs have the same underlying type, then the code works.
package main
import "fmt"
type Point struct {
X, Y, W, H int
}
type Rect struct {
X, Y, W, H int
}
type Elli struct {
X, Y, W, H int
}
func GetX[P interface { Point | Rect | Elli }] (p P) int {
return p.X
}
func main() {
p := Point { 1, 2, 0, 0}
r := Rect {2, 3, 7, 8}
e := Elli {4, 5, 9, 10}
fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}
However, I didn't find the ~ sign in the code.
[Edit], maybe the reason why it works is the constraint has a core type: https://tip.golang.org/ref/spec#Core_types
Promoted fields also work (if their underlying types are the same):
package main
import "fmt"
type Base struct {
X, Y, W, H int
}
type Point struct {
Base
}
type Rect struct {
Base
}
type Elli struct {
Base
}
func GetX[P interface { Point | Rect | Elli }] (p P) int {
return p.X // okay
}
func main() {
p := Point {}
r := Rect {}
e := Elli {}
fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}
But hybrid promoted and direct fields don't work:
package main
import "fmt"
type Point struct {
X, Y, W, H int
}
type Rect struct {
Point
}
type Elli struct {
Point
}
func GetX[P interface { Point | Rect | Elli }] (p P) int {
return p.X // error: p.X undefined (type P has no field or method X)
}
func main() {
p := Point { 1, 2, 0, 0}
r := Rect {}
e := Elli {}
fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}
Different but with some similarity:
package main
func bar[F func(int) | func(int)int] (f F) {
f(1) // invalid operation: cannot call non-function f (variable of type F constrained by func(int)|func(int) int)
}
func main() {}
@griesemer
We don't currently support field accesses of this kind even though the proposal says that this should/could work.
It doesn't support method access either.
Playground: https://gotipplay.golang.org/p/aEwSDTXYOlL
Of course method access can be solved by adding the common method to the constraint:
type AB interface {
*A | *B
Foo() bool
}
I would like to ask
- does method access not work for the same reason as field access?
- Is there a more formal explanation for why this doesn't work beside "it wasn't implemented"? It'd be useful when explaining this issue to others. It seems to me that the only relevant quote in the Go 1.18 specifications is "A type constraint [...] controls the operations supported by values of that type parameter." and then adding that selector expressions are not operations. Is there anything else to go by?
EDIT: Unlike for field access, the type parameter proposal seemed to be much more explicit about this Method sets of constraint elements:
The method set of a union element is the intersection of the method sets of the elements of the union. These rules are implied by the definition of type sets, but they are not needed for understanding the behavior of constraints.
This quote has then been ported to the Go 1.18 specs:
The method set of an interface type is the intersection of the method sets of each type in the interface's type set (the resulting method set is usually just the set of declared methods in the interface).
Emphasis on usually, which doesn't exclude pushing into the interface's method set the intersection of the methods of the type terms. So... on a second thought, method access not working seems a bug (maybe a documentation bug). Am I misreading something?
Is there a more formal explanation for why this doesn't work beside "it wasn't implemented"? It'd be useful when explaining this issue to others. It seems to me that the only relevant quote in the Go 1.18 specifications is "A type constraint [...] controls the operations supported by values of that type parameter." and then adding that selector expressions are not operations. Is there anything else to go by?
The Go 1.18 release is a huge release and we simply haven't had the bandwidth to get everything done.
@blackgreen100 Note that the method set limitation is explicitly called out in the release notes (https://go.dev/doc/go1.18#generics).
A little disappointed, but as long as the request is not rejected, it is still good.
Sorry for a question that is not related to this topic. Can someone please point me to a documentation for that ~ operator? I tried to google but couldn't find anything. I wonder how it works.
type Xer struct {
X int
}
func GetX[P ~Xer](p P) int {
return p.X
}
The type set of a term of the form
~Tis the set of types whose underlying type is T.
See https://go.dev/ref/spec#General_interfaces
I fee like without support for what was proposed in original post go generics simply incomplete. With few fields maybe we could create getter methods and do the work, but this is very cumbersome and kills the purpose of generics say struct has 10-20 fields and it is not uncommon to have this many.
We agree that the support for generics is not yet complete. We're working on it actively. This issue remains open.
That said, are there cases where generics are helpful with structs of 10 to 20 fields? This feature is only useful if you have multiple structs with that many fields, where the fields you care about have identical names and types, and it's useful to write a complex constraint that is a union of those different struct types. When does this case arise in practice?
I typically have a lot of database models with a consistent set of attributes on them (updated_at, updated_by, created_by, created_at, for example). We have about 7 common attributes (and potentially growing, just talked about adding a some location based attributes) we use for authorization on all our resources and for a generic authz check on a model it be nice to pass in any type as long as it had those attributes. We define our models in proto so while its possible to add a getter method and use an interface, its fairly error prone to ensure every developer remembers to add that method after the go code is generated.
It's important to be clear on what is being suggested here. This is not "pass in any type as long as it had those attributes." This is "pass in one of this specific list of types, each of which has these attributes, and then refer to those attributes in the generic code."
That is, code like this would be permitted.
type S1 struct { F, G int }
type S2 struct { F, H int }
type C interface { S1 | S2 }
func GetF[T C](s T) int { return s.F }
var x = GetF(S1{})
Code like this would not be permitted.
type S struct { F int }
type S1 struct { F, G int }
type S2 struct { F, H int }
type C interface { S }
func GetF[T C](s T) int { return s.F }
var x = GetF(S1{}) // INVALID: S1 does not satisfy constraint C
At work I have some complex data that consist of similar union types stored in the same database table, which have many common fields. For those, this feature would certainly help to implement a closed union type using generics.
@tkinz27 Maybe what you need is described in https://github.com/golang/go/issues/51259.
@ianlancetaylor Thanks for clarifying, I misunderstood.
@go101 yes that is much closer to what I was thinking.
Personally I am more in favor of this issue than #51259, because this does not introduce any new syntax, but only allows semantics that are currently not allowed. This issue would still be very useful for handling the common data of closed unions and other cases of a known set of structs with common fields.
This feature is only useful if you have multiple structs with that many fields, where the fields you care about have identical names and types, and it's useful to write a complex constraint that is a union of those different struct types. When does this case arise in practice?
@ianlancetaylor We have a DAG with many nodes at my company that forms the core of the product we sell. Each node is a struct which contains the same fields ID and ParentID, as well as many other fields unique to each node type. There are also a subset of nodes which have an overlapping 12 additional fields -- 14 in total.
There are 56 different types of nodes. We've needed to implement an interface with getters and setters for each node. It's easy, but it's also tedious to write and maintain and leads to arguably less conventional code in use, since now we need to do GetID() where .ID would be preferred. The file that contains these getter and setter definitions for the nodes is almost 3000 lines long.
