go
go copied to clipboard
golang
proposal: encoding/json: add "inline" struct tag
by bjruth:
Discussion was conceived on golang-nuts: https://groups.google.com/forum/#!topic/golang-nuts/bAgNll-EMkI to add support for a flag that supports unmarshaling arbitary JSON into structs. A tag was announced for the mgo/bson package here: https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/ZeP7PaXVDQo that transparently stores data into a separate struct field denoted by "inline" during an unmarshal and then merges that data back into the output during marshaling. I believe this would be a very useful feature to add into the encoding/json package and does not introduce any breaking changes in the Go 1.x series.
Edited by @dsnet on 2020-11-10 to fix formatting.
Comment 1:
I think this is worthwhile. I have an unfinished proposal in my email drafts that is basically exactly what Gustavo describes in his bson release announcement mail. It should be simple enough that it could be done before the 1.2 cutoff. Any disagreement?
Labels changed: added priority-soon, go1.2, feature, removed priority-triage.
Owner changed to @adg.
Status changed to Accepted.
Comment 3:
This issue was closed by revision 466001d05d366cbc97edfb65dc6f5cb883df049.
Status changed to Fixed.
Comment 5:
This issue was closed by revision 27f4166e372084e1d78119183d14b76391c803c.
Status changed to Fixed.
Comment 6:
And that was the undo CL closing it again :/ New CL https://golang.org/cl/13324045/
Status changed to Started.
Comment 9 by bjruth:
@rsc It's not that a manual approach doesn't work, but since the unmarshaler already handles distinguishing known vs. unknown fields relative to the target interface, the extra data can thrown into a "catch-all" field.
Comment 14:
I really need this. I have a struct that has static and dynamic fields. The dynamic fields (in a map) are chosen by the user and need to be inline with the static ones.
Comment 15:
If this is going to be similar to <labix.org/v2/mgo/bson>, ideally an "inline" map or struct pointer would not be allocated (the field value would remain nil) until there is something valid to stuff in it.
Comment 16 by relistan:
+1. I was pondering writing a similar thing and came across this. Essentially we query a 3rd party API that is not versioned and changes. Catching the leftovers would let us better get a sense of what was skipped. At a minimum it's great for debugging.
Comment 17 by [email protected]:
+1 I have something where I have a some common fields (id, path, revision, etc) and I use an anonymous struct that is embedded in other structs. However, I can't implement a custom json marshal on the common struct as then all the other structs will only return the output from the marshalled common fields.
Comment 18 by [email protected]:
+1 I have something where I have some common fields (id, path, revision, etc) and I use an anonymous struct that is embedded in other structs. However, I can't implement a custom json marshal on the common struct as then all the other structs will only return the output from the marshalled common fields.
+1 Example use case is retrieving documents stored in MongoDB with a field like this:
Extra map[string]float32 `bson:",inline"`
to account data that could be missing.
This enables reusing same struct defined for inserting also for reading data.
No, and I think the JSON package is pretty frozen for new features.
But I'll promote this to a proposal. This bug predates the proposal process.
This feature would allow to catch what is called additionalProperties in JSON Schema.
I don't think however that inline is an appropriate name.
Not for Go 1.9. I hope to take a look at all the open json proposals this summer and try to see if a coherent picture emerges.
I have something where I have a some common fields (id, path, revision, etc) and I use an anonymous struct that is embedded in other structs. However, I can't implement a custom json marshal on the common struct as then all the other structs will only return the output from the marshalled common fields.
Just ran into this.
The use of anonymous structs for inlining is excellent up until your anonymous struct has a MarshalJSON method.
In https://github.com/golang/go/issues/42417 I proposed to have the following syntax:
type Message struct {
Name string `json:"name"`
Type string `json:"type"`
Skip string `json:"-"`
Extra map[string]interface{} `json:"+"`
}
So + would mark a field into which all extra/unexpected fields would go.
+ is an interesting idea, but the + character is already a valid name, so we cannot use that in a backwards compatible way.
The - character is also a valid name but the encoding/json package lets you use it by adding the json:"-," value. See https://pkg.go.dev/encoding/json#Marshal.
@smoyer64, that's doesn't fix the issue I mentioned:
-
json:-means this field is ignored -
json:-,means this field is named "-"
However, today:
-
json:+means this field is named "+"; and we can't redefine the meaning of this -
json:+,means this field is named "+"
encoding/xml supports this via the xml:",any" tag:
- If the XML element contains a sub-element that hasn't matched any of the above rules and the struct has a field with tag ",any", unmarshal maps the sub-element to that struct field.
(https://golang.org/pkg/encoding/xml/#Unmarshal)
Should the same mechanism be used for encoding/json?
+1.
Having an element that has a field 'type', and different schema depending on the value of that field is pretty much idiomatic way of building JSON objects / arrays. That's the way most APIs work, and it's surprising that golang, which is the language for servers, does not have strong support for such a common usecase.
+1
Comment 18 by [email protected]:
+1 I have something where I have some common fields (id, path, revision, etc) and I use an anonymous struct that is embedded in other structs. However, I can't implement a custom json marshal on the common struct as then all the other structs will only return the output from the marshalled common fields.
I've read through the comments a few times, and I'm confused.
The ticket is still open, is this still an active proposal? Are we waiting for a better suggestion on syntax / naming?
At this point, I'd be happy to have ANY solution - the work-arounds are killers!