Newtonsoft.Json
Newtonsoft.Json copied to clipboard
JamesNK
C#7 Tuple Support
With C#7 coming out within a week, I was looking into the new features and the new Tuple feature with named elements seems like something Json.NET should support if possible.
As a root object I don't think this would be possible as Tuple element names are erased upon compilation and are only present at API boundaries, i.e. type fields and properties and method parameters and return parameters, through the TupleElementNamesAttribute. But when exposed as a Field or Property the element names could be retrieved through reflection with this attribute and be used instead of Item1, Item2, etc...
Given its uses and the lack of support as a root object I'm not sure the effort to get this to work is worth it.
I could be convinced either way. Technically it is a little annoying because the type is shared so the names will needed to get squished into JsonProperty somehow.
What happens when a named tuple is the value type in a dictionary? Where do the names go then?
Just upon inspection using LinqPad 5.2 Beta it appears that for generic types it applies the TupleElementNamesAttribute to the field. Given the below field
Dictionary<((bool keyC, int keyD) keyA, DateTime keyB), (string, string valueB)> Map
the following attribute gets applied to the field
[TupleElementNamesAttribute(new String[6] { "keyA", "keyB", "keyC", "keyD", null, "valueB" })]
It looks like null is used when there is no name applied.
- Edited to change the ordering of Tuples to better understand how the element names are recorded in the attribute
- Further updated to show how a tuple element with no name is recorded.
We're looking at what support for tuples looks like in ASP.NET Core MVC (https://github.com/aspnet/Mvc/issues/5885). It would be good to have a similar story for our JSON formatters.
The problem is that the root type doesn't have the attributes with the names on them. Passing in the type is no longer enough - the ParameterInfo would also need to be given to the formatter.
Does this include supporting tuples in Razor? cause I'm currently hitting a wall with that and its super annoying
I'm thinking that perhaps this should be implemented by using a new C# language feature. I'm thinking of something like this.
var value = (a: 4, b: false);
var json = JsonConvert.SerializeObject(value, elementnames(value)); // {"a":4,"b":false}
elementnames would be a new keyword that evaluates to a compile time array of the element names of the tuple or generic type with tuples as type arguments. Essentially it would be the equivalent of TupleElementNamesAttribute.TransformNames but wouldn't require ParameterInfo.
For Json.NET, we would just need to add overloads to JsonConvert.SerializeObject, JsonConvert.DeserializeObject, JsonSerializer.Serialize, and JsonSerializer.Deserialize to accept an array of strings for the tuple element names.
Do you think this would be worth proposing to the C# language design team?
That seems like a niche feature.
Is there any way to get tuple names from a local variable at the moment?
Not at all, the tuple element names are erased and there are only attributes at API boundaries.
hi firends, i can get propery names in this test, if this help you to update json component this is very good for us,
public class Test1
{
public void Test()
{
foreach (var item in this.GetType().GetMethods())
{
dynamic attribs = item.ReturnTypeCustomAttributes;
if (attribs.CustomAttributes != null && attribs.CustomAttributes.Count > 0)
{
foreach (var at in attribs.CustomAttributes)
{
if (at is System.Reflection.CustomAttributeData)
{
var ng = ((System.Reflection.CustomAttributeData)at).ConstructorArguments;
foreach (var ca in ng)
{
foreach (var val in (IEnumerable<System.Reflection.CustomAttributeTypedArgument>)ca.Value)
{
var PropertyNameName = val.Value;
Console.WriteLine(PropertyNameName);
}
}
}
}
dynamic data = attribs.CustomAttributes[0];
var data2 = data.ConstructorArguments;
}
}
}
public (int MyValue, string Name) GetMe2(string name)
{
return (5, "ali");
}
}
Hello, in the case I am right now, and in the end the purpose is make the json package smaller, why not simply (4,false), because a tuple is a tuple not a class, also is not an array
var value = (a: 4, b: false);
var json = JsonConvert.SerializeObject(value, elementnames(value)); // (4,false)
(4,false) is not valid json. If it were instead [4,false] that would be a valid json array but the order then becomes important as there are no property names to distinguish the values.
@TylerBrinkley Well tuples are a 'new' thing, why json cannot evolve in the same way? Is there any other interpretation of
{ "properties" : (4, false) }
could that break the validity of a json?
@juanpaexpedite Tuples aren't really a new thing; they're basically just syntactic sugar for a dynamic class (at least in the scope of what we're talking about here).
As such, a tuple absolutely can be expressed in JSON:
{
"tupleValue":
{
"a": 4,
"b": false
}
}
The problem here is not that there isn't any way to express a tuple in JSON; it's that the syntactic sugar of the tuple property names isn't available when going to serialize that tuple into JSON; thus you'd get something like this instead:
{
"tupleValue":
{
"Item1": 4,
"Item2": false
}
}
@juanpaexpedite just to be clear,
{ "properties" : (4, false) }
doesn't work because (a, b) is already valid javascript syntax - it is a single expression that joins two clauses with the comma operator which says "run all the clauses and return the last one. As such
(4, false)
is (mostly) equivalent to
(() => {
4;
return false;
})()
It's not a super common javascript syntax but it is used. I used it last just a few days ago in a reduce callback.
@upta Your reasoning is of course perfect, I know it has to be really useful in many cases, but why the opposite (a simpler version) is not valid, in C# they are not classes msdn explains in the remarks exactly what I mean:
A tuple is a data structure that has a specific number and sequence of elements. eg: var primes = Tuple.Create(2, 3, 5, 7, 11, 13, 17, 19);
It is not any other thing, at least in C#. The only thing I want is a flag to make the serialization of tuples without naming.
It is very disappointing that ValueTuples were added to the language without a story for serializing their field names from a value.
It would be nice to have complete support for Tuples. When I'm serializing (firstname: "John", lastname: "Doe") I'm expecting to get JSON { firstname: "John", lastname: "Doe" } and not { Item1: "John", Item2: "Doe" }. Actually in web application usually you are creating a lot of ViewModel classes which you are using as responses. But, you could use Tuples instead of a lot of ViewModel classes (at least you could reduce number of your ViewModel classes) and it's really cool. But, we need correct serializing of named Tuples to JSON because work with keys like Item1, Item2.. is not what you really want.
To be fair, anonymous classes work just fine in that context though. The advantage of tuples is that they can be passed to existing methods (even in 3rd party libraries) that only care that the number and types of the fields are right, which they can then access with Item1, Item2 etc.
I'm with Crispried.
Let me give a scenario where having the names would be useful. I want to return a standard "thing" from a web API method -- for example, a "thing" called userInfo, with "properties" like userInfo.ID, userInfo.FirstName, userInfo.LastName, userInfo.Authorizations[], etc.
Right now, I do something like:
return new OkObjectResult( new { id = userID, firstName = dbUser.FirstName, lastName = dbUser.LastName, authorizations = adAuthorizationList, etc. } );
Why a standard "thing"? Because if I have multiple return points, I don't want to depend on my reliably going to each of those multiple return points whenever I want to change something. For example, let's say I decide to change the property name from id to userID. However, in altering the code, I make the change in the four common return places, but overlook the rarely hit fifth place.
Now let's not get into a discussion revolving around single points of return. If I look up the userID in the User table and a record isn't found, then I would like to return userInfo with just the userID --there is no sense in proceeding to look up authorizations at that point, and whatever else in order to get to a single return point at the end. Yes, you could avoid this with If statements 10 levels deep, or the dreaded goto, but why?
Some will say, construct a suitable LINQ / EF query. That is valid up to the point that everything is LINQ / EF accessible, which may not be the case if say .Authorizations reside in some other store like Active Directory.
Some will say to use Unit tests, and that's a valid point, but I like to "design in" correctness, and not just rely on catching problems after the fact.
The kicker is that this standard object is used only by that method. Yes, I could create a class or a struct, but that's a bit of a maintenance overhead, and I'd prefer to void that.
It'd be nice to have a mutable anonymous object like in VB.net, but C# doesn't allow it (go figure).
A tuple with names -- as in (int ID, string FirstName, string LastName) -- could be an alternative. The problem is, that on the browser end, it gets "Item1", "Item2", etc.
Now, you could say that this is an "off-label" application of tuples, and one gets what one gets. I can understand that. On the other hand, I'm willing to bet that most tuples are defined with names because those names have meaning, and that same meaning has real value when the JSON is received at the other end (i.e., at the browser or whatever).
So, to me, passing along the names is highly desirable. If you want to include a parameter to switch it on/off, that'd be fine.
The issue isn't that its not desirable, the issue is the type information is lost so Json.NET doesn't know what the names are.
This point is good to explain cause its something I myself didn't understand until I started looking at the compiled IL.
When you do
(int foo, string bar) GetValues() {
}
You do have access to the property names. But its not by analyzing the return type of GetValues() - that is simply a ValueType with no property name annotations, instead, the property names are stored in a compile-time generated custom attribute in the MethodInfo of the GetValues() function.
What this means is that you can only figure out what the property names are so long as you have access to the GetValues() function. If you do
var tup = GetValues();
PrintPropNamesOf(tup);
There is simply no way to do that as that information isn't actually stored in the ValueTuple.
Similarly when you do
var tup = GetValues();
JsonConvert.SerializeObject(tup);
The information on what the property names are is simply not available to the SerializeObject method.
This is frankly an incredibly annoying decision by the .net team
I've managed to achieve ValueTuple de/serialization, with the caveat that you have to explicitly declare the ValueTuples in the signature of Value Object type definitions. I'm using my ValueOf library, but I'm sure the code could be adapted. It's a proof of concept, please break it!
Linqpad demo: http://share.linqpad.net/n9p7h2.linq
public class UserInfo : ValueOf<(string name, (int age, string eyes, DOB dob) features), UserInfo> { }
public class DOB : ValueOf<(int day, int month, int year), DOB> { }
void Main()
{
var ur = UserInfo.From(("harry", (36, "blue", DOB.From((22, 12, 81)))));
var ccr = new ValueTupleContractResolver();
var json = JsonConvert.SerializeObject(ur, new JsonSerializerSettings { ContractResolver = ccr, Converters = { new ValueOfConverter()} });
json.Dump(); //outputs {"name":"harry","features":{"age":36,"eyes":"blue","dob":{"day":22,"month":12,"year":81}}}
var deserialized = JsonConvert.DeserializeObject<UserInfo>(json, new JsonSerializerSettings { ContractResolver = ccr, Converters = { new ValueOfConverter()} });
deserialized.Dump(); //outputs a hydrated UserInfo
}
public class ValueTupleContractResolver : DefaultContractResolver
{
Stack<Queue<string>> names = new Stack<Queue<String>>();
protected override JsonContract CreateContract(Type objectType)
{
var jc = base.CreateContract(objectType);
var tena = objectType.GetCustomAttributes().OfType<TupleElementNamesAttribute>().SingleOrDefault();
if (tena != null){
names.Push(new Queue<string>(tena.TransformNames));
}
return jc;
}
protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
{
var property = base.CreateProperty(member, memberSerialization);
if (member.Name == "Value" && member.DeclaringType.IsConstructedGenericType && member.DeclaringType.GetGenericTypeDefinition() == typeof(ValueOf<,>))
{
property.Writable = true;
}
if (names.Count > 0){
var q = names.Peek();
var name = q.Dequeue();
if (name != null)
{
property.PropertyName = name;
}
if (q.Count == 0)
names.Pop();
}
return property;
}
}
//This isn't strictly necessary, it's used to remove the Value property when serializing the ValueOf
class ValueOfConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
if (objectType.BaseType.IsGenericType)
if (objectType.BaseType.GetGenericTypeDefinition() == typeof(ValueOf<,>))
return true;
return false;
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var jt = Newtonsoft.Json.Linq.JToken.ReadFrom(reader);
var valueType = objectType.BaseType.GetGenericArguments()[0];
var value = jt.ToObject(valueType, serializer);
var from = objectType.GetMethod("From", BindingFlags.Static | BindingFlags.Public | BindingFlags.FlattenHierarchy);
return from.Invoke(null, new[] { value});
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var innerValue = ((dynamic) value).Value;
serializer.Serialize(writer, innerValue);
}
}
@wizofaus reason the anonymous classes does not work is that they are not type safe. Tuples would be a type safe way to return a record. This is important if you generate typesafe SDK or documentation with Swagger.
Would anyone be up for opening a issue on the Microsoft's C# Language Design repository that we could vote/"thumbs up" on?
EDIT: I opened it myself: https://github.com/dotnet/csharplang/issues/1906
Any news on this topic? Can you read that ominous TupleElementNamesAttribute?
It doesn't work in a lot of scenarios, e.g. the tuple is assigned to an object property, or passing the tuple directly to the serializer/converter.
How come it wouldn't be able to work when it's assigned to an object property? (Or nested in a generic type on an object property, or generally any situation other than passing the tuple in directly.) Can't it read the attribute on the prop? It would be great to at least have this subset of usage covered.
public class ValueContainer
{
public object Value { get; set; }
}
ValueContainer c = new ValueContainer();
c.Value = GetTuple();
string json = JsonConvert.SerializeObject(c);
or
string json = JsonConvert.SerializeObject(GetTuple());
Given the way names in tuples work, names will never be serialized in either of these cases. Both scenarios are common.
I don't want to add something that only works 75%~ of the time.
I definitely agree with not having tricks around tuples - they're a recipe for confusion, and once you've used a construct with surprising serialization behavior in stored form, you're stuck with that format forever.
100% red herring aside: @JamesNK's can I get you to rescan https://github.com/JamesNK/Newtonsoft.Json/issues/1827 which you've clearly muted, please? ;) Even if the answer is "no, too big", I'd love an answer.
Is there any way to retrive method/action meta-info while serializing? Just like we able to do that with swashbuckle, for example
I think @a-a-k 's comment needs more thought / consideration here.. the main use case for this is controllers and web apis to get something a bit lighter than standard C# Lasagna layers eg F# Micro services or dynamic js. Though IActionResult makes this harder . Note in this use case it is NOT order sensitive.
Structs are more of an internal thing and serialization is a slower ( compared to memory) external thing. Should the guide line to just be to use an anonymous type instead. Surely there must be some easy helper / extension method to go from tuple to anonymous type .. select new { prod.Color, prod.Price }; and then serialize.
just an aside, i used this code https://github.com/SimonCropp/ObjectApproval/blob/master/src/ObjectApproval/ObjectApprover_Tuple.cs to support this scenario https://github.com/SimonCropp/ObjectApproval#named-tuples
Here's a fiddle using my ValueOf based approach. Works for the API scenario @bklooste is talking about https://dotnetfiddle.net/Ec7IKN
Hi. I've just been working with the styles. I've come to the conclusion that the tuple can be wrapped with the new operator, which should serialize the answer correctly. Example:
private static readonly IDictionary<string, (string dict, string file)> styles = new Dictionary<string, (string dict, string file)> { { "default", ("default", "styles.2GPJa.css") }, { "http://localhost:9001", ("other", "styles.3Z37u.css") } };
[HttpGet("styles")] public async Task<IActionResult> Styles(string url) { var (dict, file) = !string.IsNullOrWhiteSpace(url) && styles.ContainsKey(url) ? styles[url] : styles["default"]; return await Task.FromResult(Ok(JsonConvert.SerializeObject(new { dict, file }))); }
Result (without parameter url): {"dict":"default","file":"styles.2GPJa.css"}
Considering that the name information is lost when a value tuple is transferred back and forth, would it be so terrible to serialize to / deserialize from simple arrays? The order would then be important, but I think that's the case in the tuple implementations in most languages.
Tuples aren't intended to be a nominal data type used in such a manner. They are meant to be a positional grouping of loosely related data elements, like a parameter list. The names are there only to help the developer in referencing the elements, but they're largely ephemeral.
What about serializing tuples to what they are, namely tuples or a list of heterogenous values? As tuples are also a thing in TypeScript for example, why not serializing to them? In both worlds the names of tuple elements are nothing else than syntactic sugar, a "hint to the programmers" and can safely be replaced or omitted, they are not part of any output:
(int a, int b) = (1, 2);
(int foo, int bar) = (a, b);
(int, int) tuple = (a, b);
// a == foo, a == tuple.Item1
// b == bar, bar == tuple.Item2
So if anyone want to keep the names and work with JSON objects, why not work with objects (classes) in C# as well?
This is what I use for tuples and it works well. The deserialization is a bit tricky due to the generic character of the tuple type and the fact that a tuple can hold 7 values max, if more the 8th is a tuple again, recursively. (Although I think this should be rarely used, because not intuitive. Again, use classes here.)
public sealed class TupleJsonConverter : JsonConverter
{
/// <inheritdoc />
public override bool CanConvert(Type objectType) => typeof(ITuple).IsAssignableFrom(objectType);
/// <inheritdoc />
public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer)
{
if (value is ITuple tpl)
{
writer.WriteStartArray();
for (int i = 0; i < tpl.Length; i++)
{
serializer.Serialize(writer, tpl[i]);
}
writer.WriteEndArray();
}
else
{
writer.WriteNull();
}
}
/// <inheritdoc />
public override object? ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.StartArray)
{
var arr = JArray.Load(reader);
// Tuples have 7 elements max., 8th must be another tuple
var genericsStack = new Stack<Type[]>();
var generics = objectType.GetGenericArguments();
genericsStack.Push(generics);
while (generics.Length > 7 && typeof(ITuple).IsAssignableFrom(generics[7]))
{
generics = generics[7].GetGenericArguments();
genericsStack.Push(generics);
}
// Check generics length against tuple length
if ((genericsStack.Count - 1) * 7 + genericsStack.Peek().Length != arr.Count)
{
// As you can omit tail elements in TypeScript tuples you might do some advanced check here
// (if fewer elements than generics, are the types of the omitted elements nullable or things like that...).
throw new JsonSerializationException("Cannot deserialize Tuple, because the number of elements do not match the specified Tuple type.");
}
var argIndex = arr.Count;
object? value = null;
// deserialize tuples from inside do outside
foreach (var chunk in genericsStack)
{
var tupleType = GetTupleTypeDefinition(objectType, chunk.Length);
var args = new object?[chunk.Length];
if (chunk.Length > 0)
{
// make concrete tuple type
tupleType = tupleType.MakeGenericType(chunk);
int i = chunk.Length - 1;
// append previous tuple as 8th, inner tuple
if (i == 7)
{
args[7] = value;
i--;
}
// deserialize elements
for (; i >= 0; i--)
{
args[i] = arr[--argIndex].ToObject(chunk[i], serializer);
}
}
// create tuple instance
value = Activator.CreateInstance(tupleType, args);
}
return value;
}
throw new JsonSerializationException($"Cannot deserialize token {reader.TokenType} to Tuple.");
}
private static Type GetTupleTypeDefinition(Type objectType, int elementCount)
{
// ValueTuple<>
if (objectType.IsValueType)
{
return elementCount switch
{
8 => typeof(ValueTuple<,,,,,,,>),
7 => typeof(ValueTuple<,,,,,,>),
6 => typeof(ValueTuple<,,,,,>),
5 => typeof(ValueTuple<,,,,>),
4 => typeof(ValueTuple<,,,>),
3 => typeof(ValueTuple<,,>),
2 => typeof(ValueTuple<,>),
1 => typeof(ValueTuple<>),
0 => typeof(ValueTuple),
_ => throw new IndexOutOfRangeException(),
};
}
// Tuple<>
return elementCount switch
{
8 => typeof(Tuple<,,,,,,,>),
7 => typeof(Tuple<,,,,,,>),
6 => typeof(Tuple<,,,,,>),
5 => typeof(Tuple<,,,,>),
4 => typeof(Tuple<,,,>),
3 => typeof(Tuple<,,>),
2 => typeof(Tuple<,>),
1 => typeof(Tuple<>),
_ => throw new IndexOutOfRangeException(),
};
}
}
