[Proposal]: Dictionary expressions

[CyrusNajmabadi]

Dictionary Expressions

• [x] Proposed
• [ ] Prototype: Not Started
• [ ] Implementation: Not Started
• [ ] Specification: Not Started

Summary

Collection Expressions were added in C# 12. They enabled a lightweight syntax [e1, e2, e3, .. c1] to create many types of linearly sequenced collections, with similar construction semantics to the existing [p1, p2, .. p3] pattern matching construct.

The original plan for collection expressions was for them to support "dictionary" types and values as well. However, that was pulled out of C# 12 for timing reasons. For C# 13 we would to bring back that support, with an initial suggested syntax of [k1: v1, k2: v2, .. d1]. As a simple example:

private readonly Dictionary<string, int> _nameToAge = [
    "Cyrus": 21,
    "Dustin": 22,
    "Mads": 23,
];

The expectation here would be that this support would closely track the design of collection expressions, with just additions to that specification to support this k:v element syntax, and to support dictionary-like types as the targets of these expressions.

Motivation

• Collection-like values are hugely present in programming, algorithms, and especially in the C#/.NET ecosystem. Nearly all programs will utilize these values to store data and send or receive data from other components. Currently, almost all C# programs must use many different and unfortunately verbose approaches to create instances of such values.

• An examination of the ecosystem (public repos, nuget packages, and private codebases we have access to), indicate that dictionaries are used as a collection type in APIs around 15% of the time. While not nearly as ever-present as the sequenced collections (like arrays, spans, lists, and so on), this is still substantial, and warrants the equivalently pleasant construction provided by collection-expressions.

• Like with linear collections, there are numerous different types of dictionary-like types in the .net ecosystem. This includes, but is not limited to, simply constructed dictionaries like Dictionary<TKey, TValue> and ConcurrentDictionary<TKey, TValue>, but also interfaces like IDictionary<TKey, TValue> and IReadOnlyDictionary<TKey, TValue>, as well as immutable versions like `ImmutableDictionary<TKey, TValue>. Supporting all these common forms is a goal for these expressions.

Detailed design

Main specification here: https://github.com/dotnet/csharplang/blob/main/proposals/collection-expressions-next.md

The only grammar change to support these dictionary expressions is:  

collection_literal_element
  : expression_element
+ | dictionary_element
  | spread_element
  ;

+ dictionary_element
  : expression ':' expression
  ;

Spec clarifications

• dictionary_element instances will commonly be referred to as k1: v1, k_n: v_n, etc.

Conversions

The following implicit collection literal conversions exist from a collection literal expression:

• ...

• To a type that implements System.Collections.IDictionary where:

  • The type contains an applicable instance constructor that can be invoked with no arguments or invoked with a single argument for the 0-th parameter where the parameter has type System.Int32 and name capacity.
  • For each expression element Ei:
    • the type of Ei is dynamic and there is an applicable indexer setter that can be invoked with two dynamic arguments, or
    • the type of Ei is a type System.Collections.Generic.KeyValuePair<Ki, Vi> and there is an applicable indexer setter that can be invoked with two arguments of types Ki and Vi.
  • For each dictionary element Ki:Vi, there is an applicable indexer setter that can be invoked with two arguments of types Ki and Vi.
  • For each spread element Si:
    • the iteration type of Si is dynamic and there is an applicable indexer setter that can be invoked with two dynamic arguments, or
    • the iteration type is System.Collections.Generic.KeyValuePair<Ki, Vi> and there is an applicable indexer setter that can be invoked with two arguments of types Ki and Vi.

• To an interface type I<K, V> where System.Collections.Generic.Dictionary<TKey, TValue> implements I<TKey, TValue> and where:

  • For each expression element Ei, the type of Ei is dynamic, or the type of Ei is a type System.Collections.Generic.KeyValuePair<Ki, Vi> and there is an implicit conversion from Ki to K and from Vi to V.
  • For each dictionary element Ki:Vi there is an implicit conversion from Ki to K and from Vi to V.
  • For each spread element Si, the iteration type of Si is dynamic, or the iteration type is System.Collections.Generic.KeyValuePair<Ki, Vi> and there is an implicit conversion from Ki to K and from Vi to V.

Syntax ambiguities

• dictionary_element can be ambiguous with a conditional_expression. For example:

  var v = [ a ? [b] : c ];
  

  This could be interpreted as expression_element where the expression is a conditional_expression (e.g. [ (a ? [b] : c) ]). Or it could be interpreted as a dictionary_element "k: v" where a?[b] is k, and c is v.

Alternative designs

Several discussions on this topic have indicated a lot of interest and enthusiasm around exploring how close this feature is syntactically (not semantically) to JSON. Specifically, while we are choosing [k: v] for dictionary literals, JSON chooses { "key": value }. As "key" is already a legal C# expression, this means that [ "key": value ] would be nearly identical to JSON (except for the use of brackets instead of braces). While it would make it so we would have two syntaxes for collection versus dictionary expressions, we should examine this space to determine if the benefits we could get here would make up for the drawbacks.

Specifically, instead of reusing the collection_expression grammar, we would introduce:

primary_no_array_creation_expression
  ...
+ | dictionary_expression
  ;

+ dictionary_expression
  : '{' '}'
  | '{' dictionary_element ( ',' dictionary_element )* '}'
  ;

+ dictionary_element
  : expression_element
  | dictionary_element
  | spread_element
  ;

+ dictionary_element
  : expression ':' expression
  ;

You could then write code like:

private readonly Dictionary<string, int> _nameToAge = {
    "Cyrus": 21,
    "Dustin": 22,
    "Mads": 23,
};

Or:

Dictionary<int, string> combined = { .. ageToName1, .. ageToName2 };

Or

Dictionary<int, string> combined = { kvp1, key2: value2, .. ageToName1, .. ageToName2 };

The downside here is:

1. The potential ambiguity with existing constructs.
2. The potential conflicts with future constructs.
3. The challenge in creating a corresponding pattern form.

In order:

First, the above syntax already conflicts with int[] a = { 1, 2, 3, 4 } (an array initializer). However, we could trivially sidestep this by saying that if the initializer contained a spread_element or dictionary_element it was definitely a dictionary. If it did not (it only contains normal expressions, or is empty), then it will be interpreted depending on what the target type is.

Second, this could definitely impact future work wanted in the language. For example, "block expressions" has long been something we've considered. Where one could have an expression-valued "block" that could allow statements and other complex constructs to be used where an expression is needed. That said, such future work is both speculative, and itself could take on some new syntax. For example, we could mandate that an expression block have syntax like @{ ... }.

Third, the pattern form here presents probably the hardest challenges. We would very much like patterns and construction to have a syntactic symmetry (with patterns in the places of expressions). Such symmetry would motivate having a pattern syntax of { p1: p2 }. However, this is already completely the completely legal pattern syntax for a property pattern. In other words, one can already write if (d is { Length: 0 }). Indeed, it was all the ambiguities with { ... } patterns in the first place that motivated us to use [ ... ] for list patterns (and then collection-expressions). We will end up having to resolve these all again if we were to want this to work. It is potentially possible, but will likely be quite subtle with various pitfalls. Alternatively, we can come up with some new syntax for dictionary-patterns, but we would then break our symmetry goal.

Regardless, even with the potential challenges above, there is a great attractiveness on aligning syntactically with JSON. This would make copying JSON into C# (particularly with usages in API like Json.net and System.Text.Json) potentially no work at all. However, we may decide the drawbacks are too high, and that it is better to use the original syntactic form provided in this specification.

That said, even the original syntactic form is not without its own drawbacks. Specifically, if we use [ ... ] (just with a new element type, then we will likely have to address ambiguities here when we get to the "natural type" question.

For example:

var v1 = []; // What if the user wants a dictionary?
var v2 = [.. dict1, .. dict2]; // should combining dictionaries produce a dictionary?

and so on.

Open Questions

1. Should dictionary expressions have "Add" semantics or "Overwrite semantics". The working group is leaning toward the latter, specifically so you can write code like:

Dictionary<TKey, TValue> updated = [.. initialValues, k: v]; //or
Dictionary<TKey, TValue> merged = [.. d1, .. d2]; //or
Dictionary<TKey, TValue> overwritten = [k1: v1, k2: v2, .. augments]; // etc.

The space of allowing merging, with "last one wins" seems pretty reasonable to us. However, we want a read if people would prefer that throw, or if there should be different semantics for dictionaries without spreads for example.

2. Unlike linear-collections, augmenting a dictionary is a much more common practice. For example, providing a suitable IEqualityComparer. Should that be something supported somehow in this syntax. Or would/should that require falling back out to a normal instantiation?

Design Meetings

• https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-03-11.md#dictionary-expressions
• https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-05-15.md#dictionary-expressions
• https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-09-06.md#dictionary-expressions

[KennethHoff]

As noted by open question 2, I think losing support for IEqualityComparer - especially for string dictionaries - would be very unfortunate. For me personally equality comparisons is one of the primary reasons I use dictionaries over other data structures.

[HaloFour]

Would an extension method not suffice here?

var dictionary = [ "foo": "bar", "fizz": "buzz" ].WithComparer(StringComparer.OrdinalIgnoreCase);

// or

var dictionary  = StringComparer.OrdinalIgnoreCase.CreateDictionary([ "foo": "bar", "fizz": "buzz" ]);

[GabeSchaffer]

Would an extension method approach to specifying comparers be able work without creating a second dictionary?

[CyrusNajmabadi]

@GabeSchaffer Likely yes (though we would still need to get extensions working with collection exprs). Specifically, the keys/values in the expr woudl likely be passed to the extension method as a ReadOnlySpan of KeyValuePairs. These would normally just be on the stack, and would be given as a contiguous chunk of data for the dictionary to create its internal storage from once.

[GabeSchaffer]

Apologies for my lack of understanding, but it seems like a builder approach to augmentation could be made to work, like with String.Create.

That seems like it could make a syntax like this possible:

var dictionary = Dictionary.Create(StringComparer.OrdinalIgnoreCase, [ "foo": "bar", "fizz": "buzz" ]);

Another options that seems feasible is to allow constructor arguments to be specified in parens after the collection expression:

[ "foo": "bar", "fizz": "buzz" ](StringComparer.OrdinalIgnoreCase) // function call syntax for passing ctor args

An uglier possibility is to use a semicolon:

[ "foo": "bar", "fizz": "buzz"; StringComparer.OrdinalIgnoreCase ] // ; delimits comparer

I think if we want to have hope of being able to specify a comparer in a pattern, the latter two are better.

[Perksey]

Wow I had no idea the C# team were so young ;)

private readonly Dictionary<string, int> _nameToAge = [
    "Cyrus": 21,
    "Dustin": 22,
    "Mads": 23,
];

[hez2010]

How about reusing the existing dictionary syntax?

Given that we already have

var dict = new Dictionary<K, V> { [key] = value }

We can somehow use a similar syntax:

var dict = { [key]: value }

This is also matching how TypeScript defines an interface with indexed field:

interface Foo {
    [key: string]: number;
}

And we can even support dictionary patterns along with existing patterns as well:

var obj = new C();

if (obj is {
    [string key]: 42,
    ["foo"]: int value,
    SomeProp: float x
}) ... // matching an object which has a indexed value 42 while its key is string, and a key "foo" whose value is int

class C
{
    public float SomeProp { get; }
    public int this[string arg] => ...
}

As well as the case where the indexer has multiple parameters:

if (obj is {
    ["foo", "bar"]: 42
})

[TahirAhmadov]

Specifically, the keys/values in the expr woudl likely be passed to the extension method as a ReadOnlySpan of KeyValuePairs.

What about spreads, those will have to be iterated and placed on the stack before calling the extension method? Or is there a way to pass them in somehow?

[GabeSchaffer]

Specifically, the keys/values in the expr woudl likely be passed to the extension method as a ReadOnlySpan of KeyValuePairs.

What about spreads, those will have to be iterated and placed on the stack before calling the extension method? Or is there a way to pass them in somehow?

I don't see how that can be implemented in any way other than to copy every element into a contiguous array in order to create a ReadOnlySpan. Because if the elements aren't contiguous, it's not a span, right?

[mpawelski]

@hez2010

How about reusing the existing dictionary syntax?

Given that we already have

var dict = new Dictionary<K, V> { [key] = value }

We can somehow use a similar syntax:

var dict = { [key]: value }

I think dictionary-like objects that have only one indexer are so common that it's worth to consider special, more succinct, syntax that doesn't require unnecessary [ and ]. Such as the one proposed in this thread.

Actually, your proposed syntax is not far from what we can do now in C#:

Dictionary<string, int> dict = new() { 
    ["aaa"] = 1, 
    ["bbb"] = 1
};

And we can even support dictionary patterns along with existing patterns as well:

var obj = new C();

if (obj is {
    [string key]: 42,
    ["foo"]: int value,
    SomeProp: float x
}) ... // matching an object which has a indexed value 42 while its key is string, and a key "foo" whose value is int

class C
{
    public float SomeProp { get; }
    public int this[string arg] => ...
}

As well as the case where the indexer has multiple parameters:

if (obj is {
    ["foo", "bar"]: 42
})

Actually I was kinda surprised that C# doesn't have pattern matching for indexers. This code doesn't work:

Dictionary<string, int> dict = new() { 
    ["aaa"] = 1, 
    ["bbb"] = 2
};

if (dict is { ["aaa"] : 1 })
{
}

But this feels looks like another "pattern matching improvements" feature, not strictly related to dictionary expressions we discuss in this thread.

[scarletquasar]

I suggest using object notation, like TypeScript.

Like:

{
    Type = "Person",
    Name = "John"
}

[TahirAhmadov]

Can we make the "add/overwrite" be switchable? Say, we default to the safe "add" approach, and if overwrite is desired:

var d = [(k1: v1)!, (..d1)!];

Here, ! means "overwrite". Frankly, I'm not crazy about this syntax, but I couldn't come up with anything better at the moment (admittedly, I thought about it for a few minutes only). However, syntax aside, I'd still like to raise this as a suggestion.

PS. In addition to the above, if it should be "overwrite" for the whole list:

var d = ([k1: v1, ..d1])!;

[colejohnson66]

That syntax looks a bit weird. If an "or overwrite" operation was added, I'd prefer to augment the with syntax:

IDictionary<string, string> d = GetDictionary();
(string k, string v) = GetNewKVP();
IDictionary<string, string> d2 = d with { [k1]: v1 };

But maybe that would be confusing, as one would have to remember that spreading is adding and with is overwrite (or add if missing).

[TahirAhmadov]

@colejohnson66 with your approach, you would always have to create a temp dictionary variable to then "overwrite" things onto it to produce the final desired result, which effectively changes it from a "collection expression" to a "procedural code block".

[alrz]

You can do this today

Dictionary<string, int> _nameToAge = new(OptionalComparer) {
    ["Cyrus"] = 21,
    ["Dustin"] = 22,
    ["Mads"] = 23,
};

or just [new(key ,value)] given an extension Add(this Dictionary, KeyValuePair).

I'd rather see dictionary/indexer patterns to complete the matching side of the existing [e]=v initialization syntax. Meanwhile, I think stronger type inference could help a lot within the existing syntax for more concise maps.

var x = new Dictionary() { .. };

There's endless optimization possibilities for collection expressions, not so much with dictionaries, and spreads with maps are just confusing or so rare at best.

Having said that, I think immutable dictionaries could use some less awkward initialization API, if possible.

[Ai-N3rd]

I would like to add my take on the IEqualtiyComparer argument:

var dict = 
(new SomeEqualityComparer())
[
    // Whatever syntax we decide on here
    ["Cyrus"] = 21,
    ["Dustin"] = 22,
    ["Mads"] = 23,
]

This could also work for constructor parameters on other types, if that is found necessary.

[glen-84]

What about using the JSON-like syntax for declaration:

private readonly Dictionary<string, int> _nameToAge =
{
    "Cyrus": 21,
    "Dustin": 22,
    "Mads": 23
};

But for pattern matching, use the indexer syntax suggested by @hez2010:

// Match a KV pair in the dictionary.
if (_nameToAge is { ["Cyrus": 69] })

// Match a property of the dictionary.
if (_nameToAge is { Length: 0 })

[KennethHoff]

@glen-84 One of the goals of collection expressions were parity with pattern matching, so this (hopefully) won't happen.

[glen-84]

This is probably as close to parity as you'll get when going with this JSON-like syntax. I think it's intuitive – you're matching elements of the dictionary, not the dictionary itself.

Otherwise it's back to option 1. 🙂

[Timovzl]

At least when it comes to expressions for creating a dictionary, in order to avoid the confusion of having too many options (especially ones that are almost the same), I think it would help to stay close to this existing dictionary initializer syntax:

new Dictionary<string, int>()
{
   ["Cyrus"] = 21,
   ["Dustin"] = 22,
   ["Mads"] = 23,
};

For example:

[
   ["Cyrus"] = 21,
   ["Dustin"] = 22,
   ["Mads"] = 23,
]

This avoids the confusion of yet another style, and it separates key and value more clearly than the other existing collection initializer syntax:

new Dictionary<string, int>()
{
   // Meh for complex keys/values, especially when their expressions have commas and braces of their own
   { 111, new StudentName() { FirstName="Sachin", LastName="Karnik", ID=211 } },
   { 112, new StudentName() { FirstName="Dina", LastName="Salimzianova", ID=317 } },
   { 113, new StudentName() { FirstName="Andy", LastName="Ruth", ID=198 } },
};

[TahirAhmadov]

For example:

[
   ["Cyrus"] = 21,
   ["Dustin"] = 22,
   ["Mads"] = 23,
]

This adds a lot of annoying extra characters to type, and tries to address a non-existent goal - to maintain similarity to existing approaches. Collection expressions (including dictionary expressions) are meant to come up with a standardized new syntax for all (most) collections, not try to continue older approaches. With respect to the syntax above, you can already do:

new()
{
   ["Cyrus"] = 21,
   ["Dustin"] = 22,
   ["Mads"] = 23,
};

[colejohnson66]

Well, it's the same thing for lists and arrays. I can also currently do:

new() {1, 2, 3} // generates whatever collection target type is
new[] {1, 2, 3} // generates an array

But collection expressions were still added.

[Ai-N3rd]

Just clarifying, which of these would we allow in pattern matching:

[
    // Type validation and accessing by value
    [string name] = 21,
    // Discarding
    [var _] = 22,
    // Accessing unknown values
    ["Mads"] = var age
]

[TahirAhmadov]

But collection expressions were still added.

It saves typing even in the simple [1, 2, 3] scenario, but collection expressions also support spreads, which the old style syntaxes don't. Also, the language needs the { } for other constructs, and it's better to use [ ] for collections. Plus ability to target type interfaces, etc. However, for dictionaries specifically, the ["Cyrus"] = 21 style of element is a good amount more ceremony per element than "Cyrus": 21, and that difference doesn't exist for regular collections. That's the argument I was making - why add more typing to new syntax, just to maintain similarity with older syntax which is explicitly being (soft-)replaced with the new syntax?

[Ai-N3rd]

@TahirAhmadov

Collection expressions (including dictionary expressions) are meant to come up with a standardized new syntax for all (most) collections, not try to continue older approaches.

This syntax is well known. We can create a standard without reinventing the wheel. It's not a rule that we cant reuse old syntax.

[TahirAhmadov]

This syntax is well known. We can create a standard without reinventing the wheel.

Personally, I never use the ["Cyrus"] = 21 "overwrite" syntax in my existing code specifically to keep my code safer - I prefer the Add semantics. However, my personal preferences aside, the new "Cyrus": 21 syntax is not a difficult thing to learn in terms of introducing new syntax, and it is so intuitive, I'm frankly surprised by the amount of debate about it. Also, I need to repeat - the new collection and dictionary expressions are meant to replace existing initializations, so it doesn't really matter that the old syntax is well known - think of it as something to forget going forward, really.

It's not a rule that we cant reuse old syntax.

There is no rule that we can't reuse old syntax, but there is zero reason to do so, and the old syntax is more typing. So, if it's worse than the proposed new one, and we are under no obligation to reuse old syntax, why reuse it?

[TahirAhmadov]

Another thought - perhaps the ["Cyrus"] = 21 syntax can be used going forward, to explicitly switch to the "overwrite" mode? Basically, ["Cyrus": 21, ["Cyrus"] = 22] would not throw and instead have 22 for "Cyrus", as opposed to ["Cyrus": 21, "Cyrus": 22], which would throw. PS. However this is probably a bad approach, because it doesn't address the question of switching a spread to "overwrite" semantics, and doesn't allow switching the whole dictionary expression to "overwrite", either.

[Ai-N3rd]

@TahirAhmadov What makes it worse? If you have autocomplete, it's 1 extra character you have to type, 2 without. It's not the end of the world. It also conveys how it really works with the override semantics, which is far more useful than saving typing. And again, it doesn't have to replace the current to create a standard. We do not have to reinvent the wheel. ["Cyrus"] = 21 is just as intuitive.

[TahirAhmadov]

@Ai-N3rd hmm that's incorrect. "Cyrus": 21 is one symbol - :. ["Cyrus"] = 21 is 3 symbols - [, ], and =. And auto-complete doesn't help here, because these are symbols - it would be very difficult for an IDE to predict when your key expression ends, at best it could suggest = for you after ], but even then you'd have to press Tab to accept it, so not really an improvement. So either way you look at it, it's 3 keystrokes per element instead of 1, multiply by say, 10 KVPs, and you're talking about 20 extra keystrokes for no good reason.

[Ai-N3rd]

That being said, you make a good point about explicit overwrite sementics. As for spreads, perhaps we could do something like this:

[
    // Overwrite
    [..someDict],
    // Overwrite
    ["Cyrus"] = 21,
    // Add
    ..differentDict,
    // Add
   "Dustin": 22
]

This would be more confusing and less safe, but I can see the idea.