vblang
vblang copied to clipboard
dotnet
[Proposal] For Each with Index
I propose that we extend the definition of LoopControlVariable to allow the loop to also include the index of each item.
Syntax
LoopControlVariable
: Identifier ( IdentifierModifiers 'As' TypeName )?
| Expression
;
Syntax WithIndexSyntax
WithKeyword As Keyword
Index As Identifier
End Syntax
Syntax LoopControlWithIndexVariable Inherits LoopControlVariable
WithIndex As WithIndexSyntax
End Syntax
Example
For Each x As t With index In xs
Console.WriteLine( $"({index}) = {x}")
Next
Lowering
The example will be lowered into the following code.
' compiler-generated {
Dim index = 0
' } end of compiler-generated
For Each x As t In xs
Console.WriteLine($"({index}) = {x}")
' Compiler-generated {
index += 1
' } end of compiler-generated
Next
Caveats
The WithIndex is to considered as declaring an implied Dim index As Integer definition, just before the for each statement. We makes it subject the restrictions and errors around already declared variables.
I was going to suggest a tuple-returning extension method on IEnumerable(Of T) as a workaround, then I remembered that we never got deconstruction (#278, #346) when they added tuples.
If find this to be so uncompelling, and potentially confusing for the purposes of understanding what meaning Index is meant to convey. If anything, it only tracks the apparent sort order of elements as they get yielded by an iterator - in which case, so what? On the other hand, some iterators are actually tied to objects that have true indexers, and that is where the potential confusion lies.
If you need the index of an object's indexer, what's so onerous about this?:
For index As Integer = 0 To xs.Count - 1
Dim x As T = xs(index)
WriteLine($"[{index}] {x}")
Next
If all you care about is an element's apparent placement from the iterator, what's so onerous about this (which is all this proposal does, I guess)?:
Dim index As Integer = -1
For Each x As T In xs : index += 1
WriteLine($"[{index}] {x}")
Next
It will be easier for both Implementer and user to use the For.Index expression to get the index:
For Each x In xs
Console.WriteLine( $"({For.Index}) = {x}")
Next
I'm not sure whether a keyword (FOR) can have properties, though.
Won't be new; Me is a keyword but we access the object members through it. In fact For will not have any properties. the compiler will replace For.Index with the auto generated variable it uses to hold the current index. The exact thing @AdamSpeight2008 described:
' compiler-generated {
Dim index = 0
' } end of compiler-generated
For Each x In xs
Console.WriteLine($"({index}) = {x}")
' Compiler-generated {
index += 1
' } end of compiler-generated
Next
I have to agree with @rskar-git; we already have a construct for this. If you desire an index, then iterate over the values using an index. For Each is what was added to make it simpler (to my knowledge) BECAUSE many times we don't need to know the specific index and/or care about the count. Adding it back in would just create a different version of what we already have for the sake of having an alternate method of doing what we can already do... or is there something that I'm missing? (Add to all of this... the "index" that would be in the For might not be the underlying index.. or would it... and would it be in sequence... so many issues I see with regard to this.)
@DualBrain It has nothing to do with the original index why would you assume that it was?
If that source enumerable was some form of where, group or a sort. I'd rarely expect it to exactly match the original indices.
In the form ForEach item With iteration_index In souce, kind of give us access to the Linq method Select(Of T, R)( item As T, index As Int32 ) As R. Just without the transformation functionality.
Example
Dim top40Chart = From sale In album_sales
Where sale.Date.IsThisWeek( week)
Group By sale.Title InTo albumsThisWeek
Order By albumsThisWeek.Group.Count Descending
Take 40
For Each album With position In top40Chart
Console.WriteLine($"{position:2} {album.Title} {album.Artist}")
Next
True we could use an explicitly defined variable to store the "iteration index" in, but personally why should I do that every time I require this form. It get in the way of writing the core intent of the code. I want the language to work with me, not needlessly get in the way just because of grammar requires it so.
Expanding on my comment earlier, with what we have right now, we could use a function like this:
Iterator Function Indexed(Of T)(source As IEnumerable(Of T)) As IEnumerable(Of (index As Integer, item As T))
Dim index = 0
For Each item In source
Yield (index, item)
index += 1
Next item
End Function
We then could write the For Each like this:
Dim msg = "Hello, World!"
Dim chars = msg.ToCharArray()
For Each item In Indexed(chars)
Console.WriteLine(item.index & " " & item.item)
Next item
If we got tuple deconstruction, it could be more like this:
Dim msg = "Hello, World!"
Dim chars = msg.ToCharArray()
For Each (index, ch) In Indexed(chars)
Console.WriteLine(index & " " & ch)
Next
(With the growing reluctance to add new syntax to VB in recent years, I have found myself turning to thinking about library-based solutions more and more, and, if we get new language features, knowing there might not be much, I would rather they enable libraries to fix multiple scenarios rather than the language itself fix a few small ones. In this case, I would rather have tuple deconstruction, which would have broader applications, and optimizations to LINQ and/or iterator functions, which could help a lot of things, rather than one seldom-used tweak to For Each. The scenario and related frustration is real; I have shaken my head over having to put an index += 1 statement inside a For Each loop a few times, but I don't think it's enough to warrant it's own syntax.)
It has nothing to do with the original index why would you assume that it was?
Because the proposal illustrated its use case like so: For Each x As t With index In xs. Perhaps if it was instead, at the start, For Each album With position In top40Chart, then such confusion might have been averted? Frankly, this syntactic sugar for some sort of "position" variable (presumably always an integer?) is fraught with needless confusion. If this thing ever becomes a reality, I expect a lot of people to misconstrue this: They will nonetheless think this With variable implies it is guaranteed to be usable on any object that implements an indexer or IList. I.e. they will think something like For Each x As t With index In xs might mean that t.Item(index) is going to be the same instance as x.
I want the language to work with me, not needlessly get in the way just because of grammar requires it so.
But what exactly is the hindrance here? The language is already quite expressive. For instance:
With top40Chart : Dim position = 0
For Each album In .AsEnumerable()
Console.WriteLine($"{position:2} {album.Title} {album.Artist}")
position += 1
Next
End With
That ensures position has scope only for the loop. Or maybe this, using Enumerable.Zip(Of TFirst, TSecond):
Function ItemCounter() As IEnumerable(Of Integer)
Return Enumerable.Range(0, Integer.MaxValue)
End Function
For Each item In top40Chart.Zip(ItemCounter())
Dim position = item.Second, album = item.First
Console.WriteLine($"{position:2} {album.Title} {album.Artist}")
Next
(You'll note, by design, Zip will do only as many iterations as the lowest count of the collections.)
I think a better idea may be to have a version of For Each that works about the same as Enumerable.Zip(Of TFirst, TSecond) Consider something like this:
Dim xsa = {"abc", "def", "ghi"}
Dim xsq = From xyz In xsa
Where xyz > "c"
For Each x In xsq.Zip(ItemCounter())
WriteLine($"{x.Second} {x.First}")
Next
What if we could have a version of For Each like this:
For Each {x, position} In {xsq, ItemCounter()}
WriteLine($"{position} {x}")
Next
In today's VB, that would be equivalent to this:
With (xsq.GetEnumerator(), ItemCounter().GetEnumerator())
Do While .Item1.MoveNext AndAlso .Item2.MoveNext
Dim x = .Item1.Current, position = .Item2.Current
WriteLine($"{position} {x}")
Loop
End With
@rskar-git
Indeed. If we could expand For Each, I would want to support a lot more scenarios, like
For Each {item1, item2} In {list1, list2}
If item1 < item2 Then
' do stuff
End If
Next
or
For Each {item1, item2} In {list1, list2}
If item2.IsSpecial() Then
item1.Prop = item2.Prop
End If
Next
or even
For Each {ByRef str, item} In {strList, objList}
If item2.IsSpecial() Then
str = item2.ToString()
End If
Next
If the language supported those scenarios, making one of those collections return ~~an index~~ a counter would be simple. (And now I'm wondering if we ought to open a separate issue for these scenarios.)
This works now without any language changes:
<TestMethod>
Private Sub EnumerableWithIndex_Test()
For Each e In {"A", "B", "C"}.WithIndex
With e
If .IsFirst Then
Console.WriteLine($"First one: { .Index}, { .Value}")
Assert.Ok(.Index = 0)
Assert.Ok(.Value = "A")
ElseIf Not .IsLast Then
Console.WriteLine($"{ .Index}, { .Value}")
Assert.Ok(.Index = 1)
Assert.Ok(.Value = "B")
Else
Console.WriteLine($"Last one: { .Index}, { .Value}")
Assert.Ok(.Index = 2)
Assert.Ok(.Value = "C")
Assert.Ok(.IsLast)
End If
End With
Next e
End Sub
<Extension>
Public Iterator Function WithIndex(Of T)(
source As IEnumerable(Of T),
Optional minIndex As Integer = 0 ' optionally skip elements at begin
) As IEnumerable(Of WithIndex_(Of T))
Using enumerator = source.GetEnumerator
Dim isFirst = True
Dim hasNext = enumerator.MoveNext
Dim i = 0
While hasNext
Dim current = enumerator.Current
hasNext = enumerator.MoveNext
If i >= minIndex Then
Yield New WithIndex_(Of T) With {.Index = i, .Value = current, .IsFirst = isFirst, .IsLast = Not hasNext, .MinIndex = minIndex}
isFirst = False
End If
Threading.Interlocked.Increment(i) ' index += 1, may be called with .AsParallel
End While
End Using
End Function
Public Structure WithIndex_(Of T)
Public Property Index As Integer ' first element has index = 0
Public Property Value As T
Public Property IsFirst As Boolean
Public Property IsLast As Boolean
Public Property MinIndex As Integer
End Structure
They will nonetheless think this
Withvariable implies it is guaranteed to be usable on any object that implements an indexer or IList.
I say again it has nothing to do with indexer or IList. (The only person confusing it is you.)
If there is no variable in scope the example's case index the scope of that auto-generated variable is tied to the for each loop.
I say again it has nothing to do with indexer or IList. (The only person confusing it is you.)
I say again it isn't only me who mentioned this confusion.
I point out the exact verbiage of this proposal: "I propose that we extend the definition of LoopControlVariable to allow the loop to also include the index of each item."
I reiterate how you framed the implementation and use-case of this proposal, using the word "index" in many places: Syntax WithIndexSyntax ... Index As Identifier; For Each x As t With index In xs.
I cite a definition of the word "index" as it is generally understood in computer science, from https://www.computingstudents.com/dictionary/index.php?word=Index: A number used to select an element of a list, vector, array or other sequence.
So, I hear you, For Each ... With is about a loop counter, which presumably is an integer which counts up from zero. Fine. Where we further disagree is not only on the point of confusion (index versus counter), but also when you expressed: "I want the language to work with me, not needlessly get in the way..."
I pointed out what was already possible in VB to accomplish the same, and expressed my opinion that I did not find anything about such approaches that got needlessly in the way. So we have different opinions here, and that's OK.
I then suggested that a more useful feature to give For Each would be a "Zip" feature, with which one could implement a "LoopControlVariable" concept, and whole lot more.
@gilfusion
If the language supported those scenarios, making one of those collections return an index would be simple.
However, by definition, a "Collection" does not imply an index. You'll note that the two principle members of Enumerator are Current and MoveNext; there is no Index or Key to be found there. Microsoft very intentionally designed For Each (and foreach in C#) to operate as if on a stream of elements passing through in only a forward direction (and sometimes that is really the case).
On the other hand, if the For Each ... With proposal stipulated that this would be only for IList objects - well then maybe. That seems to be the crowd pleaser here. But to do that right means a whole new set of "lowered code" that does not use Enumerator; it will probably end up looking like a regular For loop on an index variable that loads via IList (or via inferencing maybe instead the indexer of the object's true underlying type).
@rskar-git
With everyone speaking about confusion, I confused index with counter in the post you quoted, and of course, Enumerable.Range already gives us a collection (or an enumerable to be more accurate) that counts for us.
To me (and I suspect we might agree but are talking past each other on it), it looks like the For Each...With scenario could be handled as a case of the enhanced For Each you described that iterates on two or more collections (IEnumerables) at once, where one of them is Enumerable.Range or some "better" alternative.
Zip would better suited being a LINQ query operator (see #535), but that would be overkill for the simple case of including the sequence-index
@Paul M Cohen
Error BC30311 Value of type 'IndexStruct(Of MemberDeclarationSyntax)' cannot be converted to 'MemberDeclarationSyntax'.
' node.Members is defined as SyntaxList(Of CSS.MemberDeclarationSyntax) ' SyntaxList is a list of Microsoft.CodeAnalysis.SyntaxNode. ' <DefaultMember("Item")> <IsReadOnly> <NullableAttribute(0)> <NullableContextAttribute(1)> ' Public Structure SyntaxList(Of TNode As SyntaxNode) ' Implements IReadOnlyList(Of TNode), IEnumerable(Of TNode), IEnumerable, IReadOnlyCollection(Of TNode), IEquatable(Of SyntaxList(Of TNode))
For Each m As CSS.MemberDeclarationSyntax In node.Members.WithIndex Next
Use For Each m In node.Members.WithIndex(Of CSS.MemberDeclarationSyntax) instead of For Each m As CSS.MemberDeclarationSyntax In node.Members.WithIndex
I use the following extension method:
<Extension> Public Function ForEach(Of T)(src As IEnumerable(Of T), action As Action(Of T, Integer))
Dim current = 0
For Each item In src
action(item, current)
current += 1
Next
Return src
End Function
like this:
Dim seq = {"zero", "one", "two", "three", "four"}
seq.ForEach(Sub(item, index) Console.WriteLine($"{item} - {index}"))
@WalterLederer I started using this and can replace and simplify many of my For loops. Is there any way to skip forward (similar to i += 1) in a for loop. In some cases I peek at the next item and process 2 items together.
@paul1956
Is there any way to skip forward (similar to i += 1) in a for loop. In some cases I peek at the next item and process 2 items together.
ok, here it is:
<TestMethod>
Public Sub EnumerableWithIndex_Test()
For Each e In {"A", "B", "C", "D"}.WithIndex
With e
Dim out = Sub(s As String) Console.WriteLine($"{ .Index}, { .Value} ({s})")
If .IsFirst Then
out("first") : Assert.Ok(.Index = 0 And .Value = "A")
.MoveNext()
out("second") : Assert.Ok(.Index = 1 And .Value = "B")
ElseIf Not .IsLast Then
out("others") : Assert.Ok(.Index = 2 And .Value = "C")
Else
out("last") : Assert.Ok(.Index = 3 And .Value = "D")
End If
End With
Next e
End Sub
<Extension>
Public Iterator Function WithIndex(Of T)(
source As IEnumerable(Of T)
) As IEnumerable(Of Index_(Of T))
Using enumerator = source.GetEnumerator
Dim hasNext = enumerator.MoveNext
Dim index = -1
While hasNext
Dim wi = New Index_(Of T)(index, enumerator)
wi.MoveNext()
Yield wi
hasNext = Not wi.IsLast
index = wi.Index ' if .MoveNext was used
End While
End Using
End Function
Public Class Index_(Of T)
Public ReadOnly Property Value As T
Public ReadOnly Property Index As Integer ' first element has index = 0
Public ReadOnly Property IsFirst As Boolean
Get
Return Index = 0
End Get
End Property
Public ReadOnly Property IsLast As Boolean
Private ReadOnly Property Enumerator As IEnumerator(Of T)
Public Sub New(Index As Integer, Enumerator As IEnumerator(Of T))
_Index = Index
_Enumerator = Enumerator
End Sub
Public Sub MoveNext()
_Value = Enumerator.Current
_IsLast = Not Enumerator.MoveNext()
Threading.Interlocked.Increment(_Index) ' may be called with .AsParallel
End Sub
End Class
For Each with Index, skip, IsFirst and IsLast
Imports System.Runtime.CompilerServices
Public Module ForEachExtensions
<Extension>
Public Iterator Function WithIndex(Of T)(
source As IEnumerable(Of T)
) As IEnumerable(Of IndexClass(Of T))
If source Is Nothing Then
Throw New ArgumentNullException(NameOf(source))
End If
Using enumerator As IEnumerator(Of T) = source.GetEnumerator
Dim hasNext As Boolean = enumerator.MoveNext
Dim index As Integer = -1
While hasNext
Dim wi As New IndexClass(Of T) With {.Index = index, .Enumerator = enumerator}
wi.MoveNext()
Yield wi
hasNext = Not wi.IsLast
index = wi.Index ' if .MoveNext was used
End While
End Using
End Function
End Module
and
Public Class IndexClass(Of T)
Public Property Value As T
Public Property Index As Integer ' first element has index = 0
Public ReadOnly Property IsFirst As Boolean
Get
Return Index = 0
End Get
End Property
Public Property IsLast As Boolean
Public Property Enumerator As IEnumerator(Of T)
Public Sub MoveNext()
Value = Enumerator.Current
IsLast = Not Enumerator.MoveNext()
' may be called with .AsParallel
Threading.Interlocked.Increment(Index)
End Sub
End Class