julia icon indicating copy to clipboard operation
julia copied to clipboard

Define `length(::Iterators.Rest{<:AbstractVector})`

Open mtfishman opened this issue 3 weeks ago • 2 comments

length(::Iterators.Rest{<:AbstractVector}) currently isn't defined, which affects these kinds of use cases:

julia> length(Iterators.rest([1, 2, 3, 4], 2))
ERROR: MethodError: no method matching length(::Base.Iterators.Rest{Vector{Int64}, Int64})
The function `length` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  length(::Base.EnvDict)
   @ Base env.jl:232
  length(::BitSet)
   @ Base bitset.jl:355
  length(::Base.MethodSpecializations)
   @ Base runtime_internals.jl:1877
  ...

Stacktrace:
 [1] top-level scope
   @ REPL[1]:1

julia> length(Iterators.peel([1, 2, 3, 4])[2])
ERROR: MethodError: no method matching length(::Base.Iterators.Rest{Vector{Int64}, Int64})
The function `length` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  length(::Base.EnvDict)
   @ Base env.jl:232
  length(::BitSet)
   @ Base bitset.jl:355
  length(::Base.MethodSpecializations)
   @ Base runtime_internals.jl:1877
  ...

Stacktrace:
 [1] top-level scope
   @ REPL[1]:1

julia> versioninfo()
Julia Version 1.14.0-DEV.1286
Commit fd419bd9233 (2025-11-23 16:30 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 10 × Apple M1 Max
  WORD_SIZE: 64
  LLVM: libLLVM-20.1.8 (ORCJIT, apple-m1)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 8 virtual cores)
Environment:
  JULIA_EDITOR = code

I understand that length(::Iterator.Rest) isn't defined for general iterators since the length may not be efficiently computable from the state, but it seems like for vectors it is reasonable to define it. I could imagine a definition like:

function Base.length(itr::Iterators.Rest{T}) where {T<:AbstractVector}
    len = length(itr.itr) - itr.st + 1
    return max(len, zero(len))
end

but maybe there is something I'm not considering.

mtfishman avatar Dec 08 '25 19:12 mtfishman

This cannot be defined for AbstractVector, since users could define a new AbstractVector with another iterator state. But can be defined for Memory, Vector, and views of those.

jakobnissen avatar Dec 08 '25 19:12 jakobnissen

This cannot be defined for AbstractVector, since users could define a new AbstractVector with another iterator state. But can be defined for Memory, Vector, and views of those.

I see, I wasn't sure if there was some assumed interface requirement for AbstractVector that the states correspond to the linear indices (maybe it is ok to assume that and then users can opt-out for strange AbstractVector iterator state implementations?).

mtfishman avatar Dec 08 '25 20:12 mtfishman

What is the limitation from using IteratorSize(::Type) = HasLength()?

jakobjpeters avatar Dec 10 '25 10:12 jakobjpeters

What is the limitation from using IteratorSize(::Type) = HasLength()?

That's what I was thinking at first, but I think the issue with that in general is that you still need to determine the index from the current state of the iterator (so you can subtract it from the parent iterator's length), which may not be computable in O(1).

mtfishman avatar Dec 10 '25 13:12 mtfishman