`std/iterutils`: iterator utilities

[AmjadHD]

Abstract

Introduce an iterutils module to the standard library to provide a set of composable, lazy iterator utilities inspired by the slicerator library.

Motivation

The iterable[T] type class, introduced in https://github.com/nim-lang/Nim/pull/17196, enables passing called (inline) iterators to templates and macros. This can be leveraged to create iterator transformers that are composable to functional-style pipelines that are inlined to hand-written-like loop code. These pipelines can be consumed immediately or wrapped in a closure for later evaluation.

Bringing these capabilities to the standard library:

• Promotes clean, expressive and efficient iterator-heavy code
• Makes Nim more attractive to users from other languages.
• Avoids reinventing iterator pipelines in each project.

Description

The new module std/iterutils will provide:

• map, filter, zip, etc.: Lazy and composable iterator transformers
• asClosure: Convert an inline iterator to a reusable closure
• collect, fold, any, etc.: Functions that consume iterators.

Usage

These pipes can be iterated on with a for loop:

import std/[iterutils, strutils]

for word in str.split().filter(proc(x: string): bool = x.len > 0).mapIt(it & "!"):
  echo word

Or passed to a consumer function

import std/[iterutils, options, sets, strutils, tables]

assert countup(-25, 25).mapIt(it * 10 div 7).findIt(it > 35) == none(int)

let stocks: Table[string, tuple[symbol: string, price: float]] = {
  "Pineapple": (symbol: "PAPL", price: 148.32),
  "Foogle": (symbol: "FOOGL", price: 2750.62),
  "Visla": (symbol: "VSLA", price: 609.89),
  "Mehzon": (symbol: "MHZN", price: 3271.92),
  "Picohard": (symbol: "PCHD", price: 265.51),
}.toTable()

let shoutExpensive = stocks.pairs()
                      .mapIt((name: it[0], price: it[1].price))
                      .filterIt(it.price > 1000.0)
                      .mapIt(it.name)
                      .map(toUpperAscii)
                      .collect(HashSet[string])

assert shoutExpensive == ["FOOGLE", "MEHZON"].toHashSet()

Or wrapped in a colsure:

import std/[iterutils, strutils, tables]

let philosophers: Table[string, string] = {
  "Plato": "Greece", "Aristotle": "Greece", "Socrates": "Greece",
  "Confucius": "China", "Descartes": "France"}.toTable()

const Phrase = "$1 is a famous philosopher from $2."
let facts = philosophers.pairs()
              .filterIt(it[1] != "Greece")
              .mapIt([it[0], it[1]])
              .mapIt(Phrase % it)
              .asClosure()

assert facts() == "Confucius is a famous philosopher from China."
assert facts() == "Descartes is a famous philosopher from France."
assert facts.finished

Implementation

• Adopt core logic from slicerator

• Clean up API and doc comments for std/iterutils

• Add thorough unit tests

Alternatives

• Leave this in third-party libraries (status quo), but this reduces visibility and consistency

• Reimplement custom pipelines per project (wasteful and error-prone)

• Use sugar.collect (not as flexible).

Downsides

• Adds a new module to the stdlib, increasing its size.

• All the template expansions may introduce compile-time overhead.

Code Examples

No response

Backwards Compatibility

This is purely additive. No existing features or modules are affected. The name iterutils is consistent with similar modules like sequtils.