Channel - SendAll: work chunk-wise

[diesalbla]

Write sendAll to work at level of chunks, to avoid some overheads of Stream evalMap (since there is no downstream cut).

[SystemFw]

Have we measured this to make a difference?

[diesalbla]

Have we measured this to make a difference?

No. It is based on the knowledge and interpretation of the definition of evalMap:

  def evalMap[F2[x] >: F[x], O2](f: O => F2[O2]): Stream[F2, O2] =
    flatMap(o => Stream.eval(f(o)))

  def eval[F[_], O](fo: F[O]): Stream[F, O] =
    new Stream(Pull.eval(fo).flatMap(Pull.output1))

For each object in the source stream, the evalMap method would create 1. a Pull.Eval object, 2. a Bind object, 3. a Pull.Output object, and 4. a Chunk object, which contains 5. a boxed Boolean object (edit although it may be that the JVM already caches this Boolean). Making the code work on chunks would reduce that from once per element to once per chunk.

The idea was to replace the main code with a Pull, but it seemed that chunk-wise processing would be good enough.

def go(s: Stream[F, O]): Pull[F, INothing, Unit] = 
  s.pull.uncons.flatMap {
    case None => Pull.eval(close.void)
    case Some((h, t)) => 
      Pull.eval(sendChunk(h)).flatMap {
        case true => go(t)
        case false => Pull.eval(close.void)
      }
  }

[SystemFw]

Personally I'm a bit skeptical of reasoning about performance this way, we've had several cases where code that ought to be slower ended up being faster, including in Channel itself with the use of List. This type of change makes the code less readable, so imho it should be justified by a benchmark if the sole reason for having it is performance.

[mpilquist]

Agreed, we'll need benchmarks of some form for this - at very least informal ones if not full JMH ones.

[diesalbla]

Since the overhead of evalMap is a bit smaller now, this may not be needed.