implement non-blocking stream server handler

[naoh87]

What this PR do

• Eliminate blocking from stream server runtime.
• align unary server runtime I/F
• fix flaky ServerSuit

ClientStreaming Benchmark

method implement

request.compile.last.map(_.fold(HelloReply())(r => HelloReply(r.request)))

Request 100 messages with ghq --cpu=4 -z 60s --connections=5 ...

#503

actual cpu used: 1.0core
Summary:
  Count:	39264
  Total:	60.00 s
  Slowest:	385.42 ms
  Fastest:	2.80 ms
  Average:	74.86 ms
  Requests/sec:	654.39

main

actual cpu used: 1.0core
Summary:
  Count:	11280
  Total:	60.00 s
  Slowest:	1.09 s
  Fastest:	7.85 ms
  Average:	265.53 ms
  Requests/sec:	188.00

[ahjohannessen]

@naoh87 I have been busy with other stuff and have not forgotten about this PR. I'll look more closely into it when I find the time.

[ahjohannessen]

@naoh87 I have not forgotten this, however upcoming work with queues in cats-effect 3.4.x might give us better building blocks.

One question, does your work here take into account back pressure? For instance, a server can't keep up with a client streaming too much.

[naoh87]

I think new cats-effect 3.4.x queue will now work as good as this PR, because new CE3 queue is implemented as MPMC, but we just need SPSC. https://github.com/JCTools/JCTools#jctools

and also new implementation does not reduce Dispatcher[F] cost at onMessage.

[ahjohannessen]

I think new cats-effect 3.4.x queue will now work as good as this PR, because new CE3 queue is implemented as MPMC, >but we just need SPSC. https://github.com/JCTools/JCTools#jctools

and also new implementation does not reduce Dispatcher[F] cost at onMessage.

Ok. I think we should try to use as much code from cats-effect/fs2 where possible. That reduces the amount of code we have to maintain - I am thinking mostly about OneShotChannel.