alpakka icon indicating copy to clipboard operation
alpakka copied to clipboard

No backpressure support for Couchbase n1ql source

Open aalleexxeeii opened this issue 2 years ago • 2 comments

Do I understand it right that CouchbaseSource.fromN1qlQuery() doesn't support backpressure at all? I used akka-stream-alpakka-couchbase:3.0.3 with Scala 2.12.13 and Akka 2.6.14

A graph like this being run on a dataset with a large number of records simply runs out of memory instead of being paused by backpressure initiated by throttling.

    CouchbaseSource.fromN1qlQuery(settings, query, "sample")
      .throttle(elements = 3, per = 1.second)
      .runForeach { x ⇒
        println(x.getString("id"))
      }
      .onComplete {
        case Success(x) ⇒ logger.info(s"Completed: $x")
        case Failure(ex) ⇒ logger.info(s"Failed: $ex", ex)
      }

aalleexxeeii avatar Sep 14 '21 13:09 aalleexxeeii

@dannylesnik, @ennru, as you were the main contributors, are there any plans to introduce proper backpressure and start supporting SDK 3 as 2.7 is no longer officially supported.

aalleexxeeii avatar Mar 31 '22 09:03 aalleexxeeii

I'm not aware of any work on Alpakka Couchbase right now. If you're up for it, please suggest the necessary changes.

ennru avatar Apr 04 '22 09:04 ennru