streams
streams copied to clipboard
Add MoreStream utility methods that accept future suppliers
Currently the non-deprecated versions of inCompletionOrder and blockingStreamWithParallelism require a an executor to run each transformation on, which means that that transformation will be a blocking call. This is not ideal when the actual call is itself async (i.e. hitting a method which returns a ListenableFuture already). This PR adds versions of those methods that require a future supplier, that is a method that transforms the keys into the listenable futures of the result. This method will be called by the thread that is pulling from the Stream.
This is similar to earlier problematic methods that accepted an input a stream of futures. Those were deprecated because they did not enforce actual max parallelism. That is, the underlying stream supplier may kick off more futures than the parallelism intended to allow. I believe this solution avoids those issues because we only create new futures from within the buffering spliterator, which enforces the max parallelism.
Thanks for your interest in palantir/streams, @msg430! Before we can accept your pull request, you need to sign our contributor license agreement - just visit https://cla.palantir.com/ and follow the instructions. Once you sign, I'll automatically update this pull request.
Generate changelog in changelog/@unreleased
changelog/@unreleasedWhat do the change types mean?
feature: A new feature of the service.improvement: An incremental improvement in the functionality or operation of the service.fix: Remedies the incorrect behaviour of a component of the service in a backwards-compatible way.break: Has the potential to break consumers of this service's API, inclusive of both Palantir services and external consumers of the service's API (e.g. customer-written software or integrations).deprecation: Advertises the intention to remove service functionality without any change to the operation of the service itself.manualTask: Requires the possibility of manual intervention (running a script, eyeballing configuration, performing database surgery, ...) at the time of upgrade for it to succeed.migration: A fully automatic upgrade migration task with no engineer input required.
Note: only one type should be chosen.
How are new versions calculated?
- ❗The
breakandmanual taskchangelog types will result in a major release! - 🐛 The
fixchangelog type will result in a minor release in most cases, and a patch release version for patch branches. This behaviour is configurable in autorelease. - ✨ All others will result in a minor version release.
Type
- [ ] Feature
- [ ] Improvement
- [ ] Fix
- [ ] Break
- [ ] Deprecation
- [ ] Manual task
- [ ] Migration
Description
Check the box to generate changelog(s)
- [ ] Generate changelog entry