hazelcast-jet
hazelcast-jet copied to clipboard
hazelcast
Distributed Stream and Batch Processing
Hi, when using ProcessorWrapper and eg. list sink, Following exception is thrown: ``` Caused by: java.lang.NullPointerException at com.hazelcast.jet.impl.connector.WriteBufferedP.init(WriteBufferedP.java:67) at com.hazelcast.jet.impl.processor.ProcessorWrapper.init(ProcessorWrapper.java:80) at com.hazelcast.jet.impl.processor.ProcessorWrapper.init(ProcessorWrapper.java:80) at com.hazelcast.jet.impl.execution.ProcessorTasklet.init(ProcessorTasklet.java:228) at com.hazelcast.jet.impl.execution.TaskletExecutionService$BlockingWorker.run(TaskletExecutionService.java:289) ... 5 more ```...
I would like a batch map processor to be added to the core API. See this Stack Overflow question for a detailed description: https://stackoverflow.com/questions/61752906/custom-processor-to-map-inputs-in-batches/61761295#61761295
This code is correct but inefficient ``` .flatMap(entry -> Traversers.traverseArray(WORDS_PATTERN.split(entry.getValue()))) .map(String::toLowerCase) .filter(word -> word.length() > 2) .groupingKey(Functions.wholeItem()) .rollingAggregate(AggregateOperations.counting()); ``` The filter would be more efficient before the map. Using the...
As a follow up to #2394, it should also be possible to use the distribute to one edge when using Pipeline API. SinkImpl currently supports only a few of the...
Jet 4.1 (Enterprise) fails for the gRPC call `mapUsingServiceAsyncBatched` can fail with an exception that would be better if handled. Exception message: ``` Caused by: com.hazelcast.jet.grpc.impl.StatusRuntimeExceptionJet: RESOURCE_EXHAUSTED at io.grpc.Status.asRuntimeException(Status.java:533) ~[grpc-api-1.26.0.jar!/:1.26.0]...
For example, given a simple pipeline such as: Pipeline p = Pipeline.create(); p.readFrom(TestSources.items("the", "quick", "brown", "fox")) .aggregate(aggregator) .writeTo(Sinks.logger()); I'd like `aggregator` to be something requiring a non-serialisable dependency to do...
On Jet 4.4-SNAPSHOT, `readFrom(TestSources.items(List.of(Tuple2.tuple2("hello", "world"))))` gives ``` java.lang.IllegalArgumentException: "fillBufferFn" must be serializable at neil.demo.MyThingTest.test(MyThingTest.java:43) Caused by: java.io.NotSerializableException: com.hazelcast.jet.datamodel.Tuple2 at neil.demo.MyThingTest.test(MyThingTest.java:43) ``` Full reproducer ``` package neil.demo; import java.util.List; import org.junit.jupiter.api.AfterAll;...
In hazelcast-jet-sql:4.4-SNAPSHOT, only Java can be used to add class definitions Need to use ``` ClassDefinition myPortableClassDefinition = new ClassDefinitionBuilder(MY_FACTORY_ID, MY_PORTABLE_ID) .addUTFField("payload") .build(); config.getSerializationConfig().addClassDefinition(myPortableClassDefinition); ``` Would wish to use XML...
When querying a streaming source such as a Kafka topic, it would be useful to have a time bound to terminate the query Eg `SELECT * FROM public topicname LIMIT...
It would be useful if `CREATE EXTERNAL MAPPING` could be loaded from `hazelcast.xml`/`hazelcast.yaml`, and subject to config replacers for the likes of bootstrap servers Consideration should also be given to...