Ben Pfaff

Kafka test failures for "Partition queues should be split off for all subscribed partitions"

9

https://github.com/feldera/feldera/actions/runs/7639276759/job/20811954233 shows a failure in `https://github.com/feldera/feldera/actions/runs/7639276759/job/20811954233`: ```log +test-adapters *failed* | ---- transport::kafka::ft::test::test_empty_input stdout ---- +test-adapters *failed* | 2024-01-24 11:15:00.021943 INFO librdkafka: CONFWARN [thrd:app]: No `bootstrap.servers` configured: client will not be...

# Summary [summary]: #summary The Feldera version number, and possibly the commit number, should be available from the API and possibly the web console. # Motivation [motivation]: #motivation Making the...

Avoid need for `F64` and `F32` newtypes in DBSP

3

DBSP has newtypes `F64` and `F32` for floating-point support. It would be nice if we could just use `OrderedFloat` instead. Here are the steps I know about so far, in...

nexmark q6 and q19 should use `VecDeque`s again when `rkyv` releases

1

`rkyv` didn't support `VecDeque` when we switched to `rkyv`, so https://github.com/feldera/dbsp/pull/513 switched to `Vec`, which isn't ideal. When `rkyv` 0.7.43 releases (which will support `VecDeque`), we should switch back to...

Gather operator needs distributed implementation

2

So far, only the `exchange` operator has a distributed implementation (https://github.com/feldera/dbsp/pull/259). The `gather` operator also needs one.

DECIMAL documentation and implementation disagree

4

https://www.feldera.com/docs/sql/decimal/ says: > The maximum precision supported is 128 binary digits (38 decimal digits). but we're using https://github.com/gz/rust-decimal, which says: > The binary representation consists of a 96 bit integer...

Kafka transport configuration defers too many errors

1

The design for the Kafka input and output transport configurations have a flattened map, like this: ``` /// Options passed directly to `rdkafka`. /// /// See [`librdkafka` options](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) /// used...

The DBSP core currently distinguishes between individual batches and traces. For batches, it provides the guarantee that weights are nonzero, and some code depends on this. For traces, it does...

Experiment with merge policies

1

Some possible merge policies: * If we always merge the smallest batches of size k in a slot, we will pile up batches of size 2k in that slot if...

Get rid of circuits without a runtime

2

Circuits outside a `Runtime` cause awkward special cases in some code, including as referenced in https://github.com/feldera/feldera/pull/2115#discussion_r1695616513. We can get rid of the special cases by only allowing circuits to be...