zync
zync copied to clipboard
brimdata
Kafka connector to sync Zed lakes to Kafka topics
To test the service defined in brimdata/zync#83, this issue is to create a performance test rig using large amounts of synthetic data. We will create a large number of data...
A community zync user reported the following in their own words: > I have this error with `zed serve` and `zync`. It's less frequent with Zed 1.7 but still present....
Syncing from kakfa and doing ETL should run as a continuous service so we don't need to poll and recompute progress state on each run. Step 1 is to get...
We should think through and implement how zync should do logging and change the various printf's to log calls. This will be used from orchestration and automation so these logs...
This is a little thing, but it strikes me as odd that `zync` uses a "command" to get the version: ``` $ zync version d87d49b ``` Whereas the other Zed...
The yaml config should allow logic to refer to more than one output table and track the input cursors to multiple outputs.
The yaml config should allow logic to refer to more than two input tables to create complex joins between three or more inputs and track progress cursors accordingly.
In `etl/pipeline.go`, the commented out code cannot replace the code below for some reason... ``` //XXX This still doesn't work with the bug fix. See issue # //if vals[k].Type ==...
See this comment in `zync/design.md`: ``` > This is deprecated by design outline in the README. > We need update the implemented algorithm with cursors to limit the > scans...
Zinger should have options to run more generically by syncing a kafka topic to a pool without the strict enforcement of the sequential offset field in the `kafka` meta record....