lnbest0707
lnbest0707
tags: `feature`, `refactor`, `release-notes` This adds an evolved version of ShemaConformingTransformerV2, it evolves from the existing one with following new features: Refactored code with better readability and extensibility Support over-lapping...
After servers with invalid configuration joined the cluster, e.g. like below `{ "id": "", "simpleFields": { "HELIX_HOST": "", "HELIX_PORT": "" }, "mapFields": {}, "listFields": {} }` The broker (even in...
Pinot data ingestion from Kafka is following the 1 thread per Kafka partition mechanism. The scaling up is relying on increasing number of Kafka topic partitions. However, due to the...
Json index is a frequently used feature but currently with very limited configuration ways. Users could only have a global max level or array index limit. The existing includePaths/excludePaths(fields) could...
`feature` Reference: https://github.com/apache/pinot/issues/13780 [Design Doc](https://docs.google.com/document/d/1Er2Tmtl5Pgwdapn5iOJ5qlCDU_2P67YMdpdsmL36MCk/edit?usp=sharing) Please refer to design doc for details. TLDR: - Add support to ingest from multiple source by a single table - Use existing interface (TableConfig)...
Pinot nowadays only supports realtime table ingested from one single source stream, e.g. one Kafka topic from a Kafka cluster. And inside the table manager, the internal segment partition concept...
`feature` `ingestion` To ingest json-like data through SchemaConformingTransformer, previously, we could only dump data to either dedicated column or a single json_data field. We may build json index on json_data...