Ting Chen

> However, in reality, users may e.g. dump data like request/response body to the json data and query them. For those special json field, they may want special treatment like...

Why can't we use the _includePaths_ config in Json index configuration option to explicitly index a certain path with any depth? https://docs.pinot.apache.org/basics/indexing/json-index#enable-and-configure-a-json-index

This stackoverflow question suggests ElasticSearch (built on top of Lucene) supports similar substring search feature. https://stackoverflow.com/questions/44791075/in-elasticsearch-how-do-i-search-for-an-arbitrary-substring cc @atris @siddharthteotia

#12680 enabled the support for wildcard (including prefix and suffix) matching for the terms in a phrase search. @hpvd you can follow the https://docs.pinot.apache.org/basics/indexing/text-search-support#phrase-search-with-wildcard-term-matching to test the feature. We tested...

As can be seen from the event sequence below, the follow call `Removing upsert metadata for segment:` took 20s to finish and is the main culprit of the slow reload....

@yupeng9 The segment removal happens in the following step during adding a segment: https://github.com/apache/pinot/blob/c9e36dd0a53370d7b8dc24459426dc77497eb8ac/pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java#L157 In segment reload, a replica of a segment is in fact added using the above step.

Here is my followup investigation: (1) There is no unit test on the following method for a normal Pinot table let alone an upsert table. We should add unit tests...