apache
_enableSnapshot loads upsert segments from snapshot even if _enablePreload is disabled
Recently, on one of our customer tables we added a comparison column where enableSnapshot was true but enablePreload was false. After adding the comparison column and restarting, the expected records for a key were still not returned. Upon further investigation, I discovered that with enableSnapshot set to true, the snapshot is read to load a segment. This behavior is counterintuitive since we already have a configuration enablePreload that handles this. Consequently, we had to disable enableSnapshot, restart, then re-enable enableSnapshot, and restart again to update one configuration (2 restarts).
IMO we should not perform a preload-type operation unless enablePreload is set explicitly. We can remove these lines during addSegment flow to achieve this: https://github.com/apache/pinot/blob/8df722a22f089d088bfb03e2b8aa49549bc4c38c/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java#L436-L442
cc @Jackie-Jiang @klsince what are your thoughts on this?
