KAFKA-19970: Add configurable TTL for tiered storage index cache eviction

[nandini12396]

Caffeine cache currently only uses size-based eviction with frequency buckets. Within same frequency bucket, larger entries are always evicted. This causes old (smaller) index files to stay in cache indefinitely while newer indices thrash, resulting in:

• Poor cache utilization for backfill workloads
• Higher fetch errors
• Suboptimal memory usage

Solution:

Add time-based eviction via expireAfterAccess in addition to size-based eviction:

1. New config: remote.log.index.file.cache.ttl.ms (default: -1/disabled)
2. If TTL > 0: entries evicted after last access time expires

[showuon]

@nandini12396 , adding new config (public API) needs to go through the KIP process: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.

[github-actions[bot]]

A label of 'needs-attention' was automatically added to this PR in order to raise the attention of the committers. Once this issue has been triaged, the triage label should be removed to prevent this automation from happening again.

[kamalcph]

Nice find! Thanks for the patch, adding new configs require KIP to proceed.

[kamalcph]

@showuon Since this qualify as a bug, shall we go-ahead with internal config? And, open a KIP in-parallel to expose the config to users. WDYT?