[HUDI-6613] implement inmemory file index to allow for glob paths

[jonvex]

Change Logs

implement inmemory file index to allow for queries with glob paths

Impact

improve new reader

Risk level (write none, low medium or high below)

low

Documentation Update

N/A

Contributor's checklist

• [ ] Read through contributor's guide
• [ ] Change Logs and Impact were stated clearly
• [ ] Adequate tests were added if applicable
• [ ] CI passed

[hudi-bot]

CI report:

• 2cf4e2b968d44e10937dc2a46ee4865a8d8cee67 Azure: FAILURE

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build