spark icon indicating copy to clipboard operation
spark copied to clipboard

Published 20 hours ago • verified publisher icon apache

[SPARK-52327][Core] Glob based provider for history server

Open GauravWaghmare opened this issue 6 months ago • 2 comments

What changes were proposed in this pull request?

A new provider for history server is added for consuming event logs from multiple directories based on a glob instead of a single base directory.

Why are the changes needed?

There are usecases for eg., multi tenancy where a logical separation for event logs is required in which case the history server would have to read logs from multiple directories.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Unit tests are added

Was this patch authored or co-authored using generative AI tooling?

No

GauravWaghmare avatar May 29 '25 04:05 GauravWaghmare

+CC @thejdeep

mridulm avatar Jun 02 '25 06:06 mridulm

@thejdeep @mridulm Requesting you to help with the review please.

GauravWaghmare avatar Jun 09 '25 04:06 GauravWaghmare

++ @LuciferYang @dongjoon-hyun

GauravWaghmare avatar Jun 23 '25 05:06 GauravWaghmare