pulsar-io-cloud-storage
pulsar-io-cloud-storage copied to clipboard
[FEATURE] Add datetime string substitutions to prefix field
Is your feature request related to a problem? Please describe.
Now that the new partitioner doesn't add a datetime based prefix to the file it would be useful if we could provide the date/time as part of the prefix while using partitioner: topic
through placeholders.
Describe the solution you'd like
Something like being able to specify %Y-%M-%D
in the prefix
field of the config and have the writer substitute those placeholders for the actual date values (doesn't have to be those exact placeholders but thats the idea)
Describe alternatives you've considered
The alternative is to continue using the legacy
partitioner along with the time settings but it seems like the legacy partitioner is eventually going to be deprecated so we figured this was more future-proof.
Additional context Add any other context or screenshots about the feature request here.