pulsar-io-cloud-storage icon indicating copy to clipboard operation
pulsar-io-cloud-storage copied to clipboard

[FEATURE] Add datetime string substitutions to prefix field

Open asher-goldberg opened this issue 10 months ago • 0 comments

Is your feature request related to a problem? Please describe. Now that the new partitioner doesn't add a datetime based prefix to the file it would be useful if we could provide the date/time as part of the prefix while using partitioner: topic through placeholders.

Describe the solution you'd like Something like being able to specify %Y-%M-%D in the prefix field of the config and have the writer substitute those placeholders for the actual date values (doesn't have to be those exact placeholders but thats the idea)

Describe alternatives you've considered The alternative is to continue using the legacy partitioner along with the time settings but it seems like the legacy partitioner is eventually going to be deprecated so we figured this was more future-proof.

Additional context Add any other context or screenshots about the feature request here.

asher-goldberg avatar Apr 08 '24 17:04 asher-goldberg