`%{hex_random}` vs `%{uuid_flush}` / `${chunk_id}`

[okkez]

I'm testing fluent-plugin-s3 v1.5.1 (The previous version is v1.2.1)

I got the following warning message on boot.

#0 [user-identification-user-s3] No ${chunk_id} or %{uuid_flush} in s3_object_key_format with multiple flush threads. Recommend to set ${chunk_id} or %{uuid_flush} to avoid data lost by object conflict

https://github.com/fluent/fluent-plugin-s3/blob/6ab7c5343e2926e63cb2af7f2e1810e903bae48e/lib/fluent/plugin/out_s3.rb#L460

But I use %{hex_random} placeholder in s3_object_key_format. My s3_object_key_format is:

s3_object_key_format %{path}000.%{index}.%{hex_random}.tsv.%{file_extension}

Is there any difference between %{hex_random} and ${chunk_id} / %{uuid_flush}? Is there potential conflict using %{hex_random} without ${chunk_id} and %{uuid_flush}?

Should it be fixed the warning log?

[okkez]

%{uuid_flush} https://github.com/fluent/fluent-plugin-s3/blob/ab6191293b8840b31de5f2f4790ba334a39cbe33/lib/fluent/plugin/out_s3.rb#L387-L389

%{hex_random} https://github.com/fluent/fluent-plugin-s3/blob/ab6191293b8840b31de5f2f4790ba334a39cbe33/lib/fluent/plugin/out_s3.rb#L381-L385

${chunk_id} https://github.com/fluent/fluentd/blob/4b1ee636ec7b3123db6dfa5b3442a2f807bed35b/lib/fluent/plugin/output.rb#L727 https://github.com/fluent/fluentd/blob/4b1ee636ec7b3123db6dfa5b3442a2f807bed35b/lib/fluent/plugin/buffer/chunk.rb#L53 https://github.com/fluent/fluentd/blob/4b1ee636ec7b3123db6dfa5b3442a2f807bed35b/lib/fluent/unique_id.rb#L19-L32

I think ${chunk_id} and %{hex_random} are almost the same.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

[github-actions[bot]]

This issue was automatically closed because of stale in 30 days

[ashie]

The warning message is introduced at #326

[ashie]

I think we should fix the warning message.