fluent-plugin-gcs icon indicating copy to clipboard operation
fluent-plugin-gcs copied to clipboard

Failing to upload due to Google Storage Library Updates

Open joshughes opened this issue 2 years ago • 2 comments

2022-08-15 20:04:00 +0000 [warn]: #0 [out_gcs] got unrecoverable error in primary and no secondary error_class=ArgumentError error="wrong number of arguments (given 3, expected 1..2)"
  2022-08-15 20:04:00 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/google-cloud-storage-1.38.0/lib/google/cloud/storage/bucket.rb:1557:in `create_file'
  2022-08-15 20:04:00 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluent-plugin-gcs-0.4.1/lib/fluent/plugin/out_gcs.rb:134:in `block in write'
  2022-08-15 20:04:00 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluent-plugin-gcs-0.4.1/lib/fluent/plugin/gcs/object_creator.rb:39:in `block in create'
  2022-08-15 20:04:00 +0000 [warn]: #0 /usr/local/lib/ruby/3.1.0/tempfile.rb:358:in `create'
  2022-08-15 20:04:00 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluent-plugin-gcs-0.4.1/lib/fluent/plugin/gcs/object_creator.rb:35:in `create'
  2022-08-15 20:04:00 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluent-plugin-gcs-0.4.1/lib/fluent/plugin/out_gcs.rb:123:in `write'
  2022-08-15 20:04:00 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.15.1/lib/fluent/plugin/output.rb:1180:in `try_flush'
  2022-08-15 20:04:00 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.15.1/lib/fluent/plugin/output.rb:1501:in `flush_thread_run'
  2022-08-15 20:04:00 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.15.1/lib/fluent/plugin/output.rb:501:in `block (2 levels) in start'
  2022-08-15 20:04:00 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.15.1/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2022-08-15 20:04:00 +0000 [warn]: #0 [out_gcs] bad chunk is moved to /tmp/fluent/backup/worker0/out_gcs/5e64d1e3550ff0cf7802d2dbc6c511cc.log

Seeing this issue tyring to use fluent/fluentd-kubernetes-daemonset:v1.15-debian-gcs-1

Seems like the google libraries maybe are not locked and they expect fewer arguments...

joshughes avatar Aug 15 '22 20:08 joshughes

Thank you for your report. The cause of this error was that the automatic conversion of hashes to keyword arguments was deprecated starting with Ruby 2.7 and removed starting with Ruby 3.0.

I will create a PR to fix this issue.

daichirata avatar Aug 16 '22 10:08 daichirata

@daichirata still having similar issue with v1-debian-gcs

2023-01-05 07:52:59 +0000 [warn]: #0 [out_gcs] got unrecoverable error in primary and no secondary error_class=ArgumentError error="wrong number of arguments (given 2, expected 1)"     │
│   2023-01-05 07:52:59 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/google-cloud-storage-1.43.0/lib/google/cloud/storage/bucket.rb:1314:in `file'                              │
│   2023-01-05 07:52:59 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluent-plugin-gcs-0.4.1/lib/fluent/plugin/out_gcs.rb:156:in `check_object_exists'                          │
│   2023-01-05 07:52:59 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluent-plugin-gcs-0.4.1/lib/fluent/plugin/out_gcs.rb:180:in `generate_path'                                │
│   2023-01-05 07:52:59 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluent-plugin-gcs-0.4.1/lib/fluent/plugin/out_gcs.rb:121:in `write'                                        │
│   2023-01-05 07:52:59 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:1180:in `try_flush'                                             │
│   2023-01-05 07:52:59 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:1501:in `flush_thread_run'                                      │
│   2023-01-05 07:52:59 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin/output.rb:501:in `block (2 levels) in start'                              │
│   2023-01-05 07:52:59 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/3.1.0/gems/fluentd-1.15.3/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'                           │
│ 2023-01-05 07:52:59 +0000 [warn]: #0 [out_gcs] bad chunk is moved to /tmp/fluent/backup/worker0/out_gcs/5f17f532dd1e11863e3da33cea4bcc88.log          

pnutmath avatar Jan 05 '23 08:01 pnutmath