Protobuf serialization in Ruby very expensive

[clifton]

Hey Kyle,

@justinsteffy and I have been hitting performance snags doing 50-100 serializations during the request response cycle. We've identified that most of the time is spent in the beefcake library doing protobuf serialization. Have you had any issues with this?

r = Riemann::Client.new
# => #<Riemann::Client:0x00000001fc3040 @host="127.0.0.1", @port=5555, @timeout=5, @udp=#<Riemann::Client::UDP:0x00000001fc2fc8 @host="127.0.0.1", @port=5555, @max_size=16384, @locket=#<Mutex:0x00000001fcafc0>>, @tcp=#<Riemann::Client::TCP:0x00000001fcaf98 @host="127.0.0.1", @port=5555, @timeout=5, @locket=#<Mutex:0x00000001fcaf70>>>
r << { service: 'testing', metric: 1.0, tags: ['test', 'the', 'client'] }
# => nil
require 'benchmark'
# => true
Benchmark.measure do
  1000.times do
    r << { service: 'testing', metric: 1.0, tags: ['test', 'the', 'client'] }
  end
end
=>   0.120000   0.060000   0.180000 (  0.185700)

[ghost]

Wouldn't be surprised, but I'm pretty much all JVM these days so I haven't hit performance issues personally. If you have alternative suggestions for a protobuf library (or improvements to Beefcake) I'd be open to a PR.

[jamtur01]

@smortex I never looked very far at this but beefcake is fairly old at this point. There's the Google protobuf ruby client itself (https://rubygems.org/gems/google-protobuf) from memory and some other alternative gems like https://github.com/ruby-protobuf/protobuf too.

[jamtur01]

Looks like https://github.com/ruby-protobuf/protobuf would pretty much a near drop-in replacement assuming we could benchmark it against beefcake.

[smortex]

Reopen since we reverted the change and we must investigate this