exception happened in background worker: source sequence is illegal/malformed utf-8

[grosser]

Issue Description

Something fails in production and all I see in the logs is this message, which is not helpful since I need the original backtrace and error message to begin debugging

Reproduction Steps

raise an error that includes invalid utf8

Expected Behavior

<?> icons replace invalid utf8

Actual Behavior

error not sent to sentry and log shows message

Ruby Version

3.4.6

SDK Version

5.28.0

Integration and Its Version

none

Sentry Config

nothing special, except a custom logger that does {message: message}.to_json

[linear[bot]]

RUBY-123

[solnic]

@grosser thanks for the report. Any chance you could give me an example message that causes this? We do have handling of malformed utf-8 strings in place.

[grosser]

sadly did not find more info and don't have a way to reliably reproduce it feel free to close if you don't see any obvious spot where this could have slipped through

[solnic]

@grosser alright, I'll dig into it - could you tell me what exact custom logger do you use that does {message: message}.to_json?

[grosser]

standard logger with a formatter that does:

def call(level, _, _, m)
  message = {message: m.message, backtrace: m.backtrace.first(5).join("\n")} if m.is_a?(Exception)
  message = {message: m} if m.is_a?(String)
  {severity: level, worker: @worker}.merge(message).to_json << "\n"
end

[solnic]

@grosser how exactly do you configure this logger in Sentry config?

[grosser]

class JsonLogger
  def call(level, _, _, m)
    message = m
    message = {message: m.message, backtrace: m.backtrace.first(5).join("\n")} if m.is_a?(Exception)
    message = {message: m} if m.is_a?(String)
    {severity: level}.merge(message).to_json << "\n"
  end

  def self.build(out)
    logger = Logger.new(out, level: Logger::INFO)
    logger.formatter = new
    logger
  end
end

Sentry.init do |config|
  config.sdk_logger = JsonLogger.build($stdout)
  ...
end