SemanticLogger is incompatible with New Relic log forwarding

[texpert]

Environment

Provide at least:

• Ruby 2.7.6
• Semantic Logger Version 4.11.0
• Rails 6.1.6.1, Puma 5.6.4
• Rails Semantic Logger Version 4.10.0
• Rails configuration. Only need the settings related to Rails Semantic Logger and Semantic Logger.

  config.rails_semantic_logger.semantic = true
  config.rails_semantic_logger.started    = false
  config.rails_semantic_logger.processing = false
  config.rails_semantic_logger.rendered   = false
  config.semantic_logger.backtrace_level = nil
  # config.semantic_logger.backtrace_level = :debug

  SemanticLogger.add_appender(io: $stdout, level: :info)
  SemanticLogger.add_appender(appender: :new_relic, level: :warn)
  if ENV['MY_POD_NAME']&.start_with?('backend')
    SemanticLogger.add_appender(file_name: 'log/delayed_job.log')
  end

• New Relic Ruby agent - newrelic_rpm (8.6.0 - 8.9.0)

Expected Behavior

See the forwarded logs in the New Relic Triage->Logs dashboard, as the log forwarding support has been added in the 8.6.0 version of newrelic_rpm

Actual Behavior

No logs are forwarded, and there is a text message stating that "the log forwarding is not enabled" in the New Relic Triage->Logs dashboard

When removing the SemanticLogger gems from the app, the logs are being successfully forwarded.

On this New Relic page SemanticLogger is listed as an incompatible gem - https://docs.newrelic.com/docs/logs/logs-context/configure-logs-context-ruby

[geoffwa]

It looks like semantic_logger has SemanticLogger::Appender::NewRelic that will notify NewRelic of any logged errors.

The actual instrumentation that records the log messages appears to occur in NewRelic::Agent::Instrumentation::Logger where the an aggregator of log events gets called with a formatted log message. Potentially a custom appender could call this method (not sure if it's intended for public consumption).

[reidmorrison]

Anyone want to submit a Pull Request to add support for the above New Relic feature? Might be as simple as adding a new appender if the above NewRelic appender is not sufficient? Note that the log level for the existing NewRelic appender above can be set to any level, not just the default :error.

[reidmorrison]

If you add the New Relic appender and set the log level to something like :trace it will forward all logs to New Relic:

SemanticLogger.add_appender(appender: :new_relic, level: :trace)

Or if using Rails Semantic Logger, then in config/environments/production.rb:

config.semantic_logger.add_appender(appender: :new_relic, level: :trace)

[chriscz]

I started migrating our logging to semantic_logger yesterday with the goal of connecting it to NewRelic. I'm now at the point where I would like to connect to NR, but the API used by the NewRelicAppender isn't the logging API.

The Logging API is specified here: https://github.com/newrelic/newrelic-exporter-specs/tree/master/logging, the gem itself doesn't seem to have an API on the agent to push logs to NR without enabling forwarding, or at least so it seems to me.

Other resources I consulted

1. https://docs.newrelic.com/docs/logs/logs-context/configure-logs-context-ruby/
2. https://docs.newrelic.com/docs/logs/logs-context/logs-in-context/

I will be taking a stab at implementing a new appender tonight because I am not happy with the log forwarding facility offered by NewRelic at the moment

[chriscz]

Okay, I'm still digging a bit deeper here but I can see that the decorating formatter is not being applied for the standard logger. It seems the instrumentation from NewRelic and SemanticLogger step on each other here:

logger = NewRelic::Agent::Logging::DecoratingLogger.new "log/application.log"
2022-11-10 19:04:20.481318 I [390011:5798320 (pry):6] NewRelic::Agent::Logging::DecoratingLogger -- hello
=> true

[texpert]

If you add the New Relic appender and set the log level to something like :trace it will forward all logs to New Relic:

SemanticLogger.add_appender(appender: :new_relic, level: :trace)

Or if using Rails Semantic Logger, then in config/environments/production.rb:

config.semantic_logger.add_appender(appender: :new_relic, level: :trace)

Just tried config.semantic_logger.add_appender(appender: :new_relic, level: :trace) and it doesn't help to get the logs forwarded.

[chriscz]

Hey @texpert, not sure if you've seen my PR, but it might work for you.

I've got quite busy with end of year work so haven't integrated this into our company's application yet.

To fully use the new appender you need to enable NR log forwarding or the logs won't be dispatched, but I've raised this with the NR team already so their looking into it.

If you do give the PR a go, please let me know on that thread if you run into any issues.

[texpert]

Hey @texpert, not sure if you've seen my PR, but it might work for you.

I've got quite busy with end of year work so haven't integrated this into our company's application yet.

To fully use the new appender you need to enable NR log forwarding or the logs won't be dispatched, but I've raised this with the NR team already so their looking into it.

If you do give the PR a go, please let me know on that thread if you run into any issues.

Yay, it works after fixing a small typo in your PR, thank you!

Impatiently waiting for the PR to get merged!

[reidmorrison]

Thank you for submitting a Pull Request, merged and will be released shortly in v4.13.