v10.1.0 broke agent log forwarding feature with log4net 1.2.15

[pbering]

https://docs.newrelic.com/docs/logs/logs-context/net-configure-logs-context-all states that log4net: 1.2.10+ (.NET Framework) is supported, but on one of my solutions that has a hard dependency to log4net 1.2.15 has started to fail when I upgraded the agent from 10.0.0 to 10.1.0. The error seen in the New Relic agent log is:

2022-09-29 08:16:13,591 NewRelic  ERROR: [pid: 17472, tid: 64] Tracer invocation error: System.NullReferenceException: Object reference not set to an instance of an object.
   at NewRelic.Agent.Core.Agent.RecordLogMessage(String frameworkName, Object logEvent, Func`2 getTimestamp, Func`2 getLevel, Func`2 getLogMessage, Func`2 getLogException, String spanId, String traceId)
   at NewRelic.Providers.Wrapper.Logging.Log4netWrapper.RecordLogMessage(Object logEvent, Type logEventType, IAgent agent)
   at NewRelic.Providers.Wrapper.Logging.Log4netWrapper.BeforeWrappedMethod(InstrumentedMethodCall instrumentedMethodCall, IAgent agent, ITransaction transaction)
   at NewRelic.Agent.Core.Wrapper.WrapperService.BeforeWrappedMethod(Type type, String methodName, String argumentSignature, Object invocationTarget, Object[] methodArguments, String tracerFactoryName, String metricName, UInt32 tracerArguments, UInt64 fu
nctionId)
   at NewRelic.Agent.Core.AgentManager.GetTracerImpl(String tracerFactoryName, UInt32 tracerArguments, String metricName, String assemblyName, Type type, String typeName, String methodName, String argumentSignature, Object invocationTarget, Object[] argu
ments, UInt64 functionId)

I noticed a change in 10.1.0 https://github.com/newrelic/newrelic-dotnet-agent/pull/1228/files#diff-25a73084b562e9910081a3264247a8bcf21898057deddbba65b88e1260ca4d21R58 this is likely the problem as log4net 1.2.x does NOT have a ExceptionObject property, but it does have private readonly Exception m_thrownException, maby this can be used as fallback when ExceptionObject is not present?

This issues is a blocker for us as we can not upgrade to later versions of log4net, we need this to be fixed to be able to upgrade to newer agent versions.

[workato-integration[bot]]

https://issues.newrelic.com/browse/NEWRELIC-4004

[workato-integration[bot]]

Commented by ahemsath: @pbering: Thanks for bringing this to our attention.

I double checked that our integration tests for the log forwarding feature are testing a log4net 1.2.x version, and they are (1.2.10, specifically).  They are also testing the exception info forwarding logic, and those tests are passing.  I ran the test application manually with the agent attached in debug mode and verified that the code that gets the {{ExceptionObject}} property is being called successfully.

I looked at the log4net SDK documentation for version 1.2.15 and it shows the ExceptionObject property as being there: [https://logging.apache.org/log4net/log4net-1.2.15/release/sdk/html/P_log4net_Core_LoggingEvent_ExceptionObject.htm]

The code in our test application that logs the exception looks like this:  {code:java} _log.Error(exception.Message, exception); {code} Do you know if there is a particular logging call in your application that triggers this error?  If so, could you share what that looks like, suitably scrubbed of any sensitive information?

Since this issue does appear specific to something about your application or environment, in order to continue troubleshooting we may need to start asking for private information like agent log files.  If that should prove necessary, it will be better to handle this through our Global Technical Support team.  To submit a ticket to support, you have two options depending on you New Relic account level: if you have a paid account that allows you to submit support tickets, please open one on our support portal ([https://support.newrelic.com/new-ticket)] and reference this GitHub issue; if you're on the free tier or a paid level without support, please post a question on the New Relic Explorer's Hub ([https://discuss.newrelic.com/)] with a link to this GitHub issue.

 

[pbering]

Hi ahemsath, you are indeed correct about ExceptionObject is present in 1.2.10 which means that I'm using an even older version than I thought. I use a "special" log4net version embedded in another assembly from this package: https://sitecore.myget.org/feed/sc-packages/package/nuget/Sitecore.Logging/10.2.0 which does not have ExceptionObject property, but it does have the m_thrownException field, same as 1.2.10/1.2.15.

Is there any possibility that you are willing to do some kind of fallback to the m_thrownException field so it would work also in my case (Sitecore is a commonly used enterprise CMS) ?

To actually get New Relic to use the embedded log4net from Sitecore.Logging.dll I have overridden NewRelic.Providers.Wrapper.Log4NetLogging.Instrumentation.xml with:

<?xml version="1.0" encoding="utf-8"?>
<extension xmlns="urn:newrelic-extension">
  <instrumentation>
    <tracerFactory name="log4net">
      <match assemblyName="Sitecore.Logging" className="log4net.Repository.Hierarchy.Logger">
        <exactMethodMatcher methodName="CallAppenders" />
      </match>
      <match assemblyName="log4net" className="log4net.Repository.Hierarchy.Logger">
        <exactMethodMatcher methodName="CallAppenders" />
      </match>
    </tracerFactory>
  </instrumentation>
</extension>

Kind of a hack but worked until 10.1.0 ;-) and way easier that setting up some other log shipping solution.

I understand if you do not want to do fallback to m_thrownException as it is quite an edge case and out of scope of the supported 1.2.0+ version... If this is the case, feel free to close this issue.

[workato-integration[bot]]

https://issues.newrelic.com/browse/NEWRELIC-4119

[nr-ahemsath]

@pbering Thanks for the detailed reply. Our team thinks it is commendable that you took the initiative to modify the instrumentation XML to work with Sitecore, and we are interested in supporting users who take such steps to make our agent work in their environment. We don't currently support Sitecore's version of log4net, but we have had at least one other customer show interest in it. I've converted both this issue and our internal JIRA issue to be a feature request, which I hope can be scheduled in a reasonable timeframe.

We encourage PRs from community members, and if you are willing to take on the effort of making the modifications you suggest to use m_thrownException instead of the ExceptionObject property we will do our best to support you.

[github-actions[bot]]

This Issue has been marked stale after 90 days with no activity. It will be closed in 30 days if there is no activity.

[workato-integration[bot]]

Jira CommentId: 209906 Commented by chynes:

Moving this out of the Bug Smash – the plan is to add actual support for Sitecore logging in Q2.

[chynesNR]

Support for Sitecore.Logging was added in 10.14.0