Include PathBase when logging a transaction

[TrueGoric]

Is your feature request related to a problem? Please describe. In our k8s-based microservice we are logging requests across the request pipeline, with the main focus on two places - our ingress controller and ASP.NET Core application. Ingress controller rewrites the request path, excluding the prefix, which is then forwarded to the service in X-Forwarded-Prefix header and included in the HttpContext.Request.PathBase property.

Unfortunately the APM agent doesn't include the PathBase when logging, which prevents us from correlating spans across a transaction. Including it by default should ease the integration of Elastic APM into services sitting behind any kind of reverse proxy, but might affect users that specifically moved the APM agent further down the ASP.NET middleware pipeline.

Describe the solution you'd like Given the fact that the combination of PathBase and Path canonically represents the full request path (excluding the protocol and domain parts) I'd like the APM to include PathBase when logging the transaction. The change should preferably be configurable to avoid introducing breaking changes.

Describe alternatives you've considered As a workaround we've temporarily stopped path rewriting by the ingress controller and moved APM further up in the middleware pipeline, but this way we are forcing our microservices into a situation where they need to know their target environment.

Additional context I'm preparing a PR with the changes that would enable the agent to include PathBase during transaction logging - it should be posted tomorrow.

[gregkalapos]

Thanks for opening this @TrueGoric,

before we jump on the implementation, could we please talk about the issue itself. I'm not sure I fully understand the problem, so let me ask some questions.

If I understand it correctly, following happens:

• Let's say you have an incoming request with /MyPathBase/foo/bar/.
• MyPathBase maps the ingress controller, which forwards the request to the ASP.NET Core app.
• The forwarded request has the path /foo/bar, and it'll have an X-Forwarded-Prefix header with /MyPathBase which the ASP.NET Core pipeline sets as HttpRequst.PathBase.

Is that a correct description? If not, feel free to correct me.

Now - the part I don't understand is this one:

Unfortunately the APM agent doesn't include the PathBase when logging, which prevents us from correlating spans across a transaction.

I understand that the PathBase (in this case the /MyPathBase) won't be part of the transaction name, but why does it make it hard to correlate spans? Correlation across services works via the W3C Trace Context header and that's not related to the transaction names. So if you have another APM agent in the caller service, then it'll be correlated regardless of the transaction name. Here by "correlating" I mean that you'll see the services in the same water flow diagram in Kibana under APM and everything will have the same trace id across all the services.

Also, the headers of the requests are captured on the transaction, so you'd see an http.request.headers.X-Forwarded-Prefix field on the transaction. Does that help?

So could you please elaborate a bit on this part of the problem? I don't fully understand what you'd like to correlate and how that works. I hope it's not based on transaction name :)

So could you please explain me what is on the caller side and how you'd like to correlate that to the ASP.NET Core transaction? It'd be nice for me to understand this part before we add a config for it.

Maybe another info that could help you: We have multiple ways of enabling ASP.NET Core monitoring - the middleware based solution is just one of them, you could also enable it with the IHostBuilder based integration like this:

public static IHostBuilder CreateHostBuilder(string[] args) =>
	Host.CreateDefaultBuilder(args)
		.UseAllElasticApm() //Here we enable the agent
		.ConfigureWebHostDefaults(webBuilder =>
		{
			webBuilder.UseStartup<Startup>();
		});

The exact some code runs with zero code change setup. I'm not sure if this helps, but wanted to bring this up, because this will influence how we'd implement the config you suggest.

[gregkalapos]

@TrueGoric any thoughts on this? No rush, just want to make sure we understand the exact use-case before we add a config to support it.

[TrueGoric]

Yes, sorry for leaving this thread hanging like that, I didn't have a chance to discuss your suggestions with our devops team yet. I should get back to you by monday.

On August 17, 2022 2:49:58 PM GMT+02:00, Gergely Kalapos @.> wrote: @. any thoughts on this? No rush, just want to make sure we understand the exact use-case before we add a config to support it.

-- Reply to this email directly or view it on GitHub: https://github.com/elastic/apm-agent-dotnet/issues/1771#issuecomment-1217966903 You are receiving this because you were mentioned.

Message ID: @.***>

[gregkalapos]

No worries - I leave this open.

[gregkalapos]

Closing now - feel free to ping us if you think we can engage and process with the questions above.