apm-agent-ruby icon indicating copy to clipboard operation
apm-agent-ruby copied to clipboard

Published 20 hours ago • verified publisher icon elastic

Rails RAW_POST_DATA includes sensitive fields

Open dgoradia opened this issue 3 years ago • 4 comments

Describe the bug

Steps to reproduce

When POSTing to a auth api fields such as user and password end up in Rails' RAW_POST_DATA environment variable which doesn't get sanitized by sanitize_field_names and it ends up in the metadata section of a transaction in APM.

Expected behavior

Sensitive fields like password should be sanitized from RAW_POST_DATA env var in Rails.

Environment

  • OS: Linux
  • Ruby version: 2.7.1
  • Framework and version: Rails 6.1.4.4
  • APM Server version: 7.15.2
  • Agent version: 4.5.0

Additional context

Add any other context about the problem here.

  • Agent config options: default out of box config

dgoradia avatar Feb 23 '22 04:02 dgoradia

@dgoradia Thank you for reporting this. We'll have an update asap.

In the meantime, can you tell me which field of the transaction metadata contained the RAW_POST_DATA? And if I'm understanding the issue correctly, you can also in the meantime set the config option capture_headers to false. In looking at the Rails codebase here, I'm guessing Rails is setting the request body in the header in your case.

estolfo avatar Feb 23 '22 07:02 estolfo

@estolfo the metadata field is http.request.env.RAW_POST_DATA

I added RAW_POST_DATA to the sanitize_field_names list which filters it out for now but having the body there (without sensitive) data is helpful for troubleshooting issues. This metadata is under Errors in apm.

dgoradia avatar Feb 25 '22 04:02 dgoradia

Hi @dgoradia Thank you for the additional information. I now understand what you are looking for. The Ruby agent doesn't allow users to process the field values themselves that are filtered. They are simply replaced with the string [FILTERED]. Some of the other Elastic APM agents do allow a "processor" to be defined that does more sophisticated processing of the value. We can create a feature request if that's something you'd be interested in. In the meantime, you could create an Elasticsearch ingest node processor. The documentation can be found here Let me know if you have any other questions, thanks!

estolfo avatar Mar 01 '22 19:03 estolfo

Hi again @dgoradia, would you mind telling me why rails puts the post body in the header? I'm curious what the reason is for that?

estolfo avatar Mar 01 '22 20:03 estolfo