Sensitive Data Redaction

[svrnm]

This OTEP proposes changes that will enable sensitive data redaction. This is a follow up to https://github.com/open-telemetry/semantic-conventions/pull/971 and https://github.com/open-telemetry/semantic-conventions/pull/961 and the SemConv spec meeting (2024-04-29)

[svrnm]

I reworked the document to address (most) of the feedback, please take another look, thanks!

[svrnm]

Overall want to give a HUGE THANK YOU for making a dent in this problem and throwing a proposal out there.

Happy to get this conversation started, a big thank you from me to everyone who provided their feedback so far.

I have a lot of comments. I wish I had more time to give viable solutions to my comments, but I do not. (I also lack the time to be brief, so this is long winded).

I appreciate your feedback, a lot of things I can take and try to provide a solution for.

This has the bones of what I think we want to build.

* A configurable set of redaction rules

* A mechanism to annotate attributes with meta-infromation about sensitivity/security

* A simple user knob of "ON/OFF/my-custom-thing" for important use cases.

💯

The details need a bit more fleshing out in particular:

* The YAML model (and OTEL API) for annotating attributes with baseline sensitivity/redaction controls.  You may need MULTIPLE redaction methods for each level of redaction you support, e.g.

* Stricter lines / definitions (less possible bikeshed) on Sensitivity levels.

* More details on the interaction with the SDK, in particular is it the same interface for all three signals or different?

* More granular model (Span name redaction, e.g.)

I'll take another look into all feedback and try to evolve the document with it.

Overall a great start. Let me know if there's anything I can do to help push areas of this forward or flesh out any of your ideas.

Thanks!

[svrnm]

Thank you all for the great feedback. I was not sure what a good practice is, but I reset the PR to draft because I am in the process of incorporating all the feedback and coming back with a significant change.

[svrnm]

https://github.com/open-telemetry/semantic-conventions/pull/971#issuecomment-2437014611

I tried to rework it but there are so many moving pieces that I couldn't get a good handle of it, while also running into some bandwidth issues on my end, since other things required my attention. I still want to follow up on this, but it will take much more time than anticipated.

[pellared]

Related to https://github.com/open-telemetry/semantic-conventions/issues/128

[carlosalberto]

OTEPs have been moved to the Specification repository. Please consider re-opening this PR against the new location. Closing.