semantic-conventions icon indicating copy to clipboard operation
semantic-conventions copied to clipboard

Proposal for GenAI Security fields

Open susan-shu-c opened this issue 1 year ago • 3 comments

Related to: https://github.com/open-telemetry/semantic-conventions/issues/1034

Changes

Proposing the following fields that enable security detections to protect LLMs from attacks, misuse (violations, compliance etc.). For example these detection rules.

gen_ai.policy.name	
gen_ai.policy.action
gen_ai.policy.confidence
gen_ai.compliance.violation_detected
gen_ai.compliance.violation_code
gen_ai.performance.request_size

Merge requirement checklist

susan-shu-c avatar Aug 14 '24 14:08 susan-shu-c

CLA Signed

The committers listed above are authorized under a signed CLA.

  • :white_check_mark: login: susan-shu-c / name: Susan (a67366460be50199639a2089ed4c0e9c53d33eb7)

Notes

Some notes from 08-14-2024 working group regarding this proposal:

  • [Liudmila] Should potentially re-use these attributes:
    • https://github.com/open-telemetry/semantic-conventions/pull/903
    • Try to merge with non-genai fields when possible
    • E.g. policy – category can be db, gen_ai, etc.
  • (Advice to look at ECS - https://www.elastic.co/guide/en/ecs/current/index.html)

susan-shu-c avatar Aug 14 '24 18:08 susan-shu-c

I think this may be related to https://github.com/open-telemetry/semantic-conventions/pull/903/ cc @trisch-me

jsuereth avatar Aug 15 '24 13:08 jsuereth

Thanks, all! Now chatting with @trisch-me about what I can use from her PR #903, specifically some of those in security_rule

susan-shu-c avatar Aug 19 '24 18:08 susan-shu-c

This PR was marked stale due to lack of activity. It will be closed in 7 days.

github-actions[bot] avatar Sep 04 '24 03:09 github-actions[bot]

I'm going to close this for now -- makes more sense to use what's in #903 If needed, I will reopen or create a new branch.

susan-shu-c avatar Sep 11 '24 16:09 susan-shu-c