NeMo-Guardrails icon indicating copy to clipboard operation
NeMo-Guardrails copied to clipboard

feature: Clavata integration

Open ilias-t opened this issue 9 months ago • 1 comments

Did you check the docs?

  • [x] I have read all the NeMo-Guardrails docs

Is your feature request related to a problem? Please describe.

Add support for Clavata that provides content moderation capabilities to detect and filter inappropriate content.

Describe the solution you'd like

A user can customize the content moderation behavior by:

  1. Configuring different policies for input and output flows
  2. Specifying which labels must match within a policy
  3. Setting the label match logic to either "ALL" (all specified labels must match) or "ANY" (at least one label must match)

Example config

rails:
  config:
    clavata:
      # Only provide this if you've been told to by Clavata.ai
      server_endpoint: "https://some-alt-endpoint.com"
      policies:
        - alias: "Violence"
          id: "00000000-0000-0000-0000-000000000000"
        - alias: "Weapons"
          id: "00000000-0000-0000-0000-000000000000"
      input:
        policy: "Violence"
        # Optional: Specify labels to require specific matches
        labels:
          - "Violence"
          - "Weapons"
          - "Drugs"
        label_match_logic: ALL  # Can be "ALL" or "ANY"
      output:
        policy: "Weapons"
  input:
    flows:
      - clavata check input
  output:
    flows:
      - clavata check output

Details

  • server_endpoint: The Clavata API endpoint (only if provided by Clavata.ai)
  • policies: List of policy configurations with aliases and IDs
  • input/output: Flow-specific configurations
    • policy: The policy alias to use for this flow
    • labels: (Optional) List of specific labels to check for
    • label_match_logic: (Optional) "ALL" requires all specified labels to match, "ANY" requires at least one match

Describe alternatives you've considered

N/A

Additional context

No response

ilias-t avatar Mar 05 '25 00:03 ilias-t

@Pouyanpi the PR was merged to develop on 4/24 - is it scheduled for release?

ashwini avatar Jun 23 '25 20:06 ashwini