feat(components): add LangWatch evaluator component - New Bundle

[Vigtu]

Langwatch Component

The Langwatch component allows you to evaluate various aspects of language models using Langwatch's evaluation endpoints.

Configuration

• Evaluator Name: Select the evaluator to use. The component automatically fetches available evaluators from Langwatch's API endpoint (/api/evaluations/list)
• API Key: Your Langwatch API key for authentication
• Input: The input text to evaluate
• Output: The output text to evaluate
• Expected Output: The expected output for evaluation
• Contexts: Additional context (comma-separated)
• Timeout: Maximum time allowed for server response (default: 30 seconds)

API Integration

The component interacts with Langwatch's API in two ways:

1. Fetching Evaluators:

   • Endpoint: https://app.langwatch.ai/api/evaluations/list
   • Method: GET
   • Response: List of available evaluators and their configurations

2. Running Evaluations:

   • Endpoint: https://app.langwatch.ai/api/evaluations/{evaluator_name}/evaluate
   • Method: POST
   • Headers:
     • Content-Type: application/json
     • X-Auth-Token: your_api_key
   • Payload:

     {
       "data": {
         "input": "input text",
         "output": "output text",
         "expected_output": "expected text",
         "contexts": ["context1", "context2"]
       },
       "settings": {
         // evaluator-specific settings
       }
     }
     

Example Usage

1. Add the Langwatch component to your flow
2. Configure your API key
3. Select an evaluator (automatically fetched from Langwatch)
4. Provide input/output text
5. Configure any evaluator-specific settings that appear dynamically
6. Connect to other components as needed

Dynamic Configuration

The component dynamically updates its interface based on the selected evaluator:

• Fetches available evaluators on initialization
• Updates input fields based on evaluator requirements
• Supports various input types (text, numbers, booleans, dropdowns)
• Validates required fields automatically

Returns

Returns an evaluation result object containing metrics and analysis from the selected evaluator. The exact structure depends on the chosen evaluator type.

Langwatch Component - Case.json