Add patronus eval tool docs

[Rebecca-Qian]

Adding docs for the following:

• PatronusEvalTool
• PatronusLocalEvaluatorTool
• PatronusPredefinedCriteriaEvalTool

[joaomdmoura]

Disclaimer: This review was made by a crew of AI Agents.

Code Review Comment for Patronus Evaluator Documentation

Overview

This pull request adds documentation for three important tools in the Patronus framework:

• PatronusEvalTool
• PatronusLocalEvaluatorTool
• PatronusPredefinedCriteriaEvalTool

While the documentation generally provides clear and useful information, several areas require improvement for consistency, clarity, and usability.

Recommendations

1. Standardize Frontmatter Formatting

The frontmatter should be consistent across all documentation files. For example, ensure that all headers start with the specific YAML formatting. Here’s a corrected example:

- 
---
title: Patronus Eval Tool
+ ---
+ title: Patronus Eval Tool

2. Consistent Title Formatting

Titles must be uniformly formatted to enhance clarity in navigation. For example, the title in the patronuslocalevaluatortool.mdx should align with the others:

- title: Patronus Eval Tool
+ title: Patronus Predefined Criteria Eval Tool

3. Uniform Link Formatting

Ensure all links follow a consistent format, particularly for URLs:

- app.patronus.ai](http://app.patronus.ai)
+ [app.patronus.ai](https://app.patronus.ai)

4. Improve Numbered List Formatting and Clarity

Lists should maintain a clear structure without unintended spaces:

- 1. Fetch all available criteria
- 2. Select the most fitting criteria
- 3. Evaluates the inputs/outputs/context

+ 1. Fetch all available criteria
+ 2. Select the most fitting criteria 
+ 3. Evaluate the inputs/outputs/context

5. Use Language-Specific Code Block Tags

Specify the language in code snippets for improved clarity and syntax highlighting. For instance:

- ```shell
+ ```bash

6. Code Comments in Examples

Adding comments in code snippets can significantly enhance understanding, especially for new users. Here’s an added comment example:

patronus_eval_tool = PatronusLocalEvaluatorTool(
    evaluator="local_evaluator_name",  # Name of the registered evaluator function
    evaluated_model_gold_answer="test"  # Expected correct answer for comparison
)

7. Clear Configuration Example

Improve clarity in configuration examples to avoid confusion:

patronus_eval_tool = PatronusPredefinedCriteriaEvalTool(
    evaluators=[{
        "evaluator": "judge",     # Built-in Patronus evaluator
        "criteria": "contains-code" # Pre-defined evaluation criteria
    }]
)

8. Additional Recommendations

• Add Links to API Reference Documentation: This will facilitate users in exploring more about the functions and their applications.
• Introduce Troubleshooting Sections: Allow users to resolve common issues quickly.
• Standardize Heading Capitalization: Ensure titles have consistent capitalization styles throughout the documentation.
• Add Parameter Type Information in Code Examples: This will improve clarity for users unfamiliar with the tool's implementation.

Positive Aspects

• Well-structured overall documentation.
• Comprehensive coverage detailing each tool's capabilities and installation instructions.
• Helpful code examples lead to better understanding and implementation.

Conclusion

The documentation lays a strong foundation for using the Patronus evaluator tools but can greatly benefit from these adjustments. By incorporating these suggestions, we will enhance the quality, clarity, and maintainability of the documentation, ultimately improving the user experience.

Thank you for your efforts to improve the Patronus documentation! I look forward to seeing these enhancements implemented.

[bhancockio]

Hey @Rebecca-Qian! Can you please merge in main and shoot me a message when you do so I can get this merged in?

Thank you for updating these docs!

[bhancockio]

Hey @Rebecca-Qian! Can you please update this PR to allow edits by maintainer so I can update this branch?

Steps for the Contributor: Go to the Pull Request page in the GitHub repository. On the right-hand sidebar (near the bottom), they should see a checkbox labeled "Allow edits by maintainers" under the "Reviewer" and "Assignees" sections. Check the box to allow maintainers to push changes to their PR branch.

[bhancockio]

Closing due to inactivity.