Fix review schema strictness and update test fixtures

[hoshinotsuyoshi]

trafficstars

This commit refines the review JSON schema by:

• Replacing object with strictObject from valibot to enforce additionalProperties: false in the root object as well as in nested objects for issues and scores.
• Removing the min/max constraints on the scores' value to simplify the validation.
• Updating test fixtures to match the new output structure by eliminating the need for JSON.parse in javascript assertions.
• Adjusting the configuration for the gpt-4o-mini response format to correctly wrap the JSON schema under json_schema.

Issue

• resolve:

Why is this change needed?

What would you like reviewers to focus on?

Testing Verification

What was done

pr_agent:summary

Detailed Changes

pr_agent:walkthrough

Additional Notes

[changeset-bot[bot]]

⚠️ No Changeset found

Latest commit: d25d1d1b4e72a9b688ec07aa80194bd07e45e2ac

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
liam-app	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 7, 2025 0:04am
liam-erd-sample	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 7, 2025 0:04am

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
liam-docs	⬜️ Ignored (Inspect)	Visit Preview		Apr 7, 2025 0:04am

[supabase[bot]]

Updates to Preview Branch (fix-promptfoo-json-schema) ↗︎

Deployments	Status	Updated
Database	✅	Mon, 07 Apr 2025 00:02:35 UTC
Services	✅	Mon, 07 Apr 2025 00:02:35 UTC
APIs	✅	Mon, 07 Apr 2025 00:02:35 UTC

Tasks are run on every commit but only new migration files are pushed. Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks	Status	Updated
Configurations	✅	Mon, 07 Apr 2025 00:02:36 UTC
Migrations	✅	Mon, 07 Apr 2025 00:02:37 UTC
Seeding	✅	Mon, 07 Apr 2025 00:02:37 UTC
Edge Functions	✅	Mon, 07 Apr 2025 00:02:37 UTC

---

View logs for this Workflow Run ↗︎. Learn more about Supabase for Git ↗︎.

[github-actions[bot]]

frontend/packages/prompt-test result:

View results: https://app.promptfoo.dev/eval/f:6bcbb77b-9822-4112-9de6-76b5f12335a2

✅️ Promptfoo test succeeded

✅️ Successes	❌️ Failures	⚠️ Errors
6	0	0

[qodo-merge-for-open-source[bot]]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Schema Validation The removal of min/max value constraints for scores could allow invalid values outside the expected range. Consider if this was intentional or if validation should be handled elsewhere. value: number(), reason: string(), Configuration Structure Verify that the new response_format structure with nested json_schema is correct according to the OpenAI API specifications for gpt-4o-mini. response_format: { type: 'json_schema', json_schema: { name: 'review', schema: reviewJsonSchema, strict: true, }, },

[qodo-merge-for-open-source[bot]]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Restore score range validation The PR removes the validation for score values (previously enforced range of 0-10). Consider adding back validation for the score range using the number function's range parameter to maintain data integrity. frontend/packages/jobs/src/prompts/generateReview/reviewSchema.ts [17-40] export const reviewSchema = strictObject({ bodyMarkdown: string(), issues: array( strictObject({ kind: KindEnum, severity: SeverityEnum, description: string(), suggestion: string(), suggestionSnippets: array( strictObject({ filename: string(), snippet: string(), }), ), }), ), scores: array( strictObject({ kind: KindEnum, - value: number(), + value: number([0, 10]), // Add validation for score range reason: string(), }), ), summary: string(), [ ] Apply this suggestion Suggestion importance[1-10]: 8 __ Why: The PR removed the validation constraints (minValue(0), maxValue(10)) for the score values, which could lead to invalid data. The suggestion correctly identifies this issue and proposes adding back validation using number([0, 10]), which is important for maintaining data integrity.	Medium
[ ] More