agenta icon indicating copy to clipboard operation
agenta copied to clipboard

[AGE-2612] [Bug] Test Set Upload Accepts Objects But Fails During Evaluation

Open mmabrouk opened this issue 7 months ago • 0 comments

Summary: Test set upload accepts JSON files with object fields without validation, but evaluation fails with 500 error when using these test sets.

Envs affected:

  • OSS
  • Stage
  • Beta
  • Prod
  • other / local branch

Interface Affected

  • web
  • api

Functionalities Affected

  • Testsets
  • Evals

Current Behavior:

  1. User uploads a JSON test set containing fields with object values
  2. Upload succeeds without any validation error or warning
  3. When attempting to run an evaluation using this test set, the system returns a 500 server error
  4. No clear error message indicates that object fields are not supported

Expected Behavior:

  1. Test set upload should validate that all fields contain only string values
  2. If objects are detected, upload should fail with a clear error message explaining the requirement
  3. Alternatively, the system should handle object fields during evaluation without crashing

Steps To Reproduce:

  1. Create a JSON test set file with at least one field containing object values (not strings)
  2. Upload the test set through the web interface
  3. Navigate to create a human evaluation
  4. Select the uploaded test set
  5. Attempt to run the evaluation

Console / Container Logs: Server returns 500 error during evaluation execution

From SyncLinear.com | AGE-2612

mmabrouk avatar May 29 '25 17:05 mmabrouk