DashAI icon indicating copy to clipboard operation
DashAI copied to clipboard

Pydantic-like DataLoader Schemas

Open Riul1999 opened this issue 1 year ago • 1 comments

Summary

New Pydantic Schemas for Dataloaders.

Type of change

  • Back end new feature.
  • Refactoring.

Changes

  • New string_field, for raw strings.
  • Refactor old string_field into enum_field.
  • Refactor models and test_schemas to use enum_field.
  • Add schemas for CSVDataLoader and JSONDataLoader.
  • Add schemas for splits and more options inside BaseDataloader file.
  • Add method to replace the $defs field inside the respective field.

How to Test

Run tests\back\schemas\test_schemas.py to check the correct refactorization. Run tests\back\dataloader\test_dataloader_schema.py to check the generation of the schemas.

Notes

This PR is blocked becuase it's required to change the DataLoader configuration step in the front, especifically the way the splits and advanced configurations are obtained. This will be solved with the changes in refactoring/configurable-objects

Riul1999 avatar Feb 26 '24 16:02 Riul1999