gentropy icon indicating copy to clipboard operation
gentropy copied to clipboard

feat(spark-helpers): enforce schema of returned objects

Open DSuveges opened this issue 9 months ago • 0 comments

There has been a need create of a list of structs in multiple points of the codebase. This process is complicated as requires that the structs have consisted schema, which is complicated if they are generated by different functions. The main function in this PR can be used as a decorator.

🛠 What does this PR implement

  • [x] enforce_schema function that checks and modifies the schema of a returned data of a function
  • [x] Suitable test.
  • [x] More test for order_array_of_structs_by_field

🚦 Before submitting

  • [x] Do these changes cover one single feature (one change at a time)?
  • [x] Did you read the contributor guideline?
  • [x] Did you make sure to update the documentation with your changes?
  • [x] Did you make sure there is no commented out code in this PR?
  • [x] Did you follow conventional commits standards in PR title and commit messages?
  • [x] Did you make sure the branch is up-to-date with the dev branch?
  • [x] Did you write any new necessary tests?
  • [x] Did you make sure the changes pass local tests (make test)?
  • [x] Did you make sure the changes pass pre-commit rules (e.g poetry run pre-commit run --all-files)?

DSuveges avatar May 20 '24 20:05 DSuveges