quinn icon indicating copy to clipboard operation
quinn copied to clipboard

Extend Quinn's Data Validation functionality

Open souvik-databricks opened this issue 1 year ago • 2 comments

Add to Quinn a tool to profile incoming data, check if it adheres to registered schema and do custom data quality checks. At the end of it, a human readable report is autogenerated that can be saved separately.

  • Config driven data profiling and schema validation
  • Autogeneration of report after every run
  • Integration with datadog monitoring system
  • Extensible and highly customizable.
  • Very little boiler plate code.
  • Support for versioned schema validation.

souvik-databricks avatar Feb 02 '24 21:02 souvik-databricks

@MrPowers I will be working on this. Can we get this assigned to me please. Thanks.

souvik-databricks avatar Feb 02 '24 21:02 souvik-databricks

Have at it @souvik-databricks !

MrPowers avatar Feb 02 '24 21:02 MrPowers