great_expectations icon indicating copy to clipboard operation
great_expectations copied to clipboard

Enforce consistent ordering of great_expectations.yaml

Open jimmycfa opened this issue 2 years ago • 3 comments

Is your feature request related to a problem? Please describe. When adding new expectations the order of the entries in datasources will change. This can make Git commits for that file appear as though a lot of things have changed, when maybe only a single expectation was added.

Describe the solution you'd like Use an OrderedDict to enforce consistently with the ordering in the yaml generated.

Describe alternatives you've considered None

Additional context We are using Great Expectations 0.14

jimmycfa avatar Apr 21 '22 13:04 jimmycfa

Potential solution would be to update this function to return and use an OrderedDict vs Dict for setup and return: https://github.com/great-expectations/great_expectations/blob/c59690d933ef4c54d582beb141fb7b246cc25601/great_expectations/data_context/data_context.py#L1897

jimmycfa avatar Apr 21 '22 14:04 jimmycfa

Hey @jimmycfa ! Thanks for surfacing this; we have this on docket for review in the near future.

austiezr avatar Aug 08 '22 16:08 austiezr

Hey @jimmycfa after reviewing this issue with the team, it was determined that the issue is caused by an upstream dependency, namely ruamel. They have not yet addressed the issue and there is no clear sign of when they will do so, unfortunately. Changing that function return type won't have any effect since the data has changed the order in the underlying library. We totally hear your use case, but currently don't have a feasible solution on our end.

rdodev avatar Aug 29 '22 16:08 rdodev

Hey @jimmycfa after reviewing this issue with the team, it was determined that the issue is caused by an upstream dependency, namely ruamel. They have not yet addressed the issue and there is no clear sign of when they will do so, unfortunately. Changing that function return type won't have any effect since the data has changed the order in the underlying library. We totally hear your use case, but currently don't have a feasible solution on our end.

Any link to that ruamel issue?

dlamblin avatar Mar 24 '23 22:03 dlamblin