dataform icon indicating copy to clipboard operation
dataform copied to clipboard

Improve error messages when workflow settings have non-string values for default project or dataset

Open kolina opened this issue 1 year ago • 2 comments

Currently if defautProject or defaultDataset have an integer type in workflow_settings.yaml, a compilation will throw a not very obvious error like t.target.schema.includes is not a function. I've added a validation of these fields to be strings (if not, an explicit error is thrown).

kolina avatar Sep 14 '24 23:09 kolina

  • We should be catching this during the proto validation https://github.com/dataform-co/dataform/blob/05341b7c2e8f6e55a7002ed29b4ff4a7e2cbf12b/common/protos/index.ts#L53

I also hoped that it should have been the case, but unfortunately this code https://github.com/dataform-co/dataform/blob/05341b7c2e8f6e55a7002ed29b4ff4a7e2cbf12b/common/protos/index.ts#L45-L46 doesn't validate types and actually allows to create protoCastObject where defaultDataset has an integer type and this check later doesn't work https://github.com/dataform-co/dataform/blob/05341b7c2e8f6e55a7002ed29b4ff4a7e2cbf12b/common/protos/index.ts#L53

The thing that actually verifies types is protoType.verify(). WDYT about using it?

kolina avatar Sep 16 '24 22:09 kolina

.verify() is a lie, it doesn't actually do anything - there are no situations where the call will actually fail.

Ekrekr avatar Nov 11 '24 16:11 Ekrekr