haystack icon indicating copy to clipboard operation
haystack copied to clipboard

Provide relevant schema of pipeline serialization

Open lohit8846 opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. It's great that this library supports serializing the pipelines into a human editable format as that's a very useful feature that other packages do not have. However, I see an issue with the provided format

There is no defined schema for what fields each component needs to have set. This is important because otherwise a user purely editing the YAML will not know what fields are supported and we can't introduce any validation. I am also planning to use JSON format instead with a custom marshaller and provide a JSON schema which can be edited as a GUI since there are plenty of JSON Schema GUI editors other there. However, this requires me to manually inspect each component and come up with a JSON schema which would be a nightmare in terms of maintenance when components change

Describe the solution you'd like

I would like to propose that the haystack framework provide a way to generate a JSON schema for all the components that basically captures that input format which will allow us to validate and build on top. Is this possible in the current design?

lohit8846 avatar Mar 30 '24 20:03 lohit8846