materialize
materialize copied to clipboard
[webhook] Support `ALTER ... CHECK`
Feature
Support altering the CHECK
statement of a webhook source.
Getting the check/validation expression for a webhook source correct is difficult. If we allowed altering the CHECK
statement of a webhook source after creation that would allow a flow of:
- Create source
- Send some sample data
- Experiment with the check statement via
SELECT
-ing from the source - Adding a
CHECK
statement viaALTER ... CHECK
This is a workflow that the Console team is currently pursuing via a "webhook wizard".
It also solves for the case where the check logic of a webhook changes and the user doesn't want to drop their old source.
Implementation
This should be <1 week of work. We already have a Coordinator message GetWebhook
which will lookup information for the webhook from the Catalog, which includes the validate_using
expression. There are two things that will be tricky:
- When a webhook source changes we mark that it should be restarted, this will tell the HTTP layer/Adapter clients (which have a webhook cache) that they should re-fetch metadata for this source.
- We allow aliasing and mapping the body and headers of the request for the check expression. We need to make sure when altering the check expression the Row we create for evaluation is correct.