materialize icon indicating copy to clipboard operation
materialize copied to clipboard

[webhook] Support `ALTER ... CHECK`

Open ParkMyCar opened this issue 9 months ago • 1 comments

Feature

Support altering the CHECK statement of a webhook source.

Getting the check/validation expression for a webhook source correct is difficult. If we allowed altering the CHECK statement of a webhook source after creation that would allow a flow of:

  1. Create source
  2. Send some sample data
  3. Experiment with the check statement via SELECT-ing from the source
  4. Adding a CHECK statement via ALTER ... CHECK

This is a workflow that the Console team is currently pursuing via a "webhook wizard".

It also solves for the case where the check logic of a webhook changes and the user doesn't want to drop their old source.

Implementation

This should be <1 week of work. We already have a Coordinator message GetWebhook which will lookup information for the webhook from the Catalog, which includes the validate_using expression. There are two things that will be tricky:

  1. When a webhook source changes we mark that it should be restarted, this will tell the HTTP layer/Adapter clients (which have a webhook cache) that they should re-fetch metadata for this source.
  2. We allow aliasing and mapping the body and headers of the request for the check expression. We need to make sure when altering the check expression the Row we create for evaluation is correct.

ParkMyCar avatar May 01 '24 19:05 ParkMyCar