Add webhook support for {docket,search} alerts

[mlissner]

Talked to somebody yesterday that wants to set up webhooks for alerts and wants to create alerts dynamically.

These things ought to be possible, though I don't quite know what the API design would be:

• Do you configure individual webhook endpoints for each alert (probably not)
• Do you configure the webhook when you create the alert (maybe)
• What does the content of the POST we send to the webhook look like?
• What's the API for creating alerts look like?

Etc. This is a bit outside our wheelhouse, but we should figure it out.

For the full picture, the other todo we have around webhooks is allowing people to set up a webhook for when they request a document that we download on their behalf so that we tell them we have it.

[johnhawkinson]

Is there an overview of good webhook design practice somewhere? Are there rich tools for manipulating webhooks? Can we avoid reinventing the wheel(house)?

[mlissner]

I don't know, actually. Most of my experience comes from being a receiver of them for payments (stripe does a good job with them), but I haven't ever read any great references. Research needed, I think.

[albertisfu]

@mlissner this is the JSON object proposed for search alerts webhooks:

{
	"webhook": {
	        "version": 1,
	        "event_type": 2,
	        "date_created": "2022-07-09T02:02:31.304842+00:00"
	},
	"search_alert":{
		"id": alert.pk,
		"date_last_hit": alert.date_last_hit,
		"rate": alert.rate,
		"search_type": result.type, 
		"alert_name": alert.name,
		"hits": results.result.numFound,
	},
	"results":[{
		"url": result.absolute_url
		"case_name": result.solr_highlights.caseName
		"original_download_link": result.download_url
		"cl_download_link": result.local_path
		"highlights": result.solr_highlights.text
	}]
}

Let me know what you think.

These are some questions that popped up to me about search alerts:

• About rate, should show the FREQUENCY ids? e.g: “rt”, dly, etc
• About search_type: Should we show the SEARCH_TYPES ids? like o,oa, etc?
• About alert_name it’ll be something like this: Courts: All › Query: Testing › Precedential: True, that’s ok?
• About highlights, this is the highlighted search text, should we add it to webhooks?

[mlissner]

Thanks for the proposal, Alberto. I created a new issue and put it in your IN PROGRESS column: https://github.com/freelawproject/courtlistener/issues/2161

It has the answers to most of your questions, I think. I think the remaining ones are answered by saying that the data for the results key should be a serialized response from the search API.

[mlissner]

Let's also make sure that we:

• [ ] trim old webhook events via a cronjob

[albertisfu]

Sure, I suppose that we are going to trim webhooks based on their creation date right? So it might be a command similar that we use to delete old sent emails right?

[mlissner]

Seems, right, yep. Could do it with a cron or as part of the webhook event sending pipeline.

On Fri, Jul 29, 2022, 07:48 Alberto Islas @.***> wrote:

Sure, I suppose that we are going to trim webhooks based on their creation date right? So it might be a command similar that we use to delete old sent emails right?

— Reply to this email directly, view it on GitHub https://github.com/freelawproject/courtlistener/issues/854#issuecomment-1199441909, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABZ3KSJ35IWJ4SBRFOJVUTVWPVLLANCNFSM4FKVJ6HQ . You are receiving this because you were mentioned.Message ID: @.***>