curb-data-specification icon indicating copy to clipboard operation
curb-data-specification copied to clipboard

Add `start_time` and `end_time` as query parameters to the Events API

Open michael-danko-passport opened this issue 3 years ago • 2 comments

Is your feature request related to a problem? Please describe.

One of the issues outlined within #99 is that the Events API requires API consumers to deduplicate the events received and compare against already ingested data. This may become increasingly complicated as the volume of events within a provider's system grows.

Describe the solution you'd like

Add additional query parameters to the API specification that will enable an API consumer to designate the time window of when they would like to search for events.

Is this a breaking change

This could be a breaking change if this was made as a query parameter that was required to be implemented.

Impacted Spec

  • Events

Additional context

If this was not implemented as a required parameter, there may be technical concerns about source systems not being able to respond with the necessary data in an open ended query.

This may help alleviate some, but not all, of the concerns raised in #99.

michael-danko-passport avatar Jul 26 '22 20:07 michael-danko-passport

Another option that I think would work pretty well would be to use the event_time=YYYY-MM-DDTHH convention that MDS uses to get an hour's worth of data at a time.

jiffyclub avatar Aug 20 '24 21:08 jiffyclub

+1 for this, sometimes it's essential to do some sort of limitation to the consumers, or assist them in picking up events for a certain date they would like to, instead of having them do guesswork about which page they should pick up to compensate for a certain date. This combined with pagination should be very helpful for consumers, as well as certain implementation models that benefit from having a date range filter for better performance.

I'd also like to bring up that pagination is not a requirement and in those cases asking for events will return a whole dataset which not only would be impactful on the provider, but also hit the consumer with a huge JSON response.

ArchaicDebugger avatar Oct 10 '24 10:10 ArchaicDebugger