estuary
estuary copied to clipboard
Autoretrieve Stats - Data Ingestion
Modify Autoretrieve to collect an event stream of all retrievals, and write it to our retrieval event database.
This should be possible by implementing a Lassie RetrievalSubscriber in Autoretrieve, and then wiring it up so that every time a retrieval event fires we will save the data to the database.
This will depend on getting our database/schema figured out ref: https://github.com/application-research/estuary/issues/873
Updated task description:
-
Add an endpoint to
estuary-metrics
to be POST'ed to with retrieval stats. It will follow the struct defined here: https://github.dev/filecoin-project/lassie/blob/main/pkg/eventrecorder/eventrecorder.go#L63 -
Configure autoretrieve to use Lassie Event Publisher to publish events to estuary metrics API (it is configured by this line here) https://github.dev/application-research/autoretrieve/blob/master/autoretrieve.go#L197
estuary-metrics
task: https://github.com/application-research/estuary-metrics/issues/18