crlite icon indicating copy to clipboard operation
crlite copied to clipboard

Speed up ingestion of CT log data

Open jschanck opened this issue 4 years ago • 0 comments

CT logs rate-limit our get-entries requests, and this limits how quickly we can "reboot" CRLite when we make a change that requires us to re-ingest CT log data. This issue tracks our progress on speeding up the ingestion of CT log data.

We're currently planning three changes:

  1. Use consistency proofs to verify that entries we downloaded (potentially from a third party) match the entries in a CT log.
  2. Download entries from CT logs in most-recent-first order. The idea here is that we can get entries {0, 1, ..., k} from a third party, so we should spend our get-entries requests on new entries.
  3. Check for log entries in a local disk cache before making a get-entries request.

jschanck avatar Oct 18 '21 20:10 jschanck