Nicolas Williams
Nicolas Williams
Day 2 snapshot ingestion ingestion cannot be run until tomorrow (despite being actually Day 3 in this case), since it's necessary for today's data to appear in the PSC bulk...
Airbrake fired overnight, notifying of a crash. Investigation showed that something went wrong, seemingly within Heroku, and the app eventually got relocated to another server: ``` Apr 24 01:51:03 register-ingester-psc-prd...
Day 2 snapshot ingestion has started (although it is the 3rd day).
Day 2 snapshot ingestion has completed successfully.
Similar to https://github.com/openownership/register/issues/243 , the approach I've chosen has been to log, clear the invalid data from Redis, and reraise. This will crash the app, but once it restarts, consuming...
The existing code works for consuming from multiple Kinesis shards. However, the manner by which it does that isn't optimal: - fetch list of shards - for each shard in...
### Kinesis Quotas and Limits **Data stream throughput** _Provisioned mode_ > There is no upper limit. Maximum throughput depends on the number of shards provisioned for the stream. Each shard...
In keeping with recent work on other parts of the program, I'm not writing extra formal tests for this. I am not convinced of the benefit of doing so, especially...
The app has been created on Heroku, and is in a crash loop (intentionally). It will be activated in https://github.com/openownership/register/issues/255 .
Potentially linked to https://github.com/openownership/register/issues/213 . Requires further discussion.