Nicolas Williams
Nicolas Williams
To switch over, should happen over a 2 day period: ### Day 1 - Perform Snapshot ingestion (until complete) - Start the PSC Stream Ingester app (which should run continuously...
Instead of a monthly job run on an EC2 instance, this will need to run continuously, so that new records through the PSC Stream API are detected and ingested immediately....
Since it should be running continuously, it is important to get notifications of when it has crashed. It ought to be recovering if it is a brief crash, since the...
Heroku deploys for review apps appear to be authenticating using @spacesnottabs's account, reporting that user requested a deploy, even if they didn't. This should be reviewed to ensure that any...
The stream ingester hasn’t been run or updated since the first prototype. Some work will be required to bring it up to date with how the other apps behave. _Estimate:...
When a record is obtained through the PSC snapshots, it is in the form: ``` { company_number: "12345678", data: {} } ``` However, when read through the PSC Stream API,...
Since the records in the stream do not last forever, it is possible that a timepoint may be used from storage which is invalid, due to age. In this case,...
When using the PSC Stream API, it requires either: - A `timepoint` pointer of the last record seen through the API, which will then return records seen since here -...
For the prototype, a personal email was used to signup for a PSC Stream key for testing purposes. However, it will be necessary to have one managed by Open Ownership...
The Athena database setting, environment variable `ATHENA_DATABASE`, is ignored, at least in Ingestor OC. This has resulted in the `default` database being written to, despite it being set in production....