Look at automating indexing load distribution to speed up indexing/re-indexing.

[WadeBarnes]

When re-indexing the prod environment recently we broke the indexing load down using update_index --start and --end and distributed the processes across a number of pods (16). This greatly speed up the indexing process >8M credentials in ~6 hours. If the process were automated the time could be reduced significantly (an hour or more).

Breaking down the indexing to work on 1 hour intervals worked well for dealing with all of the indexes that were created during an initial data load, which covers the bulk of the data. One month intervals worked well during periods were indexing was done on credentials issued based on BC Registry events.

[WadeBarnes]

Example breakdown: Indexing.txt