"Internal error: Couldn't find term_id 25777562 in dict" for very large dicts (>20mio)

[PSeitz]

As reported by a user:

... Since that showed promised, I tried increasing split_num_docs_target to 20 million. However, my index appears to be corrupt and is returning the following error on queries:

internal error: (internal error: tantivy error: Internal error: 'Couldn't find term_id 25777562 in dict'`, split_id: 01J3KRXAEWJ6QP0TXSSW89HR48),...