Berkeley migration app memory leaks

[dkijania]

When running full migration using mainnet dump from 15.01.2024 I stumble with a memory leaks that prevents me to cleanly perform migration. My machine has 64GB of ram which filled up after ~40k blocks. I don't have any tracing data (please comment if there is some way i could provide such info). My memory observation lead me to conclusion that memory leaks when inserting blocks (other potential part which can leak is managing precomputed blocsk downloaded from bucket).

Each 200 block inserted memory increased by 4-10mb. This is problematic when i want to migrate archive data in one go. Potential workaround would be to split migration into smaller parts. This is doable as berkeley-migration-app has -end-global-slot parameter which can stop migration at one point. However, it has one small impact. All pending blocks will also be migrated, so we need to treat this parameter with care and do not set it above latest canonical block. Other possibility is to introduce yet another berkeley-migration-app parameter (--canonical-only ) which together with --end-global-slot will work as expected migrating only canonical blocks up to last canonical block

Issues related #13714

[dkijania]

Currently issue is fixed but change is to dangerous to merge it to berkeley now