Introduce gp_appendonly_compaction_threshold

[soumyadeep2007]

trafficstars

This commit introduces the gp_appendonly_compaction_threshold GUC, which ensures the availability of N segfiles for insert operations, before going forward with a compaction decision.

During VACUUM, we decide to drop a segfile in AWAITING_DROP iff there is no older snapshot in the system that can see the segfile. This can lead to exhaustion of segfiles for insert purposes post VACUUM (since segfiles AWAITING_DROP aren't eligible for inserts). Currently, once a segfile is chosen as a compaction "target", it is destined to go into AWAITING_DROP. So, we install the GUC guard when we are deciding on the next segfile to compact. If we see that the number of eligible segments for insert is below the threshold, we are done with compaction - for all remaining segfiles.

uao/max_concurrency2.source has been modified to demonstrate the safeguard.

Co-authored-by: Ashwin Agrawal [email protected]

Dev-pipeline: https://dev.ci.gpdb.pivotal.io/teams/main/pipelines/gp_compaction_segfle_limit