Change FLUSHALL/FLUSHDB SYNC to run as blocking ASYNC

[moticless]

Overview

Users utilize the FLUSHDB SYNC and FLUSHALL SYNC commands for a variety of reasons. The main issue with this command is that if the database becomes substantial in size, the server will be unresponsive for an extended period. Other than freezing application traffic, this may also lead some clients making incorrect judgments about the server's availability. For instance, a watchdog may erroneously decide to terminate the process, resulting in potential adverse outcomes.

While a FLUSH* ASYNC can address these issues, it might not be used for two reasons: firstly, it's not the default, and secondly, in some cases, the client issuing the flush wants to wait for its completion before repopulating the database.

Between the option of triggering FLUSH* asynchronously in the background without indication for completion versus running it synchronously in the foreground by the main thread, there is another more appealing option. We can block the client that requested the flush, execute the flush command in the background, and once done, unblock the client and return notification for completion. This approach ensures the server remains responsive to other clients, and the blocked client receives the expected response only after the flush operation has been successfully carried out.

Implementation details

Instead of defining yet another flavor to the flush command, we can modify FLUSHALL SYNC and FLUSHDB SYNC always run in this new mode.

Extending BIO Threads capabilities

Today jobs that are carried out by BIO threads don't have the capability to indicate completion to the main thread. We can add this infrastructure by having an additional dummy job, coined as completion-job, that eventually will be written by BIO threads to a response-queue. The main thread will take care to consume items from the response-queue and call the provided callback function of each completion-job.

FLUSH* SYNC to run as blocking ASYNC

Command FLUSH* SYNC will be modified to create one or more async jobs to flush DB(s) and afterward will push additional completion-job request. By sending the completion job request only at the end, the main thread will be called back only after all the preceding jobs completed their task in the background. During that time, the client of the command is suspended and marked as BLOCKED_LAZYFREE whereas any other client will be able to communicate with the server without any issue.

[CLAassistant]

All committers have signed the CLA.