Design for running multiple backups in parallel

[sseago]

Describe the problem/challenge you have

For backups with a large number of items, especially when those items include CSI VolumeSnapshot creation, it can take a long time before the backup moves to WaitingForPluginOperations, so existing backups cannot start. This is a usability concern when there are multiple users creating backups, especially with OADP self-service users who don't have full access to see progress for the current backups (since their backups are created for them via the oadp non-admin controller).

This design will build on the parallel item backup implemented in Velero 1.16, making use of the same worker pool to process items across backups

Describe the solution you'd like

This issue is tracking the design only (implementation will be a follow-up issue). One of the primary issues to work through will be avoiding running backups in parallel that have overlapping contents. Initially this will be done at a high level to simplify things -- we won't allow multiple backups with overlapping namespace inclusion to run at the same time.

Anything else you would like to add:

Environment:

• Velero version (use velero version):
• Kubernetes version (use kubectl version):
• Kubernetes installer & version:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• :+1: for "The project would be better with this feature added"
• :-1: for "This feature will not enhance the project in a meaningful way"