instancetype: Extend `PreferSpread`

[lyarwood]

/cc @0xFelix /cc @fabiand /cc @vladikr

What this PR does / why we need it:

The preferSpread preferredCPUTopology was introduced with KubeVirt v1.1 and allows for vCPUs provided by an instance type to be spread across guest visible sockets and cores using a configurable ratio that defaults to 2.

This design proposal aims to extend this implementation by introducing new configurables within a preference to control how vCPUs are spread with the goal of allowing vCPUs to also be spread to threads.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #

Special notes for your reviewer:

This supersedes the original Allow custom guest CPU topologies within instancetypes proposal given insight from a few downstream teams around their need for SMT within the guest OS for their workloads. While this proposal doesn't guarantee performant SMT aligned threads within the guest it at least allows us to express that threads will be present through the existing instance type and preference API paradigms.

Checklist

This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR. Approvers are expected to review this list.

• [ ] Design: A design document was considered and is present (link) or not required
• [ ] PR: The PR description is expressive enough and will help future contributors
• [ ] Code: Write code that humans can understand and Keep it simple
• [ ] Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
• [ ] Upgrade: Impact of this change on upgrade flows was considered and addressed if required
• [ ] Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
• [ ] Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
• [ ] Community: Announcement to kubevirt-dev was considered

Release note:

NONE

[kubevirt-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign alonakaplan for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[lyarwood]

@vladikr would you mind approving this as the implementation has already landed?

[aburdenthehand]

/approve

[kubevirt-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aburdenthehand

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [aburdenthehand]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment