sriov-network-device-plugin
sriov-network-device-plugin copied to clipboard
k8snetworkplumbingwg
Use exact match or regex for pfname selector
@rozeps can you explain the problem you are trying to solve with this PR ?
We are deploying in multi-nodes OpenShift clusters, and we support few vendors SR-IOV for use in our applications (matching on vendor PF is not sufficient here).
We are reserving VFs 0-2 for internal use then 3-7 for end user applications.
As interface name not fully predictive (systems end up with ens5s0f0 or enp65s0f0, and so on).
I am tring to come up with configuration that would match any PF from vendor/device XYZ, take only VFs 3-7.
We are deploying in multi-nodes OpenShift clusters, and we support few vendors SR-IOV for use in our applications (matching on vendor PF is not sufficient here).
We are reserving VFs 0-2 for internal use then 3-7 for end user applications.
As interface name not fully predictive (systems end up with ens5s0f0 or enp65s0f0, and so on).
I am tring to come up with configuration that would match any PF from vendor/device XYZ, take only VFs 3-7.
@rozeps Is the PCI address deterministic across your cluster for the PFs you wish to consume? You can use that instead of PF name. Or use udev rules to make your interfaces names deterministic?
Well apparently yes, but I just looked at few systems that are based on same Xeon family (same socket/numa architecture), just wonder how that may differ in more heterogeneous h/w environment (that customers would be allowed to use).
In my mind, would be much simpler to say selector: "match vendor=0x8086,device=0x158b,pf=/.*/#3-7"
With the re enclosed in special characters (illegal ifname ones) don't think that would break much existing implementations.
I am tring to come up with configuration that would match any PF from vendor/device XYZ, take only VFs 3-7.
So if you would have a "VFIndex" selector it would also satisfy your use-case ?
Sure you mean no PFNames just the VFIndex as list of id that would be perfect (and cleaner I believe).
@rozeps this PR was discussed in today's technical meeting
see: https://docs.google.com/document/d/1bEyEh_K9lGwQWHgUHUH-vqodYd1W-LCWRGlW0-ZVR5A/edit#
We would like to understand more about your use-case and whether or not it is possible to get persistent and predictable net device names.
if it is, then using pfNames while specifying all relevant netdevs with the specific range should work.
e.g all nodes have 4 NICs and the netdevs are named the same across nodes.
if it isnt, then a "VFIndex" selector might do the trick. A regex as you proposed will work but it requires to introduce a special notation for it + ensuring it does not affect existing configuration, so its less favorable.
We cannot predict all the combinations now and future, but on my test systems I have 2 nodes with mix of different cards (Mellanox/Intel) and the PCI addresses are different, and the network device names also. Anticipating all possible PF names might be error prone one day someone deploy node with slightly different config and the PF name is not on the list ... may take while to figure out.
The VFIndex I think would have been a good idea if initially tought (in addition to the PFName) but right now that could conflict right one may have non-overlapping pfname with "#0-2-3" part and VFIndex.
The regex enclosed withih "/" is backward compatible with existing syntax - when not present exact match on PF name is used otherwise the regex logic.
The VFIndex I think would have been a good idea if initially tought (in addition to the PFName) but right now that could conflict right one may have non-overlapping pfname with "#0-2-3" part and VFIndex.
AND is performed between different selectors, so result will be consistent It might not make sense to use the VF range notation and VFIndex selector. there are other combinations that might not make sense.
@zshi-redhat, @martinkennelly any [additional] thoughts on this ?
The VFIndex I think would have been a good idea if initially tought (in addition to the PFName) but right now that could conflict right one may have non-overlapping pfname with "#0-2-3" part and VFIndex.
AND is performed between different selectors, so result will be consistent It might not make sense to use the VF range notation and VFIndex selector. there are other combinations that might not make sense.
@zshi-redhat, @martinkennelly any [additional] thoughts on this ?
Do you want to introduce a match on all PF interface names or do we need a regex for advanced matchings? If it is about matching all PF interface names, perhaps we can just leave the name empty which indicates matching all. If we have advanced matching cases, it looks to me that having a special notation like re{...} would provide more readability to user and less confliction with existing selectors.
@zshi-redhat @adrianchiris @rozeps What's the status of this PR?
I think we didn't get conclusion on which approach to implement or use:
- VF Index selector
- Regex match in PfName selector
- Exhaust all PF names in PfName selector w/ defect that PF names may change in heterogeneous env, no change required
Not sure if it can help to decide, but in our case we'd like to have a selector based on "vendor", "device", "driver", and "vfIndex" the later could have syntax like "pfnames: ["#2", "#6", "#8"] for example i.e. no regex needed only VF number.
I guess one could also want: "pfnames: ["enp66s0f0#1", "#2", "enp65s0f1"] i.e. with both, pfname only or vfIndex only.
