lxc
Allow to create multiple internal subnets
Is there an existing issue for this?
- [x] There is no existing issue for this feature
What are you currently unable to do
Incus allows only one internal subnet to be created:
incus network create net1 --type=ovn network=UPLINK ipv4.address=192.168.0.1/24 ipv6.address=2001:db8:1::1/64 ipv4.nat=true ipv6.nat=false
incus launch images:ubuntu/22.04 vm1 --network net1
On the OVN side it will be translate to:
router beea0e3f-1782-4f54-be2e-e3537832c922 (incus-net86-lr)
port incus-net86-lr-lrp-ext
mac: "00:16:3e:6a:39:c7"
networks: ["100.64.1.10/24", "2001:db8::2/64"]
port incus-net86-lr-lrp-int
mac: "00:16:3e:6a:39:c7"
networks: ["192.168.0.1/24", "2001:db8:1::1/64"]
nat 5b4ddf5e-6b49-49b6-8f77-a2b1dab79345
external ip: "100.64.1.10"
logical ip: "192.168.0.0/24"
type: "snat"
The Logical router port incus-net86-lr-lrp-int is attached to a Logical Switch (internal subnet).
What do you think would need to be added
We could have more internal subnets connected to the same Logical router. An user would like to have multiple internal subnets for different puporses on the same Logical Router.
That's certainly technically doable, but it will get pretty weird with DHCP and SLAAC as far as what subnet needs to be used.
I think DHCP is configured on the switch and RA on the router port, so there shouldn't be anything shared between different switches that are attached to the same router.
Okay, so I was thinking of what would happen if we allowed one network to have a comma separate list of subnets in ipv4.address or ipv6.address which would get a bit messy as far as knowing what to use for what.
If the goal is only to reuse the Logical Router and its external port, then we'd probably instead want to have OVN networks with a parent network.
So you'd do something like:
- incus create network my-network --type=ovn
- incus create network my-extra-subnet parent=my-network --type=ovn
This would then use the same logical router but a separate switch and separate subnets.
@tiagonux is that how you were imagining it?
Hi
@stgraber sorry by the delay, I think it would work. We would have something like that in the OVN side:
router beea0e3f-1782-4f54-be2e-e3537832c922 (incus-net86-lr)
port incus-net86-lr-lrp-ext
mac: "00:16:3e:6a:39:c7"
networks: ["100.64.1.10/24", "2001:db8::2/64"]
port incus-net86-lr-lrp-int
mac: "00:16:3e:6a:39:c7"
networks: ["192.168.0.1/24", "2001:db8:1::1/64"]
port incus-net86-lr-lrp-int-child1
mac: "00:16:3e:6a:39:d8"
networks: ["192.168.1.1/24", "2001:db8:2::1/64"]
Regards,
Tiago Pires
This sounds like a very interesting feature.
If I understood correctly, this would allow multiple OVN networks to share the same LR and therefore the same external IP address for egress traffic. This would be very useful in small to medium sized deployments in which we have a limited number of public IPv4 addresses.
I generally like the approach of adding support for setting an OVN network as a parent of another OVN network to share the LR, but I'm not sure about what the desired behavior would be in some cases.
Let's say I create an OVN network with an associated LR and then create another network using the first one as a parent:
$ incus create network net1 --type=ovn parent=UPLINK
$ incus create network net2 --type=ovn parent=net1
If I then want to delete net1, would I be able to do this or the deletion would fail due to the LR being used by net2?
$ incus network ls
+----------+----------+---------+-----------------+---------------------------+-----------------+---------+---------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY | STATE |
+----------+----------+---------+-----------------+---------------------------+-----------------+---------+---------+
| UPLINK | physical | YES | | | | 1 | CREATED |
+----------+----------+---------+-----------------+---------------------------+-----------------+---------+---------+
| net1 | ovn | YES | 10.215.193.1/24 | fd42:25f7:8e73:de8c::1/64 | | 1 | CREATED |
+----------+----------+---------+-----------------+---------------------------+-----------------+---------+---------+
| net2 | ovn | YES | 10.215.193.2/24 | fd42:25f7:8e73:de8c::2/64 | | 0 | CREATED |
+----------+----------+---------+-----------------+---------------------------+-----------------+---------+---------+
$ incus network rm net1
// What should happen here?
At first glance, it makes sense that this operation would fail, as net1 is, at least from Incus's perspective, being used by net2. But from OVN's perspective, is there really a dependency between them?
I'm not sure if it's possible, but in this case we could allow the deletion of net1 and its LS and keep the LR because it's being used by net2's LS. I'm not a fan of that approach, as it seems complex for both the implementation and for users to reason about.
Is it possible to create an OVN network with just an LR and no LS associated? If that's possible, we might have an ovn-provider network that just creates the LR, and then we can create any number of OVN networks using parent=ovn-provider. That way, we don't create this hard dependency with a single subnet. In this approach, we would have five types of OVN networks:
- With an LS and LR:
parent=UPLINK - With no LS nor LR:
parent=none ipv4.address=none ipv6.address=none(see https://github.com/lxc/incus/issues/1529) - With an LS and no LR:
parent=none - With an LS and using an existing LR:
parent=another-ovn-network(as proposed in this issue) - With no LS but an LR:
parent=UPLINK ipv4.address=none ipv6.address=none(what I'm proposing, with the purpose of being the parent of other OVN networks)
I'm not sure if this makes any sense or if my assumptions are correct, but I think it may be interesting to have the ability to create an OVN network whose sole purpose is to offer an LR for other OVN networks to attach, at least from an organization perspective.
