apache
With VR + VNF + L2 Network, the VMs in L2 Network cannot be reached from the Internet (But the opposite is possible)
ISSUE TYPE
- Bug Report
COMPONENT NAME
VR + VNF
CLOUDSTACK VERSION
4.19.1
CONFIGURATION
Advanced Networking VPC VNF + L2 Network
OS / ENVIRONMENT
SUMMARY
We have 1 VPC and 1 L2 Network connected together with a VNF (PFsense) in between.
Results Summary:
- Test 1: From L2 VM (10.26.8.230) to VPC VM (10.26.1.250) - PASS
- Test 2: From VPC VM (10.26.1.250) to L2 VM (10.26.8.230) - FAILED
- Test 3: From L2 VM (10.26.8.230) to Private Gateway VM (10.88.5.82) - PASS
- Test 4: From Private Gateway VM (10.88.5.82) to L2 VM (10.26.8.230) - FAILED
Background: We’ve allowed any on the VNF firewall rules as attached in the screenshot below:
Traffic is allowed any on 10.26.1.254 interface
Traffic is allowed any on 10.26.8.254
We then run some tests.
Test 1: From L2 VM (10.26.8.230) to VPC VM (10.26.1.250)
Results: Ping and traceroute test from 10.26.8.230 to 10.26.1.250 works as expected. (PASS)
Test 2: From VPC VM (10.26.1.250) to L2 VM (10.26.8.230)
Results: Ping and traceroute test from 10.26.1.250 to 10.26.8.230 is not possible (FAILED)
Test 3: From L2 VM (10.26.8.230) to Private Gateway VM (10.88.5.82)
Results: Ping and traceroute test from 10.26.8.230 to 10.88.5.82 works as expected (PASS)
Test 4: From Private Gateway VM (10.88.5.82) to L2 VM (10.26.8.230)
Results: Ping and traceroute test from 10.88.5.82 to 10.26.8.230 does not work (FAILED)
We've been trying to debug Tests 2 and 4 for a while now with no sucess. Anyone has any ideas? Or is Cloudstack designed not to allow this to be possible?
STEPS TO REPRODUCE
NA
EXPECTED RESULTS
Tests 2 and 4 should be able to work
ACTUAL RESULTS
Tests 2 and 4 does not work
@btzq it looks like a route to 10.26.8.0/254 is missing in the VPC VR. you can retry aftering adding a route manually.
ip route add 10.26.8.0/24 via 10.26.1.254
Hey @weizhouapache , but this would require operator to manually intervene, right? And if the router is destroyed (eg. CleanUp), this setting will dissapear, and it will need to be re-added manually.
Is there another way to do it?
Hey @weizhouapache , but this would require operator to manually intervene, right? And if the router is destroyed (eg. CleanUp), this setting will dissapear, and it will need to be re-added manually.
@btzq correct.
Is there another way to do it?
no. currently static routes only applies on VPC private gateway. https://cloudstack.apache.org/api/apidocs-4.19/apis/createStaticRoute.html
We have plan to extend it to support VPC and isolated networks. probably next major release (4.21).
@weizhouapache i see, looking forward to this enhancement then!
Should i close this ticket? Or leave it open to track this request?
@weizhouapache i see, looking forward to this enhancement then!
Should i close this ticket? Or leave it open to track this request?
you can keep it open.
btw: have you tested the manual static route ? does it work in your case ?
@weizhouapache We tried your solution and it did not work.
After we've added the route, the VR is able to reach the destination, but source to destination still wouldnt work.
@weizhouapache We tried your solution and it did not work.
After we've added the route, the VR is able to reach the destination, but source to destination still wouldnt work.
What are the network acl rules of the vpc tier ? Allow all ?
Hi @weizhouapache , apologies ive missed your comment. Adding the manual static route still didnt work. We arent sure why... ACL Rules were Allow All to make sure ACL is not the issue, but it still didnt work...
Also, we had some growing requests from users where they want to implement a 'Hub and Spoke' architecture such as:
- https://cloud.google.com/architecture/deploy-hub-spoke-vpc-network-topology
- https://learn.microsoft.com/en-us/azure/architecture/networking/architecture/hub-spoke
And the 'Hub', needs to have some form of a VM Firewall (FortinetVMFW, PfsenseVMFW) to be the first point of contact to filter any incoming/outgoing traffic. To my knowledege, this is best done using L2 networks and not possible in VPC/Isolated Networks because Static Route is not supported in this network offering. But, our users want to have VPC/Isolated Network as the 'Hub' as they dont want to use L2 Networks.
I guess its what i have reported in this ticket. I really hope 4.21 resolves this at it would really help!!
Btw, is it possible to 'Force' a spoke VPC to use the Hub for its default gateway? (Eg. Internet access) using Static Routes on VPCs when it does become available in 4.21?
@btzq any update on this ?
the static route feature is available in ACS 4.21 (see #10458)
@weizhouapache yes this is still something we need.
Static Routes sounds like it would solve the problem for us. Ive looked at the PR, does this include a GUI for admins to set the static route onto the Virtual Router?
@weizhouapache yes this is still something we need.
Static Routes sounds like it would solve the problem for us. Ive looked at the PR, does this include a GUI for admins to set the static route onto the Virtual Router?
yes @btzq
please note the next hop should be a private ip in the VPC cidr or a public ip in the same range as VR