Gcloud deployment

[konradkar2]

Hi, I'm trying to deploy the cluster on GCE but I'm having a problem with the connection between UPF's n6 <-> Internet. When I run: root@3:/ueransim/build# ping -I uesimtun0 1.1.1.1 That's traffic I see on a host eth1 interface

15:42:44.885833 ARP, Request who-has 192.168.11.1 tell 192.168.11.101, length 28
15:42:45.906374 ARP, Request who-has 192.168.11.1 tell 192.168.11.101, length 28
15:42:46.929860 ARP, Request who-has 192.168.11.1 tell 192.168.11.101, length 28

This traffic goes directly to the gateway and there is no reply to that

I suspect that google's gateway verifies the source MAC address, and it has to match any VMs related to the project. So UPF N6 MACVLAN interface has a new mac address that is not recognized by google, therefore traffic gets dropped. I've tried to manually add ARP entry on the UPF pod but still, I could only see ping requests on the very same interface, no reply.

Here's my configuration.

Worker node

It hosts all the pods

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1460
      inet 192.168.10.3  netmask 255.255.255.255  broadcast 0.0.0.0
      inet6 fe80::4001:c0ff:fea8:a03  prefixlen 64  scopeid 0x20<link>
      ether 42:01:c0:a8:0a:03  txqueuelen 1000  (Ethernet)
      RX packets 682476  bytes 1501059680 (1.5 GB)
      RX errors 0  dropped 0  overruns 0  frame 0
      TX packets 651967  bytes 72658725 (72.6 MB)
      TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1460
      inet 192.168.11.100  netmask 255.255.255.255  broadcast 0.0.0.0
      inet6 fe80::4001:c0ff:fea8:b64  prefixlen 64  scopeid 0x20<link>
      ether 42:01:c0:a8:0b:64  txqueuelen 1000  (Ethernet)
      RX packets 62885  bytes 5374035 (5.3 MB)
      RX errors 0  dropped 0  overruns 0  frame 0
      TX packets 312444  bytes 30253882 (30.2 MB)
      TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

UPF n6

n6@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc noqueue state UP group default 
  link/ether 0e:73:67:10:eb:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0
  inet 192.168.11.101/24 brd 192.168.11.255 scope global n6
     valid_lft forever preferred_lft forever
  inet6 fe80::c73:67ff:fe10:eb85/64 scope link 
     valid_lft forever preferred_lft forever

I couldn't find that mentioned in google's documentation, but I found a thread with a similar problem

So has anyone succeeded with deployment on GCE? Or do you have any ideas for a workaround?

I was thinking about deploying UPF on a separate VM, like here https://github.com/Orange-OpenSource/towards5gs-helm/issues/25 - could that do the job?

Thanks in advance

[raoufkh]

Hello @konradkar2

The use of MACVLAN interface needs promiscuous mode to be enabled on master interfaces.

I'm not sure that deploying the UPF on a separate node could resolve the problem. In fact, some problems may occur when SMF communicates with UPF because of the promiscuous mode being disables

Bests Raouf

[konradkar2]

I've tried that and still no success

eth1: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1460
        inet 192.168.11.100  netmask 255.255.255.255  broadcast 0.0.0.0
        inet6 fe80::4001:c0ff:fea8:b64  prefixlen 64  scopeid 0x20<link>
        ether 42:01:c0:a8:0b:64  txqueuelen 1000  (Ethernet)
        RX packets 344  bytes 23326 (23.3 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 550  bytes 32314 (32.3 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Also, I've checked if these ping packets are getting transmitted - they do, TX counter gets incremented with each ping. And about SMF-UPF communication - it is fine, I've deployed free5gc + ueransim, and on UE ping tells about an unreachable host at UPF's n6 interface.

You can check my config here in the worker directory https://github.com/konradkar2/easy-free5gc I've tried the same configuration with minor changes on the ESXI platform, and it works fine.

Any ideas what I could also check?

[raoufkh]

Hello

Can you check if promiscuous mode is enabled on the gateway interface of UPF N6 interface (not its master interface)?

[konradkar2]

Well, the gateway is provided by Gcloud - when you create a VM instance, there's a host-like gateway made. I'm using it and I don't think I can check or configure it. Thanks for response

[raoufkh]

Promiscuous mode need to be enabled on the gateway. Otherwise, UE's ping requests will reach the destination but responses will not reach the UEs.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.