ULCL Routing Issue

[andrewferguson]

Hi Free5GC Team,

I firstly wanted to thank you for your amazing work on Free5GC! It is a remarkable project, and incredible work.

I'm trying to get the ULCL setup to work and am running into (what I think is) a routing issue. I am able to get the UE to connect, and when I ping 1.0.0.1 the traffic correctly exists at the I-UPF, rather than continuing through to the PSA-UPF (as per the uerouting.yaml configuration). However, when I try to ping another public IP (e.g. 8.8.8.8 or 1.1.1.1) I get no response.

I've looked at the traffic flow inside the I-UPF using tshark, and observed the following (I've edited the tshark log slightly to include the names of the components rather than their IPs).

Source (GTP Src)         | Destination (GTP Dest)    | Info
-------------------------|---------------------------|---------------
UERANSIM (GTP 10.60.0.1) => I-UPF   (GTP 1.1.1.1)      GTP <ICMP> 144 Echo (ping) request  id=0x001e, seq=1/256, ttl=64
I-UPF    (GTP 10.60.0.1) => PSA-UPF (GTP 1.1.1.1)      GTP <ICMP> 144 Echo (ping) request  id=0x001e, seq=1/256, ttl=64
PSA-UPF  (GTP 1.1.1.1)   => I-UPF   (GTP 10.60.0.1)    GTP <ICMP> 144 Echo (ping) reply    id=0x001e, seq=1/256, ttl=253

My interpretation is that the ping to 1.1.1.1 is correctly sent through the I-UPF to the PSA-UPF, the PSA-UPF sends it to 1.1.1.1, and receives a reply, which is sent back to the I-UPF. However the I-UPF does not forward the packet back to the UE - there are no more GTP messages in the flow, only the three I've listed above.

I have also confirmed that the non-ULCL configuration works as expected, and with this setup I am able to ping various IPs including 8.8.8.8 and 1.1.1.1. So I do not believe that this issue is caused by a network misconfiguration on my host machine.

Would you be able to assist in resolving this issue? I am happy to provide further information / logs / PCAP traces as needed.

Many thanks for your help!

[abousselmi]

Hi @andrewferguson

Did you check the iptables nat config ?

[andrewferguson]

Hi @abousselmi ,

Thanks for the reply. I have confirmed that the iptables config is as specified in the config/upf-iptables.sh file.

After doing some more investigating, I have realised that the gtp5g kernel module is logging an error every time I try a ping that should go through the I-UPF and PSA-UPF:

upfgtp:[gtp5g] gtp1u_udp_encap_recv: No PDR match this skb : teid[1]

I have attached the PCAP traces of the PFCP and GTP messages for both the I-UPF and PSA-UPF. Perhaps this can help pinpoint the issue?

Other things to note:

• I have confirmed that the UE is using the correct config file (uecfg-ulcl.yaml rather than uecfg.yaml).
• Pinging an IP that just goes through the I-UPF and not the PSA-UPF (i.e. 1.0.0.1) does not give the above error.

Thanks!

[abousselmi]

Hi @andrewferguson

Take a look at this: https://forum.free5gc.org/t/upf-est-createfar-error-invalid-argument/2111

Cheers,

[andrewferguson]

Hi @abousselmi ,

Thanks for linking to that post. However, I am using Ubuntu 20.04 with Kernel 5.4.0-212-generic so that is not the issue.

I have however managed to find the problem - if I downgrade the SMF to v4.0.0 (compared to v4.0.1) everything works. This is partly (OK, mostly!) my fault - upon re-reading the instructions in this repo I observed the This configuration have been tested using release v4.0.0 note which I didn't notice before. I apologise if I have been wasting your time!

I am planning to leave an issue in the free5gc/smf repo as I think that's the best place for it. I'm a little confused about matching up the version numbers (that repo is on v1.3.2 but the docker container is on v4.0.1)?

For this issue, I am happy to close it. But should I submit a PR in this repo for either of the following:

1. Noting that v4.0.1 does not work for ULCL in the README?
2. Downgrading the SMF container image in the docker-compose-ulcl.yaml file to v4.0.0 so it does work? (With a note in the README)

Thanks again for your time and patience in responding to this issue!

[abousselmi]

Thanks @andrewferguson for your comment.

Option 2 is fine, we can mention it as a.workaround, until the issue that you would be raising on the SMF repo gets resolved.

Please feel free to make a PR.

Cheers,