Resolved: T-Mobile Home Internet VPN Issue

NOTE: This article will help you resolve VPN issues on your T-Mobile Home Internet, irrespective of the type of VPN provider or host OS (Windows/Mac/Linux). Read till the end…

-Mobile Home Internet is my new ISP (Internet Service Provider). I was using Cox Internet for a long time, but couldn’t resist checking T-Mobile's new Home Internet fascinating deal:

  • Not just great download but upload speed too.
  • Great cost-saving, the monthly bill reduced to half.
  • Wireless/Cableless setup.
  • Fewer setup devices (Modem + Router > Gateway), though Cox offers this too in a newer setup.
  • Portable WiFi, take it anywhere with you.
  • No monthly device rental fee.
  • Taxes are included in the price.
  • Totally win-win :).

I was so happy to set up their provided Internet Gateway over the weekend, saw the numbers on speedtest result twice of what I used to, made the whole internet browsing & streaming experience great.

hen comes the working weekday, and the problem surfaced when I could connect to VPN but couldn’t browse some Corp and Internet sites on my VPN. Symptoms I was experiencing:

  • Very slow connection.
  • Many sites don’t load, eventually times out.
  • Some sites only the page title loads.

Here is the configuration I have:

Host: Apple Mac OS X 10.15.7

VPN:
Provider: GlobalProtect v5.0.9–15 Palo Alto Networks
Tunnel Mode: IPSEC

T-Mobile Home Internet Gateway:
Model: 5G21–12-A Grey
Hardware Version: 3TG00739AABB
Software Version: 1.2003.03.0178

Example failing Sites:
Internal (Corp): JIRA, ServiceNow
External: Stackoverflow.com

Everything works fine on my Cox internet or my TMobile Hotspot but not just TMobile Home Internet. Frustrating isn’t it!
After hours of effort: Googling, talking to Office Tech Support & talking to T-Mobile Tech Support, I could put together scattered information to help me resolve the riddle.

MTU (Maximum Transmission Unit): the Culprit

MTU can be thought of as the largest packet size that can be sent from your computer over the network to its destination. During the packet’s journey if it encounters a hop that can’t process its size then it will be fragmented or dropped — in which case an ICMP error message “Frag needed and DF set or Datagram too big or Message too long” will be returned. Based on this error message, the optimal MTU is discovered via the PMTUD (Path MTU Discovery) technique (check below we will do this manually).

It is very likely that one encounters MTU problems when tunneling(VPN) traffic. So in my case, MTU on GlobalProtect client interface was set too high for TMobile Home Internet.

This is how I found out the issue:

Step 1: Run terminal command ifconfig | grep mtu
It shows me all my network interfaces with their respective MTU’s limitation.

en0: Mac LAN/WIFI Adapter, Default set to 1500. It can also be viewed or changed from Apple menu icon > System Preferences > Network > Advanced > Ethernet or Hardware > Configure ‘Manually’ > MTU ‘Custom’
gpd0: PANGP Virtual Adapter used by GlobalProtect App, Defaults to 1400

As you see, the max packet limit without fragmentation is set to 1400 Bytes.

Step 2: Quick test to find out what are the different MTU settings between my WIFI options without VPN (VPN Disconnected). Run terminal command ping -D -s <packet-size> yahoo.com
where:
-D
Set the Don’t Fragment bit.
-s The number of data bytes to be sent.

NOTE: Adjust packet-size, until you receive the message frag needed and DF set with set MTU limit.

a) With TMobile Home Internet: As you can see the MTU setting on the Nokia 5G Gateway/Router is limited to 1420. The Router page (192.168.12.1) on the TMobile gateway is pretty locked, users are only allowed to view/change limited settings.

b) With TMobile Tethering/HotSpot: The MTU is limited by the phone device is 1440.

c) With Cox Internet: I didn’t get the MTU limit displayed since my router & Mac en0 limits are same, set to high 1500. The en0 adapter itself is rejecting the message before it can reach the router. But you can see I can run a packet size up to 1500 = 1472 bytes + 28 Bytes (20 Bytes IP Header, 8 Bytes ICMP Header).

Now we understand the TMobile Home Internet setup has the lowest MTU setting, which is causing the VPN issues. Though it's still higher than gpd0 limit of 1400, however considering during IPSec VPN tunneling, there are additional headers that get added to the datagram/data bytes which exceed the difference. Find more details of variable header (overhead) size in IPSec Tunnel here:
https://live.paloaltonetworks.com/t5/globalprotect-articles/troubleshooting-globalprotect-mtu-issues/ta-p/384894

Solution

irst step, let's find out the optimal MTU value required for our network, VPN connected with TMobile Home Internet, by manual PMTUD technique (sweeping ping test).
Run command:

ping -D -g <sweepminsize> -G <sweepmaxsize> -h <sweepincrsize> yahoo.com where:
-g
Specify the size of ICMP payload to start with when sending sweeping pings.
-G Specify the maximum size of ICMP payload when sending sweeping pings.
-h Specify the number of bytes to increment the size of ICMP payload after each sweep when sending sweeping pings
NOTE: Considering our max MTU is limited to 1400 by gpd0 adapter, this is the command we will run:ping -D -g 1300 -G 1400 -h 2 yahoo.com

As you can see the max bytes packet size could successfully be sent is 1330, then adding 20 bytes of IP Header, the optimal MTU derived value is 1350.

econd step, since we can’t change the TMobile Home Internet Gateway MTU setting, we will set GlobalProtect adaptor MTU to lower value:

1. For Mac GP client: ifconfig can change virtual adapter MTU size:sudo ifconfig gpd0 mtu 13502. For Windows GP client 5.0.x and older: MTU can be changed by modifying PANGP virtual adapter MTU setting directly as follows:a. use netsh command to change the MTU size: (<index> is the virtual interface index, use 'netsh interface ipv4 show interfaces' to list all interface indexes and names)
netsh interface ipv4 set interface <index> mtu=1350
netsh interface ipv6 set interface <index> mtu=1350

b. use powershell command to automatically find out virtual interface and set mtu
powershell -Command "Get-WmiObject win32_networkadapter | where-object ServiceName -eq PanGpd | ForEach {netsh interface ipv4 set interface $_.InterfaceIndex mtu=1350; netsh interface ipv6 set interface $_.InterfaceIndex mtu=1350}"
NOTE: You need root/admin access to be able to change it through command line.Source: https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PPbZCAW

Starting GlobalProtect app 5.2.4, MTU can be changed through the Adapter App setting.
Reference: https://docs.paloaltonetworks.com/globalprotect/5-2/globalprotect-app-new-features/new-features-released-in-gp-app/configurable-maximum-transmission-unit-for-globalprotect-connections

hird Step, test your sites, it should be accessible now. You can also do ping/traceroute test.
> traceroute takes total message size, including the header.

hird Step, test your sites, it should be accessible now. You can also do ping/traceroute test.
> traceroute takes total message size, including the header.

ourth Step, for GP version less than 5.2.4, since we are overriding the default for GP adapter using the command line, it will reset on the next system startup to 1400. So after every system restart either we can manually run this command:
sudo ifconfig gpd0 mtu 1350

Or if you are lazy like me, we would want to automate this MTU setting update process. Let’s use Maclauchd service to do that for us:
1) Create a shell script file in your workspace, say /Users/akumar/workspace/scripts/vpn-gp-set.sh

#!/bin/bash
# lowering vpn gp mtu limit
echo $(date) Starting with delay…
sleep 60
ifconfig | grep mtu
echo ‘Setting gpd0 mtu’
ifconfig gpd0 mtu 1350
ifconfig gpd0 | grep mtu
echo ‘gpd0 mtu set’

NOTES: Since launchd process can’t be sequenced/ordered, putting an explicit sleep of 60s (it can be less or more for you, depending on your system startup time), gives ample time for the GP app to boot up, and create gpd0 interface for us to override. This waiting period will not slow down your system boot time.

2) Make your script file executable with the command:
chmod +x /Users/akumar/workspace/scripts/vpn-gp-set.sh

3) Create a plist file at /Library/LaunchDaemons/com.local.vpn.gp.mtu.plist

<?xml version=”1.0" encoding=”UTF-8"?>
<!DOCTYPE plist PUBLIC “-//Apple Computer//DTD PLIST 1.0//EN” “http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version=”1.0">
<dict>
<key>Label</key>
<string>com.local.vpn.gp.mtu</string>
<key>Program</key>
<string>/Users/akumar/workspace/scripts/vpn-gp-set.sh</string>
<key>StandardErrorPath</key>
<string>/Users/akumar/workspace/scripts/vpn-gp-set.err</string>
<key>StandardOutPath</key>
<string>/Users/akumar/workspace/scripts/vpn-gp-set.log</string>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>

oila!, it's done. Now, your MTU setting will be persisted between system reboots. If not, then you can check the logs in the workspace directory for error (.err) or run logs (.log) for debugging.

Additionals:

  • If you don’t see gpd0 interface in your ifconfig and your VPN app doesn’t let you configure the MTU through the GUI, you can also limit one of theutunX interface where X can be 0, 1, 2, etc. tun (short for tunneling) interfaces are created by Mac following VPN installation. Start with utun with lowest MTU to further lower it to optimal MTU. You will have to play around with each utun to see which fixes your issue.

I hope you liked ❤️ this article, stay tuned for more posts. All feedback, comments & questions are welcomed. 🏳️‍🌈

Donation😇

If this helped you reduce time to debug, you can buy me a cup of coffee ☕

Lead Full Stack & DevOps Engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store