Linux, FreeBSD, Juniper, Cisco / Network security articles and troubleshooting guides

Juniper SRX MTU / MSS / Fragmentation problems with Ipsec vpn tunnel

The MSS (Maximum Segment Size) is a TCP connection mechanism or parameter through which a TCP side informes the other side the maximum size tcp segment size it can receive for that specific connection.

MSS is set only during tcp 3-way handshake and is part of the "TCP Options" field of tcp header only for SYN packets. TCPDUMP example:


Under normal conditions, ethernet MTU is 1500 bytes and the MSS (MTU - ip header - tcp header) is 1460.

In any VPN configuration, due to ipsec encapsulation (and, in some cases, GRE), the MTU for the vpn interface is lowered to a safe value of i.e.: 1400 in my case.

(Excelent article from Cisco: Resolve IP Fragmentation, MTU, MSS, and PMTUD Issues with GRE and IPSEC http://www.cisco.com/en/US/tech/tk827/tk369/technologies_white_paper09186a00800d6979.shtml)

Some situations when TCP client is accessing an external host through an Ipsec vpn tunnel for the first time, it is unaware of the MTU/MSS parameters for that specific path.

Let's say our PC 10.0.0.1 tries to connect to a webserver on IP 95.233.23.13.
MTU on the segment between our pc and first hop is 1500.
First hop Juniper SRX-1 sends the packet through an Ipsec tunnel over to Juniper SRX-2.
Juniper SRX-2 does a source nat for traffic from vpn to internet.

Connection breakdown.
#1. PC sends a syn packet to 95.233.23.13 port 80 with MSS 1456 (1500-40ip/tcp headers-5 vlan encapsulation)
#2. Web server 95.233.23.13 responds with a SYN-ACK packet with MSS set to 1460 (1500 -40 ip/tcp headers)
#3. PC sends an ACK packet to 95.233.23.13 (3way handshake is complete)
#4. PC sends the HTTP GET / request to 95.233.23.13 and sets the PSH flag (PSH flag in TCP tells the receiving end of the connection to "push" all buffered data to the receiving #application)
#5. Web server 95.233.23.13 acknowledges the GET request with a packet with ACK set to the seq of packet (seq 1:111 of packet #4, ack 111 of packet #5)
#6. Web server 95.233.23.13 starts sending data in segments of 1456 bytes (the MSS that PC told the server it can receive)
#7. Web server 95.233.23.13 sends the rest of the HTTP response and sets the PSH flag (tells the client's tcp stack/kernel to send all buffered data over to browser).

This is where the problem occurs as packets with 1456 bytes segments and don't fragment bit set in IP header will not enter the Ipsec tunnel (unless clear-bit is set, but this is not the purpose of this article) as the tunnel interfaces have an MTU of 1400 bytes.
Packets #8 and #9 are the icmp fragmentation needed packets.

Thus, SRX-2 will drop these large segments and will send an "ICMP fragmentation needed" back to the webserver. Tcpdump example of such behavior:

To a normal user this will seem like internet is very slow and that specific website doesn't load (as packets that won't fit the tunnel get dropped).

To fix this, Junos can be configured to override MSS parameter of packets entering Ipsec/GRE (in/out for GRE) tunnels (this is done on both devices SRX-1 and SRX-2).


And here's the tcpdump example:

In my case, the TCP client is always the browser and, unless I'm sending a huge HTTP request, my computer will send smaller-than-Ipsec-MTU packets, so it will not be aware of the MTU limitation of the ipsec segment along the path, as the ICMP "fragmentation needed" is sent only when a large packet arrives to the border of a low-MTU segment, not when a SYN packet with a large MSS is detected (although that wouldn't be such a bad idea).

hi thanks you so much for up dating us with this precis information.you did verb great job keep it up.......

_________________
"ONe Day I will be Part of your World.")