Vyatta, DSL and intermittent failed downloads solved

As part of the OneIP project, I wanted to consolidate in to one box (preferably VM), if possible, routing and firewall functions as well as DSL termination.  The first objective was to see if Vyatta was up to the task.

The telco hands off their connection to me from the DSL modem as an Ethernet interface, which I switch in to a VLAN on a core switch, then trunk in to a dedicated virtual network interface inside of vmWare for the Vyatta image to connect to.  The DSL modem is configured to operate in passthru mode, it does none of the PPPoE authentication. That's left to the endpoint, in this case the Vyatta VM.

The config on the Vyatta side is fairly easy and simple.  It looks like this:

 ethernet eth0 {
     description "DSL Interface"
     duplex auto
     hw-id 00:0c:29:88:bf:fa
     pppoe 0 {
         default-route auto
         mtu 1492
         name-server auto
         password <<PASSWORD GOES HERE>>
         user-id <<ACCOUNT GOES HERE>>
     smp_affinity auto
     speed auto

Simple, right?  The interface came up, got an IP address, ping and traceroute worked.  I thought we were in business.

However, downloads of content of varying sizes started failing.  It was impossible to download a large file like zip files, etc.  Random images would break in pages, or pages would hang on load.  It seemed a lot like an MTU issue, but I confirmed with the telco the MTU was correct.

The culprit ended up being a bug introduced in VC6.5 regarding TCP MSS.  The solution was to clamp MSS to MTU.  Unfortunately, there is no code in the standard config to do this, so I ended up with the following line in  /opt/vyatta/etc/config/scripts/vyatta-postconfig-bootup.script [1]

sudo iptables -t mangle -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu [2]
The problem is solved and sticks across reboots.