Friday, September 14, 2012

traceroute through MPLS

traceroute is often used as an effective analysis and troubleshooting tool. It is easily interpreted in a hop by hop routing network. Tracing packets through an MPLS network, however, requires more in-depth understanding of the internetworking between routing and tag switching.

The best place to start is the MPLS PE router. On the PE router, each customer’s VPN is represented by a vrf, in this case vrf “bigco”. Examining routing table for customer’s remote destination network (, notice its “next hop” is the remote PE (BGP RR address). This may be counter-intuitive that a customer VPN has a next hop in the global routing table (effectively leaping from one vrf to another), but this is precisely where MPLS does its magic.

A_PE1#sho ip route vrf bigco
Routing entry for
  Last update from 5d18h ago
  Routing Descriptor Blocks:
  * (Default-IP-Routing-Table), from, 5d18h ago

Note in the above display, customer VPN has a routing next hop which exists only in the global routing table.  “Under the hood”, when customer VPN traffic arrives at PE, it is tag switched (not routed) through the MPLS network.

  • Customer VPN destinations are learned from BGP peers (in this case is BGP RR)
  • Note is not in vrf “bigco”, rather it is global “Default-IP-Routing-Table”
  • How can a VPN route’s next hop to be global? On PE it is necessary, it is a special internal hook to make the linkage between routing and tag switching. All VPN route next hops are PE peers at layer 3

Here is a command that clearly illustrates the linkage between PE next hop and tag switching of VPN routes: “show bgp vpnv4 unicast vrf … tag”. The “tag” option is hidden. Here it shows that the next hop for VPN traffic is a remote PE.

A_PE1#sh bgp vpnv4 uni vrf bigco tag
   Network          Next Hop      In tag/Out tag      notag/15

In order to reach the remote PE, PE looks up its tag switching table. In this case, tag switching identifies with a local tag of 78, and out tag of 34. Tag switching continues through the MPLS network, until it reaches the remote PE.
A_PE1#sh mpls forward
78     34     0             Gi0/1

The topology represents the simplest form of an MPLS network which consists of P and PE. The sample VPN has a customer destination of When packet to that destination arrives at A_PE1, routing table indicates its “next hop” as the remote B_PE1. To reach remote next hop, packet is tag switched through the MPLS network. The core routers (P) have no concept of VPN destinations, they are simply tag switching between PE destinations.

Traceroute ,when interpreted correctly, provides a nice end to end view. Here it shows tag switching from PE on. Note the inside tag identifies the VPN destination which does not change during transport. The outside tags (34 and 64) are tagging switching through the MPLS network (P and PE). Once it gets to the remote PE, the inside tag (15) is popped and regular routing applies to next hop (CE).
A_PE1#traceroute vrf bigco
  1 [MPLS: Labels 48/15 Exp 0] 128 msec 184 msec 216 msec
  2 [MPLS: Labels 43/15 Exp 0] 196 msec 232 msec 152 msec
  3 [MPLS: Label 15 Exp 0] 152 msec 88 msec 168 msec
  4 656 msec 704 msec 644 msec

No comments:

Post a Comment