MPLS-based VPNs

When I was learning about MPLS, I was a little bit confused about the fact, how BGP distributes vpn labels and how there are used. 

On the picture (click to enlarge) we can see, that customer bank has 2 sites connected to an ISP, who is running BGP in AS 60 and provides an MPLS-based VPNs. The ISP has divided its AS into two sub-AS. Although I am using in my example confederations, in fact the same concept is behind an inter-provider MPLS-based VPNs. In addition to this, I am using a simple formula, that router Rn can distribute labels from range n00 to n99 (R3 sends labels with values between 300-399 and so on).

1) In the first scenario, I will merge OSPF1 and OSPF2 (assuming that the ISP is running only single IGP within the whole AS) and establish a single BGP session between R3 and R6. Next I will send ping from R3 to R2 (154.0.0.1).

In this case, R6 is responsible for assigning a vpn label for the 154.0.0.0/24 network and sending this label via MP-BGP to R3:

R6#show mpls forwarding-table
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
tag    tag or VC   or Tunnel Id      switched   interface
                                                 
!Some lines omitted
609    706         12.0.0.0/30       0          Se0/0      point2point
610    707         11.0.0.0/30       0          Se0/0      point2point
611    Untagged    154.0.0.0/24[V]   0          Se0/1      point2point


We can see, that R6 has chosen label with value of 611 for this network. Next via MP-BGP this label was advertised to R3:

R3#show ip bgp vpnv4 all 154.0.0.0
BGP routing table entry for 1:1:154.0.0.0/24, version 17
                                            
! Some lines omitted
    6.6.6.6 (metric 321) from 6.6.6.6 (6.6.6.6)
      Origin incomplete, metric 1, localpref 100, valid, external, best
      Extended Community: RT:1:1
      mpls labels in/out nolabel/611

To see the whole stack of labels, which R3 will use to forward packets to 154.0.0.0/24, we can use:

R3#show ip cef vrf bank 154.0.0.0
154.0.0.0/24, version 14, epoch 0, cached adjacency to Serial0/1
0 packets, 0 bytes
  tag information set
    local tag: VPN-route-head
    fast tag rewrite with Se0/1, point2point, tags imposed: {405 611}
  via 6.6.6.6, 0 dependencies, recursive
    next hop 11.0.0.2, Serial0/1 via 6.6.6.6/32
    valid cached adjacency
    tag rewrite with Se0/1, point2point, tags imposed: {405 611}

And finally, the traceroute:

R3#traceroute vrf bank 154.0.0.1

Type escape sequence to abort.
Tracing the route to 154.0.0.1

  1 11.0.0.2 [MPLS: Labels 405/611 Exp 0] 140 msec 80 msec 100 msec
  2 12.0.0.2 [MPLS: Labels 509/611 Exp 0] 116 msec 180 msec 84 msec
  3 13.0.0.2 [MPLS: Labels 801/611 Exp 0] 76 msec 112 msec 228 msec
  4 14.0.0.1 [MPLS: Labels 701/611 Exp 0] 64 msec 148 msec 196 msec
  5 16.0.0.2 [MPLS: Label 611 Exp 0] 64 msec 136 msec 36 msec
  6 16.0.0.1 192 msec 232 msec *

In this scenario, we can see, that the vpn label is assigned by R6 (611) and is unchanged when packet traverses in the LSP. However the top label is swapped on every single router within the LSP. Next R7 pops the top label and sends the packet with only one label (only the vpn label: 611) to R6. Finally R6 exactly knows what to do with packets destined to 154.0.0.1.

2) In this scenario I assume, that each sub-AS is running a separate IGP (OSPF1 & OSPF2). In contrast to the 1st scenario, I establish three BGP sessions: R3-R5, R5-R8 and R8-R6. Moreover R5 and R8 are configured with next-hop-self, when sending updates to R3 and R6, respectively:

 R3#traceroute vrf bank 154.0.0.1

Type escape sequence to abort.
Tracing the route to 154.0.0.1

  1 11.0.0.2 [MPLS: Labels 400/504 Exp 0] 284 msec 144 msec 100 msec
  2 13.0.0.1 [MPLS: Label 504 Exp 0] 216 msec 168 msec 268 msec
  3 14.0.0.2 [MPLS: Label 805 Exp 0] 116 msec 212 msec 112 msec
  4 14.0.0.1 [MPLS: Labels 701/611 Exp 0] 84 msec 244 msec 244 msec
  5 16.0.0.2 [MPLS: Label 611 Exp 0] 208 msec 140 msec 140 msec
  6 16.0.0.1 144 msec 148 msec *

Here we see three different vpn labels: 504, 805 and 611. Both R5 and R8 are swapping vpn labels:

R8#show ip bgp vpnv4 all 154.0.0.1
BGP routing table entry for 1:1:154.0.0.0/24, version 17
Paths: (1 available, best #1, table bank)
  Advertised to update-groups:
        2
  Local
    6.6.6.6 (metric 129) from 6.6.6.6 (6.6.6.6)
      Origin incomplete, metric 1, localpref 100, valid, internal, best
      Extended Community: RT:1:1
      mpls labels in/out 805/611

This behaviour is due to the fact, that routers in AS 65000 have no idea about routes in AS65001. The connectivity between them is provided only by the BGP session between R5 and R8. So when R3 is sending packets to R2:

• R3 imposes two labels: 504 as a vpn label assigned by R5, and 400 label assigned by R4 as a label to reach R5 
• R4 only pops the top label (PHP) and send the packet with vpn label only, (504)
• R5 pops its vpn label (504) and sends packet to R8 with the new vpn label (805)
• R8 does the same thing as R5, (swaps the vpn label). R8 also pushes a new label (701) as a label to reach R6
• R7 pops the top label (PHP) and sends the packet with vpn label only, (611)
• Finally R6 receives packet with its own vpn label, so R6 pops all labels and sends the unlabelled packet to R2

 3)  Basically this scenario is very similar to the 2nd one, however this time routers R5 and R8 are configured without next-hop-self option, and the network between them (13.0.0.0/30) are redistributed into both OSPF1 and OSPF2 processes. This means, that R3 can ping R8, and R6 can ping R5:

R3#traceroute vrf bank 154.0.0.1

Type escape sequence to abort.
Tracing the route to 154.0.0.1

  1 11.0.0.2 [MPLS: Labels 403/805 Exp 0] 188 msec 272 msec 168 msec
  2 12.0.0.2 [MPLS: Labels 503/805 Exp 0] 160 msec 172 msec 80 msec
  3 14.0.0.2 [MPLS: Label 805 Exp 0] 124 msec 288 msec 448 msec
  4 14.0.0.1 [MPLS: Labels 701/611 Exp 0] 196 msec 272 msec 72 msec
  5 16.0.0.2 [MPLS: Label 611 Exp 0] 96 msec 160 msec 132 msec
  6 16.0.0.1 192 msec 172 msec *

 In this example we see only two vpn labels: 805 and 611. Moreover, only R5 and R7 are performing PHP. In this scenario R3 uses R8 as a next hop (R3 has a route to R8, and R5 doesn't change the next-hop attribute in updates sending to R3) so R3 can use a vpn label directly assigned by R8 (this is the main difference between the previous scenario).

I hope that after reading this post, some of you have a basic idea how packets are forwarded when an MPLS-based VPNs are deployed.