MPLS VPN Label Basics – The LIB, the LFIB and the RIB(s)

LDP, or Label Distribution Protocol, is used to advertise label bindings to peers in an MPLS network.

The Label Information Base, or LIB, contains all received labels from remote peers and is similar to the IP RIB. Not all labels received from LDP neighbors are used since there will be a best path selected and to be used for forwarding for each prefix. Forwarding decisions are based on the Label Forwarding Information Base, or LFIB, once the best path towards the next-hop LSR is determined. How this is determined is based on the close relationship between the LIB, the LFIB and the IP routing table (RIB).

For clarity, we’ll be talking about non-ATM MPLS forwarding. ATM MPLS uses different LDP discovery, label retention and distribution methods because of ATM’s unique forwarding method and encapsulation(s).

Here’s our simple MPLS topology. We have two PE routers, connecting two customer sites. We also have a route reflector to reduce the number of IBGP connections required between PE routers. This is part of my MPLS lab so the irrelevant routers and configs will be omitted.

PE1 Router ID: 10.255.255.3/32
PE2 Router ID: 10.255.255.4/32
RR Router ID: 10.255.255.2/32

Routing within the MPLS network is provided by basic single-area IS-IS.

So how does MPLS build its Label FIB? First, let’s look at the VRF’s defined for this customer. We’ll be using VRF “Red” on both PE routers:

PE1#show ip vrf
  Name                             Default RD          Interfaces
  Red                              65000:1             Fa1/0
----
PE2#show ip vrf
  Name                             Default RD          Interfaces
  Red                              65000:1             Fa1/0

For VPNv4 routing between customer sites, MP-BGP is used to distribute label bindings for VRF routes. LDP will distribute label bindings for the Loopback0 BGP next-hop’s. OSPF is used between CE and PE routers.

On PE1, here are all the customer routes connected via Fa1/0

PE1#show ip route vrf Red ospf | in FastEthernet1/0
O IA    10.10.1.0/24 [110/2] via 10.1.1.2, 00:23:55, FastEthernet1/0
O       10.30.100.0/30 [110/101] via 10.1.1.2, 00:23:55, FastEthernet1/0
O       10.30.1.101/32 [110/2] via 10.1.1.2, 00:23:55, FastEthernet1/0

OSPF routes running in VRF Red are redistributed into MP-BGP under “address-family ipv4 vrf Red”.

PE1#show ip bgp vpnv4 rd 65000:1 10.10.1.0/24
BGP routing table entry for 65000:1:10.10.1.0/24, version 14
Paths: (1 available, best #1, table Red)
  Advertised to update-groups:
        1
  Local
    10.1.1.2 from 0.0.0.0 (10.255.255.3)
      Origin IGP, metric 2, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:65000:1 OSPF DOMAIN ID:0x0005:0x000000010200 
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.100.1.101:0
      mpls labels in/out 25/nolabel
PE1#

Here we can see the MPLS label binding that will be sent to other PE routers. PE routers with a VRF matching the same route targets will import these routes into the VRF of other sites.

In the MPLS LDP Forwarding table, an entry is created for these “local” VRF routes. That is, the routes reachable via the next-hop CE router:

PE1#show mpls forwarding-table vrf Red 10.10.1.0
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop    
Label  Label or VC   or Tunnel Id      Switched      interface              
25     No Label      10.10.1.0/24[V]   0             Fa1/0      10.1.1.2

This is the label that will be advertised to MP-BGP peers (in this case, reflected to PE2).

PE1 will also have a label binding for its own BGP next-hop IP address, which is the Loopback0 interface under the global routing table:

PE1#show mpls ldp bindings local 10.255.255.3 32
  lib entry: 10.255.255.3/32, rev 4
        local binding:  label: imp-null

This is advertised as an Implicit Null label, to avoid performing two lookups (once in the LFIB and another in the RIB for its connected prefix). Core P routers will have a label binding for this prefix:

CoreP#show mpls ldp bindings local
...
  lib entry: 10.255.255.3/32, rev 14
        local binding:  label: 17

In order for the correct labels to be used for forwarding, two labels will have to be used. The top label will be used to forward packets in the core (P) MPLS network to the BGP next-hop (either the loopback of PE1 or PE2, depending on the packet destination from the CE sites). The bottom label will be used to identify the VRF and outgoing interface to route packets towards the customer router(s).

So, for customer at Site B to reach network 10.10.1.0/24 at Site A, PE2 will use the following labels:

  • Label 17 for the transport label to PE1, received from MPLS core router(s); identified via RIB lookup in the VRF “Red” to identify next-hop IP address
  • Label 25 for the VPN label, received from PE1 via MP-BGP; identified in the VPNv4 BGP RIB

To verify:

Packet received on Fa1/0 destined for 10.10.1.1/24 from Site B router(s), performs VRF Red RIB lookup:

PE2#show ip route vrf Red ospf  
Routing Table: Red

     10.0.0.0/8 is variably subnetted, 9 subnets, 3 masks
O IA    10.10.1.0/24 [110/52] via 10.255.255.3, 00:46:32

PE2 identifies next-hop IP address, which is the BGP next-hop of PE1. Since it is traversing the MPLS network on the outgoing interface FastEthernet2/0 into the core, it needs to be labeled before transit:

PE2#show ip bgp vpnv4 rd 65000:1 10.10.1.0/24
BGP routing table entry for 65000:1:10.10.1.0/24, version 34
Paths: (1 available, best #1, table Red, RIB-failure(17))
  Not advertised to any peer
  Local
    10.255.255.3 (metric 20) from 10.255.255.2 (10.255.255.2)
      Origin IGP, metric 2, localpref 100, valid, internal, best
      Extended Community: RT:65000:1 OSPF DOMAIN ID:0x0005:0x000000010200 
        OSPF RT:0.0.0.0:3:0 OSPF ROUTER ID:10.100.1.101:0
      Originator: 10.255.255.3, Cluster list: 10.255.255.2
      mpls labels in/out nolabel/25
PE2#show mpls ldp bindings 
...
lib entry: 10.255.255.3/32, rev 8
        local binding:  label: 17
        remote binding: lsr: 10.255.255.1:0, label: 17

Therefore, packets destined for customer Site A will be sent with the labels 17 and 25.

PE2#traceroute vrf Red 10.10.1.1

Type escape sequence to abort.
Tracing the route to 10.10.1.1

  1 10.10.1.9 [MPLS: Labels 17/25 Exp 0] 76 msec 52 msec 72 msec
  2 10.1.1.1 [MPLS: Label 25 Exp 0] 84 msec 40 msec 40 msec
  3 10.1.1.2 132 msec *  60 msec
PE2#

Below I will attempt to illustrate the decision process and relationship between all the entries in an MPLS router to demonstrate these relationships:

  1. An incoming packet from Site B, destined for 10.10.1.1, is received on PE2’s VRF interface Fa1/0.
  2. IP lookup is performed in the VRF table “Red” and identifies next-hop IP address known via global routing table. This route was redistributed from BGP into OSPF (hence the RIB failure) via PE1 next-hop of 10.255.255.3.
  3. BGP RIB lookup is performed to identify the VPN label. Under the VPNv4 address family, outgoing label is 25, as advertised by PE1
  4. Global RIB lookup is performed for BGP next-hop learned in VRF. Actual IP next hop in the MPLS core is identified (10.10.1.9) via outgoing interface FastEthernet2/0.
  5. Outgoing interface is an MPLS-enabled interface. LIB lookup performed to find bound address of the MPLS core next-hop of 10.10.1.9. Based on LDP neighbor that has bound IP address 10.10.1.9, remote label received from that LDP neighbor is used for transport label to PE1 loopback.
  6. LFIB entry created with Label 17, outgoing interface FastEthernet2/0 with next-hop IP address of 10.10.1.9 into core MPLS network and is routed onto PE1.

Example of “show mpls ldp neighbor” displays bound addresses for core P router(s). LIB entry selected for forwarding in LFIB is based on which LDP neighbor this next-hop IP address in the global RIB is bound to. In this case, only one LDP neighbor exists:

PE2#show mpls ldp neighbor 
    Peer LDP Ident: 10.255.255.1:0; Local LDP Ident 10.255.255.4:0
        TCP connection: 10.255.255.1.646 - 10.255.255.4.34846
        State: Oper; Msgs sent/rcvd: 118/119; Downstream
        Up time: 01:33:30
        LDP discovery sources:
          FastEthernet2/0, Src IP addr: 10.10.1.9
        Addresses bound to peer LDP Ident:
          10.10.1.1       10.255.255.1    10.10.1.5       10.10.1.9      
          10.10.1.13      
PE2#

In an MPLS VPN network, the label bindings received from remote peers (LIB), the label forwarding table (LFIB) and the various IP routing tables (VRF RIB, global RIB, BGP RIB, etc.) all work together in tandem to create the label stack used to forward packets from one VPN site to another. This is the basic forwarding paradigm of Multiprotocol Label Switching and enables service providers to provide L3VPN services to customers along with proper separation of customer routing via the use of VRF’s. References used in this post are Luc De Ghein’s MPLS Fundamentals book from Cisco Press and Cisco documentation, found at http://www.cisco.com/go/mpls.

BGP+MPLS Exam Passed! QoS and other things

Hi All,
I’ve been staying away from the Twitters and blogging to focus down on my BGP+MPLS composite exam. I wrote it this afternoon and passed, w00t! I wanted to give a HUGE thanks to Jarek Rek at his blog hackingcisco.blogspot.com. His labs are great to practice configuring Cisco IP routing and I recommend anyone preparing for CCNP ROUTE, CCIE R&S or anything routing-related to check it out. Thanks again Jarek!

So other than beating my chest, I will be finishing up some outstanding blog posts around my BGP and MPLS studies before moving on to my QOS exam. I’ve also been involved more and more with Juniper at work, along with trying to get up to speed with L2VPN technologies like basic EoMPLS. Metro Ethernet is a whole other rabbit hole that I wish to descend into eventually but at the moment, it’s still a bit of a mystery. It makes keeping up with blogging and goofing off at home challenging since I’m in study mode for CCIP while getting pulled in twenty different directions for real-world job stuff.

I’m currently looking for my next book go to through in prep of my QOS exam. My coworker had recommended Cisco Press’ “End to End QoS Network Design” while most of Learning@Cisco seems to recommend the IP Telephony QOS Exam study guide. That’s still up in the air until I review the exam topics. If anyone has a solid recommendation for 642-642, please let me know in the comments!

Last update, I picked up the newest edition of “TCP/IP Illustrated Volume 1”. Stevens book is often recommended by the experts and is considered the bible of Layers 4 and up. It’s a comprehensive tome and a great reference.

More technical posts coming shortly.

BGP Aggregate Addresses

This month I am studying in preparation for my CCIP BGP+MPLS exam (booked June 27th). I decided to go through with the CCIP certification, despite my annoyance with the new Service Provider track because the BGP, MPLS and QoS topics are covered in length on the CCIE Routing & Switching blueprints. I figure this is a good bridge to fill in the gaps that CCNP R&S leaves out (in particular MPLS and QoS, which isn’t covered at all in any of the CCNP blueprints).

As such, I’ve been able to dive into all the knobs and switches that BGP offers to control routing policy. For those who have gone through the newer CCNP R&S track, the BGP fundamentals are explained and covered enough to get engineers familiarized with its operations. There’s a lot of depth lacking in CCNP and for good reason…BGP can be a career in and of itself. In service provider environments, when you’re pulling half a million IPv4 routes from upstream peers and providing L3VPN services to your customers via MPLS, you need a protocol like BGP that can scale.

Route summarization, when half a million routes are available on the global Internet table, can help keep specific and unnecessary routes from propagating out to upstream providers and thus alleviate memory and CPU required for carrying these thousands of routes. To summarize a set of routes in BGP, you have a few options:

  • Manual static Null0 routes advertised in BGP
  • aggregate-address command

Let’s look at a scenario. This is taken out of a BGP topology I’ve been working on this week to help me gain a better understanding of some of the more advanced BGP topics.

Subnets*:

  • 100.100.255.0/24 for all CE-facing Point-to-Point links
  • 100.100.254.0/24 for BGP update souce loopbacks
  • 100.100.253.0/24 for all inter-AS Point-to-Point links
  • 100.100.200.0/24 allocated to Enterprise A from this ISP
  • 100.100.0.0/16 allocated to ISP from registry

*Note: this is my best guess of how an ISP would assign addressing in its network. Being an enterprise guy, I’ve yet to be exposed to any service provider network. For those with more experience, any corrections on this please let me know in the comments below 🙂

In this topology, we have one route reflector “RR” with IBGP running between RR and all the PE routers (just PE1 and PE2 in this case).
We want to aggregate all of the ISP’s routes to advertise upstream to Upstream SP at AS 200.

Below is the BGP RIB on our route reflector before any aggregation. These are all the routes advertised by the PE routers as well as any allocations given to customers who require more than a single address.

RR#sh ip bgp
BGP table version is 31, local router ID is 100.100.254.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
* i100.100.200.0/24 100.100.254.3            0    100      0 65501 i
*>i                 100.100.254.1            0    100      0 65501 i
*>i100.100.255.0/31 100.100.254.1            0    100      0 i
*>i100.100.255.8/31 100.100.254.3            0    100      0 i
!
!
AS200#sh ip bgp
BGP table version is 37, local router ID is 100.100.253.0
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 100.100.200.0/24 100.100.253.1                          0 400 i
*> 100.100.255.0/31 100.100.253.1                          0 400 i
*> 100.100.255.8/31 100.100.253.1                          0 400 i

As you can see, in a huge service provider network, the BGP RIB would be filled with any public IP addresses used to connect its customers to the outside world, as well as any allocations given by this ISP to its larger customers (such as Enterprise A in this case, which is dual homed at PE1 and PE2). Also included is the BGP RIB of the upstream AS 200 router, who receives these specific prefixes from AS 400.

Now let’s reduce the routing table by aggregating them into a summarized route. First, we’ll start by adding in a static route to the Null0 interface and advertise it in BGP:

! On RR:
conf t
 ip route 100.100.0.0 255.255.0.0 Null0
!
router bgp 400
 network 100.100.0.0 mask 255.255.0.0
!
RR#sh ip ro static
100.0.0.0/8 is variably subnetted, 11 subnets, 4 masks
  S 100.100.0.0/16 is directly connected, Null0
RR#sh ip bgp
BGP table version is 25, local router ID is 100.100.254.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
          r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*> 100.100.0.0/16   0.0.0.0                            32768 i
*>i100.100.255.0/31 100.100.254.1            0    100      0 i
*>i100.100.255.8/31 100.100.254.3            0    100      0 i

And to verify on the upstream AS:

AS200#sh ip bgp
BGP table version is 31, local router ID is 100.100.253.0
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
          r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*> 100.100.0.0/16   100.100.253.1            0             0 400 i
*> 100.100.200.0/24 100.100.253.1                          0 400 i
*> 100.100.255.0/31 100.100.253.1                          0 400 i
*> 100.100.255.8/31 100.100.253.1                          0 400 i

Since we’re still advertising the more specific routes inside the ISP AS 400, manual filtering will be required on the router reflector. This can be accomplished by a simple prefix list or route-map on RR.

! on RR
ip prefix-list OurAlloc permit 100.100.0.0/16 
!
! Match only our allocated address space
!
router bgp 400
 neighbor 100.100.253.0 prefix-list OurAlloc out

The problem with this approach is that, while it is fairly simple, does require you to manually filter any more-specific routes on the edge of your AS. Also, if you are serving multihomed customers with their own address allocation (independent of this ISP’s allocation), you will have to take those into account as well in your filtering.

The other way to aggregate a set of routes in BGP is through the aggregate-address command. This command not only creates a Null0 route automatically but also suppresses more-specific routes from the BGP RIB. Using only the aggregate-address summarization, upstream peers will only receive the aggregated route and not the individual more-specific prefixes.

! on RR
router bgp 400
 aggregate-address 100.100.0.0 255.255.0.0 summary-only
!
RR#sh ip bgp
BGP table version is 31, local router ID is 100.100.254.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
          r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*> 100.100.0.0/16   0.0.0.0                            32768 i
s i100.100.200.0/24 100.100.254.3            0    100      0 65501 i
s>i                 100.100.254.1            0    100      0 65501 i
s>i100.100.255.0/31 100.100.254.1            0    100      0 i
s>i100.100.255.8/31 100.100.254.3            0    100      0 i

AS200#sh ip bgp
BGP table version is 37, local router ID is 100.100.253.0
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
          r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*> 100.100.0.0/16   100.100.253.1            0             0 400 i

As you can see, after using the aggregate-address command on AS 400 RR, only our configured summarized address is advertised out to AS200. You can also see the suppressed routes in RR’s BGP RIB, since we used the “summary-only” parameter in the aggregate-address command. All more-specific routes are suppressed from being advertised to BGP peers thus reducing what used to be many routes to just what’s configured.

Aggregation, combined with proper filtering, should be performed wherever and whenever possible. As of today, CIDR Report indicates over 410,000 routes exist in the global table. With aggregation (as estimated by CIDR Report), as much as half of all the routes in existence today can be aggregated.