Monday, August 31, 2015

Cisco IP SLA - Investigating IP Service Levels & Change Routing

Cisco IP SLAs is a part of Cisco IOS that allows Cisco customers to analyze IP service levels for IP applications and services by using active traffic monitoring for measuring network performance. With Cisco IOS IP SLAs, service provider customers can measure and provide service level agreements, and enterprise customers can verify service levels, verify outsourced service level agreements, and understand network performance. Cisco IOS IP SLAs can perform network assessments, verify quality of service (QoS), ease the deployment of new services, and assist with network troubleshooting.

IP SLAs sends data across the network to measure performance between multiple network locations or across multiple network paths. It simulates network data and IP services and collects network performance information in real time. Cisco IOS IP SLAs generates and analyzes traffic either between Cisco IOS devices or from a Cisco IOS device to a remote IP device such as a network application server. Measurements provided by the various Cisco IOS IP SLAs operations can be used for troubleshooting, for problem analysis, and for designing network topologies. 

IP SLAs collects a unique subset of these performance metrics:

  • Delay (both round-trip and one-way)
  • Jitter (directional)
  • Packet loss (directional)
  • Packet sequencing (packet ordering)
  • Path (per hop)
  • Connectivity (directional)
  • Server or website download time
In this article, I will use the ICMP Echo operation measures end-to-end response time between a Cisco router and a web server using IP. Response time is computed by measuring the time taken between sending an ICMP Echo request message to the destination and receiving an ICMP Echo reply.

Example

Suppose that the CiscoEXM router has two different links, one is the main connection (red link) and the other one (blue link) is the backup connection; the question is: how can I enable the backup link if the main connection goes down? In general, the best solution for this scenario is to use the dynamic routing protocol, but what can I do if I can’t use them? The solution is the IP SLA.



 1. Define the ip sla operation. The CiscoEXM router will send an ICMP request to 172.16.255.2 (the CiscoEXM default gateway) every 10 second with a timeout of 1000ms and a threshold value of 500ms.

CiscoEXM(config)#ip sla 1
CiscoEXM(config-ip-sla)#icmp-echo 172.16.255.2 source-interface FastEthernet1/0
CiscoEXM(config-ip-sla-echo)#timeout 5000
CiscoEXM(config-ip-sla-echo)#frequency 10
CiscoEXM(config-ip-sla-echo)#threshold 500

 2. Start the ip sla. It is possible schedule the SLA operation in different ways but in this tutorial I want to start the ip SLA operation immediately and forever. Notice that the “1”  refers to “ip sla 1” command.

CiscoEXM(config)#ip sla schedule 1 start-time now life forever

3 . Track the state of IP SLA. Every IP SLAs operation maintains an operation return-code value. This return code is interpreted by the tracking process. The return code may return OK, Over Threshold, and several other return codes. Two aspects of an IP SLAs operation can be tracked: state and reach-ability.


 In this case, I prefer to use the “reachability”, so the “track state” will be down only in case of a ICMP timeout.

CiscoEXM(config)#track 10 ip sla 1 reachability

Remember: If you want to use the “state”, remember that the “track state” will be down also if the the threshold is reached.

Note: with Cisco IOS Release 12.4(20)T, 12.2(33)SXI1, 12.2(33)SRE and Cisco IOS XE Release 2.4, the track rtr command is replaced by the track ip sla command. See the track ip sla command for more information.

 4. Define the tracked route. At the end, I must delete the old default gateway entry, add the default gateway with the track feature (notice that the number 10 represents the track object defined in the previous step) and insert a default route with a distance administrative less “strong” . Hence if the track status is down the last route will be used to forward all the traffic (notice that the number 5 define the administrative distance).

CiscoEXM(config)#ip route 0.0.0.0 0.0.0.0 172.16.255.2 track 10
CiscoEXM(config)#no ip route 0.0.0.0 0.0.0.0 172.16.255.2
CiscoEXM(config)#ip route 0.0.0.0 0.0.0.0 172.16.255.6 5

 5. Check the IP SLA.
Now that I have defined the IP SLA object, I can check some useful informations when the main link (red link) is UP or NOT.

Red link UP
To display information about the IP route track table:

CiscoEXM#show ip route track-table
 ip route 0.0.0.0 0.0.0.0 172.16.255.2 track 10 state is [up]
CiscoEXM#

To display information about the IP routing table:

CiscoEXM#show ip route 
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is 172.16.255.2 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 172.16.255.2
      172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
C        172.16.255.0/30 is directly connected, FastEthernet1/0
L        172.16.255.1/32 is directly connected, FastEthernet1/0
C        172.16.255.4/30 is directly connected, FastEthernet1/1
L        172.16.255.5/32 is directly connected, FastEthernet1/1
  
To display information about IP SLA

CiscoEXM#show track
Track 10
  IP SLA 1 reachability
  Reachability is Up
    12 changes, last change 00:33:48
  Latest operation return code: OK
  Latest RTT (millisecs) 24
  Tracked by:
    STATIC-IP-ROUTING 0
 
Red link DOWN
First of all, I ping the web server (192.168.1.10) that it is on the headquarter, then I unplug the CiscoEXM fastethernet1/0 cable

CiscoEXM#ping 192.168.1.10 repeat 200
Type escape sequence to abort.
Sending 200, 50-byte ICMP Echos to 192.168.1.10, timeout is 2 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!....
May 8 03:23:30.082: %TRACKING-5-STATE: 10 ip sla 1 reachability Up->Down.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 97 percent (195/200), round-trip min/avg/max = 20/66/248 ms
CiscoEXM#


As you can see, when the cable is unplugged there are three different events:

  1. there are some timeout
  2. the tracking state goes down (May 8 03:23:30.082: %TRACKING-5-STATE: 10 ip sla 1 reachability Up->Down)
  3. the route tracked goes down and the backup default route (ip route 0.0.0.0 0.0.0.0 172.16.255.6 5) is up.
So the output of the previous show commands will be:

The track object is down:

CiscoEXM#show ip route track-table
ip route 0.0.0.0 0.0.0.0 172.16.255.2 track 10 state is [down]

The default route is 172.16.255.6 (the backup connection):
 
CiscoEXM#show ip route 
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is 172.16.255.6 to network 0.0.0.0

S*    0.0.0.0/0 [5/0] via 172.16.255.6
      172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
C        172.16.255.0/30 is directly connected, FastEthernet1/0
L        172.16.255.1/32 is directly connected, FastEthernet1/0
C        172.16.255.4/30 is directly connected, FastEthernet1/1
L        172.16.255.5/32 is directly connected, FastEthernet1/1

The return code is “Timeout”: 

CiscoEXM#show ip sla statistics 
IPSLAs Latest Operation Statistics

IPSLA operation id: 1
        Latest RTT: NoConnection/Busy/Timeout
Latest operation start time: 10:42:03 UTC Wed May 8 2013
Latest operation return code: Timeout
Number of successes: 8
Number of failures: 3
Operation time to live: Forever


The track object is down:

CiscoEXM#show track
Track 10
  IP SLA 1 reachability
  Reachability is Down
    13 changes, last change 00:00:22
  Latest operation return code: Timeout
  Tracked by:
    STATIC-IP-ROUTING 0

Red link again UP
At this point I ping the web server (192.168.1.10) , then I reconnect the CiscoEXM fastethernet1/0 cable

CiscoEXM#ping 192.168.1.10 size 50 repeat 200
Type escape sequence to abort.
Sending 200, 50-byte ICMP Echos to 192.168.1.10, timeout is 2 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (200/200), round-trip min/avg/max = 12/71/172 ms
CiscoEXM#



CiscoEXM#show ip route 
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is 172.16.255.2 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 172.16.255.2
      172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
C        172.16.255.0/30 is directly connected, FastEthernet1/0
L        172.16.255.1/32 is directly connected, FastEthernet1/0
C        172.16.255.4/30 is directly connected, FastEthernet1/1
L        172.16.255.5/32 is directly connected, FastEthernet1/1

As you can see, when the main line comes up (now the default gateway is again 172.16.255.2), there isn’t a packet lost.

*** Note: Switches running the IP base image support only IP SLAs responder functionality and must be configured with another device that supports full IP SLAs functionality, for example, a switch. 

**** Useful commands for Monitoring IP SLAs Operations

Command

Purpose

show ip sla application

Display global information about Cisco IOS IP SLAs.

show ip sla authentication

Display IP SLAs authentication information.

show ip sla configuration [entry-number]

Display configuration values including all defaults for all IP SLAs operations or a specific operation.

show ip sla enhanced-history {collection-statistics | distribution statistics} [entry-number]

Display enhanced history statistics for collected history buckets or distribution statistics for all IP SLAs operations or a specific operation.

show ip sla ethernet-monitor configuration [entry-number]

Display IP SLAs automatic Ethernet configuration.

show ip sla group schedule [schedule-entry-number]

Display IP SLAs group scheduling configuration and details.

show ip sla history [entry-number | full | tabular]

Display history collected for all IP SLAs operations

show ip sla mpls-lsp-monitor {collection-statistics | configuration | ldp operational-state | scan-queue | summary [entry-number] | neighbors}

Display MPLS label switched path (LSP) Health Monitor operations,

show ip sla reaction-configuration [entry-number]

Display the configured proactive threshold monitoring settings for all IP SLAs operations or a specific operation.

show ip sla reaction-trigger [entry-number]

Display the reaction trigger information for all IP SLAs operations or a specific operation.

show ip sla responder

Display information about the IP SLAs responder.

show ip sla statistics [entry-number | aggregated | details]

Display current or aggregated operational status and statistics.


References:

Tuesday, August 11, 2015

VMware ESXi 5.5 / 6.0 GA "Driver Not Found" problems solution for HP P440ar RAID controllers on New Gen9 Servers.


Symptoms

When configuring and using ESXi 5.5 / 6.0 Scripted Install there are several factors that can cause an installation to fail. Reviewing the log files and any on-screen prompts can often indicate the root cause of the issue. This article is to help you troubleshoot these issues.
Basically, The ESXi 5.5/6.0 GA ISO provided by VMware does not have the latest HP Smart Array driver that supports HP Gen 9 Smart Array controllers.


As a result, an ESXi installation or upgrade to ESXi 5.5/6.0 may fail, you see errors similar to:


Error (see log for more info)
An error has occurred while parting the installer script

error:file:///ks.cfg:line 3: Error (see log for more infor):
Could not find first boot partition

 SOLUTION:

 *** If you face any problem regarding download links given below then please leave a comment in this post, I'll fix that as soon as possible.

Your kind cooperation will help me as well as other readers. ***

1. Install the host using HP's custom ESXi 6.0 GA installation media which includes the scsi-hpsa-5.5.0.74-1OEM.550.0.0.1331820 driver.
To install the host using the HP ESXi 6.0 Custom Image, download the ISO from VMware downloads and install ESXi as detailed in the Installing ESXi section in the vSphere 6.0 Installation and Setup guide.
 


2. Create a custom ISO and inject the updated driver separately.

To manually create a custom ESXi 6.0 image, inject the driver and install the host:

  • Create a custom ESXi build using the ESXi-Customizer-v2.7.2 software.

    Note: ESXi-Customizer is not a VMware product. You can download the software at ESXi-Customizer Downloads page.
  • Replace the current version of the driver 6.0.0.44-4vmw.600.0.0.2494585 with version scsi-hpsa-5.5.0.74-1OEM.550.0.0.1331820.x86_64.vib. 
  •  
 To replace the driver:


  1. Download the driver vib file from Drivers & software ESXI 6.0.Drivers & software ESXI 5.5.0

    Note: The preceding link was correct as of December 17, 2015. If you find the link is broken, provide feedback and a VMware employee will update the link.
  2. Run this command to install the driver:
    esxcli software vib install -d path_to_vib

Other Common Errors & Solutions:



This table outlines common errors and how to troubleshoot them:


Error Solution
error:nfs://192.168.0.1/FileShare
/ks.cfg:line12: install --firstdisk
specified, but no suitable disk found
This error can occur if there is insufficient disk space on the host to install ESXi. The minimum requirement for the size of the boot disk is 900 MB. Ensure that your disk meets this requirement.
NFS mount failure for URL nfs://192.168.0.101:/NFS/ks1.cfg
Log onto the host and examine /var/log/weasel.log. This reveals a more informative message. According to the logs, you see the error:
Could not open file over NFS nfs://192.168.0.101:/NFS/ks1.cfg
Exception: Error (see log for more info):
Error copy / download file
This means the ks1.cfg file was not found. Either the file was not created or the mount point is incorrect. Ensure that the file exists and that the share has been created.
Did not get an IP Address from DHCP server
This error indicates that the host cannot contact a DHCP server. The host requires an IP address to be able to mount the NFS share containing the ks.cfg file.
If you do not specify IP information at the boot command. If no network choices were specified at the command line, the installer defaults to DHCP. If there is no DHCP server on the network, then the installer has no way of retrieving files over the network. Ensure your DHCP server is working correctly.
In environment where there is no DHCP server available, static network configuration entries need to be added to the boot command. A network configuration must be specified to allow access to the ks.cfg file if no DHCP server is available on the network. This can be achieved by specifying these parameters at the boot prompt:
ip=, netmask=, gateway=
warning:nfs://192.168.0.101/
NFS/ks.cfg:line 5: bootproto was
set to static but "--hostname="
was not set. Hostname will be set automatically.
This is not an error, but a warning that temporarily halts installation. When setting a static IP, specify a hostname to avoid this warning.

Saturday, July 4, 2015

BGP - A Detail Explanation



Border Gateway Protocol (BGP)

BGP is a protocol which performs routing information exchange among routers to determine the optimal paths for the traffic flow. A BGP router forms a neighbor relationship by connecting to its neighbors and exchanging the routes, once the connection is established. 

Without BGP the Internet as we know it would be quite a bit more inefficient. As it is today the Internet BGP routing tables have over 300,000 active forwarding entries and this is with stigmatization of over 2 billion addresses. Imagine what these tables would be like without stigmatization.

For those interested, some ISP’s allow the ability to telnet into their edge BGP routers to view the BGP routing tables (Check out ‘route-server.ip.att.net’).


BGP's purpose is not only to exchange its information, but also to exchange network reach-ability and availability information for the Autonomous Systems (AS) paths with other BGP systems on the network. This process allows all systems to construct topology graphs of the entire network infrastructure on both sides of the BGP link.

This process also allows these systems to identify loops and other issues that may affect network performance and availability.





BGP Overview


·  BGP uses the concept of autonomous systems (AS). An autonomous system is a group of networks under a common administration. The Internet Assigned Numbers Authority (IANA) assigns AS numbers: 1 to 64511 are public AS numbers and 64512 to 65535 are private AS numbers.

·  Autonomous systems run Interior Gateway Protocols (IGP) within the system. They run an Exterior Gateway Protocol (EGP) between them. BGP version 4 is the only EGP currently in use.

·  Routing between autonomous systems is called interdomain routing.

·  The administrative distance for EBGP routes is 20. The administrative distance for IBGP routes is 200.

·  BGP neighbors are called peers and must be statically configured.

·  BGP uses TCP port 179. BGP peers exchange incremental, triggered route updates and periodic keepalives.

·  Routers can run only one instance of BGP at a time.

·  BGP is a path-vector protocol. Its route to a network consists of a list of autonomous systems on the path to that network.

·  BGP's loop prevention mechanism is an autonomous system number. When an update about a network leaves an autonomous system, that autonomous system's number is prepended to the list of autonomous systems that have handled that update. When an autonomous system receives an update, it examines the autonomous system list. If it finds its own autonomous system number in that list, the update is discarded.

 

BGP Databases

BGP uses three databases. The first two listed are BGP-specific; the third is shared by all routing processes on the router:

  • Neighbor database: A list of all configured BGP neighbors. To view it, use the show ip bgp summary command.
  • BGP database, or RIB (Routing Information Base): A list of networks known by BGP, along with their paths and attributes. To view it, use the show ip bgp command.
  • Routing table: A list of the paths to each network used by the router, and the next hop for each network. To view it, use the show ip route command.

 

BGP Message Types

BGP has four types of messages:

  • Open: After a neighbor is configured, BGP sends an open message to try to establish peering with that neighbor. Includes information such as autonomous system number, router ID, and hold time.
  • Update: Message used to transfer routing information between peers. Includes new routes, withdrawn routes, and path attributes.
  • Keepalive: BGP peers exchange keepalive messages every 60 seconds by default. These keep the peering session active.
  • Notification: When a problem occurs that causes a router to end the BGP peering session, a notification message is sent to the BGP neighbor and the connection is closed.

BGP Routing

As with any routing protocol, BGP maintains routing tables, transmits routing updates, and bases routing decisions on routing metrics. The primary function of a BGP system is to exchange network-reach ability information, including information about the list of autonomous system paths, with other BGP systems. This information can be used to construct a graph of autonomous system connectivity from which routing loops can be pruned and with which autonomous system-level policy decisions can be enforced.
Each BGP router maintains a routing table that lists all feasible paths to a particular network. The router does not refresh the routing table, however. Instead, routing information received from peer routers is retained until an incremental update is received.
BGP devices exchange routing information upon initial data exchange and after incremental updates. When a router first connects to the network, BGP routers exchange their entire BGP routing tables. Similarly, when the routing table changes, routers send the portion of their routing table that has changed. BGP routers do not send regularly scheduled routing updates, and BGP routing updates advertise only the optimal path to a network.

BGP Operation

BGP performs three types of routing: interautonomous system routing, intra-autonomous system routing, and pass-through autonomous system routing.

Interautonomous system routing occurs between two or more BGP routers in different autonomous systems. Peer routers in these systems use BGP to maintain a consistent view of the internetwork topology. BGP neighbors communicating between autonomous systems must reside on the same physical network. The Internet serves as an example of an entity that uses this type of routing because it is comprised of autonomous systems or administrative domains. Many of these domains represent the various institutions, corporations, and entities that make up the Internet. BGP is frequently used to provide path determination to provide optimal routing within the Internet.

Intra-autonomous system routing occurs between two or more BGP routers located within the same autonomous system. Peer routers within the same autonomous system use BGP to maintain a consistent view of the system topology. BGP also is used to determine which router will serve as the connection point for specific external autonomous systems. Once again, the Internet provides an example of interautonomous system routing. An organization, such as a university, could make use of BGP to provide optimal routing within its own administrative domain or autonomous system. The BGP protocol can provide both inter- and intra-autonomous system routing services.

Pass-through autonomous system routing occurs between two or more BGP peer routers that exchange traffic across an autonomous system that does not run BGP. In a pass-through autonomous system environment, the BGP traffic did not originate within the autonomous system in question and is not destined for a node in the autonomous system. BGP must interact with whatever intra-autonomous system routing protocol is being used to successfully transport BGP traffic through that autonomous system.   

Figure: In pass-through autonomous system routing, BGP pairs with another intra-autonomous system-routing protocol.



What is the difference between eBGP and iBGP?

1. EBGP is peering between two different AS, whereas IBGP is between same AS (Autonomous System).


2. Routes learned from eBGP peer will be advertised to other peers (BGP or IBGP); however, routes learned from IBGP peer will not be advertised to other IBGP peers.


3. By default, EBGP peers are set with TTL = 1, which means neighbors are assumed to be directly connected, which is not in the case of IBGP. We can change this behavior for EBGP by using command “neighbor x.x.x.x ebgp-multihop <TTL>”. Multihop is the term used in EBGP only.


4. EBGP routes have administrative distance of 20, whereas IBGP has 200.


5. Next hop remains unchanged when route is advertised to IBGP peer; however, it is changed when it is advertised to EBGP peer by default.

This default behavior of IBGP can be changed by the command “neighbor x.x.x.x next-hop-self”; this changes the next hop, while advertising, as a local route.
 

Configuring BGP

In its most basic configuration BGP acts very similarly to a distance vector routing protocol. Each network which is advertised is selected by choosing the shortest path. BGP just uses a path (Autonomous Systems – AS) hop count instead of a device hop count.
For example, BGP works by routing traffic between AS’s, so if Verizon had AS 12345 and AT&T had AS 54321 then traffic destined for the AT&T network would be routed from Verizon to AT&T with a path of (54321). What this means is in order to reach a specific network on the AT&T network, traffic on the Verizon network would have to be routed to AS 54321.
In the following figure I show three AS’s and their corresponding path metrics:







BGP also has loop prevention built in (although this is an open debate); this is implemented with a simple mechanism which disallows routes to be advertised into an AS if the local AS is part of the path metric.

With the example above, only the most basic AS_Path metric is shown, however there are many different path attributes which can be used with BGP to affect path selection along with AS_Path.



Basic BGP Configuration

The first thing that must be understood is that each BGP device can have both internal and external BGP connections to other devices. Internal BGP connections are within the same AS while external BGP connections are between different AS’s. This is important because the configuration and behavior is slightly different between the two.


eBGP Configuration

 
At its most basic the configuration of eBGP requires only two commands, these include:

  1. router bgp as-number
  2. neighbor ip-address remote-as remote-as-number
What makes eBGP configuration obvious from iBGP configuration is that the AS-number which is used in the neighbor command is different than the AS-number configured with the router bgp command.
It must also be known that with eBGP by default there is a direct connection requirement which is enforced by an advertised TTL of 1. Now when configuring BGP using loopback interfaces this can become an issue as the packet actually takes two hops from the remote device to the physical interface and from the physical interface to the loopback interface.
This issue can be resolved by using the neighbor ebgp-multihop command on Cisco equipment.



iBGP Configuration

 
iBGP configuration is very similar to eBGP configuration but requires a little understanding of iBGP requirements. By default, iBGP requires that all iBGP devices being used are fully meshed (although there are ways of getting around this). This does not however mean that a direct connection is required but that each iBGP peer must neighbor with each other iBGP router.
The following configuration shows that configuration of an iBGP neighbor is the same as with eBGP:

  1. router bgp as-number
  2. neighbor ip-address remote-as remote-as-number
The other thing that must be understood is how external BGP routes are advertised into iBGP. See the following figure:





In this example, when Level3-2 advertises the eBGP route for the 192.168.128.0/17 network to Level3-1 it will do this with a next-hop of 10.10.10.1 by default. Now if Level3-1 does not have a valid route to the 10.10.10.1 address then it will be unable to route traffic destined for the 192.168.128.0/17 network.


The most common method of resolving this issue is by using the neighbor neighbor-ip-address next-hop-self-command. When using this command the local eBGP peer will advertise the next-hop with its own IP address and not the address configured with the BGP neighbor command.


In this case, Level3-2 could be configured with the neighbor 10.100.100.2 next-hop-self-command which would advertise the 192.168.128.0/17 network with a next-hop of 10.100.100.1 instead of 10.10.10.1.

BGP Next-Hop Selection

The next hop for a route received from an EBGP neighbor is the IP address of the neighbor that sent the update.
When a BGP router receives an update from an EBGP neighbor, it must pass that update to its IBGP neighbors without changing the next-hop attribute. The next-hop IP address is the IP address of an edge router belonging to the next-hop autonomous system. Therefore, IBGP routers must have a route to the network connecting their autonomous system to that edge router. For example, in Figure, RtrA sends an update to RtrB, listing a next hop of 10.2.2.1, its serial interface. When RtrB forwards that update to RtrC, the next-hop IP address will still be 10.2.2.1. RtrC needs to have a route to the 10.2.2.0 network to have a valid next hop.






To change this behavior, use the neighbor [ip address] next-hop-self-command in BGP configuration mode. In Figure, this configuration goes on RtrB. After you give this command, RtrB advertises its IP address to RtrC as the next hop for networks from AS 65100, rather than the address of RtrA. Thus, RtrC does not have to know about the external network between RtrA and RtrB (network 10.2.2.0).


How does BGP select the best routing path /

BGP Attributes


BGP Best Path Selection Algorithm determines the best route by selecting the shortest path to the destination. An Autonomous System is a single network or a set of networks and routers, which are under the control of one administrative entity. Nevertheless, network administrators frequently manipulate such options as local preference, lowest multi-exit discriminator and weight.


The list of the selection criteria is presented below in the same order in which BGP uses them to select the optimal routes to be injected into the IP Routing table:


1) Weight — weight is the first criterion used by the router and it is set locally on the user’s router. The Weight is not passed to the following router updates. In case there are multiple paths to a certain IP address, BGP always selects the path with the highest weight. The weight parameter can be set either through neighbor command, route maps or via the AS-path access list.

Figure: BGP Weight Attribute

In Figure: BGP Weight Attribute, Router A is receiving an advertisement for network 172.16.1.0 from routers B and C. When Router A receives the advertisement from Router B, the associated weight is set to 50. When Router A receives the advertisement from Router C, the associated weight is set to 100. Both paths for network 172.16.1.0 will be in the BGP routing table, with their respective weights. The route with the highest weight will be installed in the IP routing table.


2) Local Preference — this criterion indicates which route has local preference and BGP selects the one with the highest preference. Local Preference default is 100.

Figure: BGP Local Preference Attribute


In Figure: BGP Local Preference Attribute, AS 100 is receiving two advertisements for network 172.16.1.0 from AS 200. When Router A receives the advertisement for network 172.16.1.0, the corresponding local preference is set to 50. When Router B receives the advertisement for network 172.16.1.0, the corresponding local preference is set to 100. These local preference values will be exchanged between routers A and B. Because Router B has a higher local preference than Router A, Router B will be used as the exit point from AS 100 to reach network 172.16.1.0 in AS 200.


3) Network or Aggregate — this criterion chooses the path that was originated locally via an aggregate or a network, as the aggregation of certain routes in one is quite effective and helps to save a lot of space on the network.


4) Shortest AS_PATH — this criterion is used by BGP only in case it detects two similar paths with nearly the same local preference, weight and locally originated or aggregate addresses.

Figure: BGP AS_PATH Attribute


AS1 originates the route to 172.16.1.0 and advertises this route to AS 2 and AS 3, with the AS_path attribute equal to {1}. AS 3 will advertise back to AS 1 with AS-path attribute {3,1}, and AS 2 will advertise back to AS 1 with AS-path attribute {2,1}. AS 1 will reject these routes when its own AS number is detected in the route advertisement. This is the mechanism that BGP uses to detect routing loops. AS 2 and AS 3 propagate the route to each other with their AS numbers added to the AS_path attribute. These routes will not be installed in the IP routing table because AS 2 and AS 3 are learning a route to 172.16.1.0 from AS 1 with a shorter AS_path list.


5) Lowest origin type — this criterion assigns higher preference to Exterior Gateway Protocol (EGP) and lower preference to Interior Gateway Protocol (IGP).


6) Lowest multi-exit discriminator (MED) — this criterion, representing the external metric of a route, gives preference to the lower MED value.

Figure: BGP Multi-Exit Discriminator Attribute

In Figure: BGP Multi-Exit Discriminator Attribute, Router C is advertising the route 172.16.1.0 with a metric of 10, while Route D is advertising 172.16.1.0 with a metric of 5. The lower value of the metric is preferred, so AS 100 will select the route to router D for network 172.16.1.0 in AS 200. MEDs are advertised throughout the local AS.

7) eBGP over iBGP — just like the “Lowest origin type” criterion, this criterion prefers eBGP rather than iBGP.


8) Lowest IGP metric — this criterion selects the path with the lowest IGP metric to the BGP next hop.


9) Multiple paths — this criterion serves as indication whether multiple routes need to be installed in the routing table.


10) External paths — out of several external paths, this criterion selects the first received path.


11) Lowest router ID — this criterion selects the path which connects to the BGP router that has the lowest router ID.


12) Minimum cluster list — in case multiple paths have the same router ID or originator, this criterion selects the path with the minimum length of the cluster list.


13) Lowest neighbor address — this criterion selects the path, which originates from the lowest neighbor address.



Example Configuration of BGP

To start BGP on a Cisco router, the following command is required:


router bgp autonomous system number
 
To define networks to be advertised, apply the following command:


network network-number mask network-mask
 
You must be aware that the network command is not used in the same way you use it when you apply networks in OSPF or EIGRP. With BGP, you use the network command to advertise networks that originate from the router and need to be advertised through BGP.
To identify peer routers, apply the following command:


neighbor ip-address | peer-group name remote-as autonomous system number
 
Next, we will see how to configure IBGP and EBGP among the three routers in below Figure.







Example-1 displays the IBGP configuration on R1 to R2.


Example-1. IBGP on R1




R1(config)#router bgp ?
  <1-65535>  Autonomous system number
R1(config)#router bgp 1
R1(config-router)#neighbor 131.108.1.2 remote 1 
 
Example-2 displays the IBGP configuration to R1 and EBGP configuration to R3.


Example-2. IBGP/EBGP on R2


R2(config)#router bgp 1
R2(config-router)#neighbor 131.108.1.1  remote-as 1
R2(config-router)#neighbor 131.108.255.2 remote-as 2
 
Finally, Example-3 displays the EBGP connection from R3 to R2.


Example-3. EBGP on R3


R3(config)#router bgp ?
  <1-65535>  Autonomous system number
R3(config)#router bgp 2
R3(config-router)#neighbor 131.108.255.1 remote-as 1
 
At this stage, because no network statements have been applied, no BGP entries are on any routers. Use some loopback interfaces on R1 and advertise them through BGP to R2 and R3.


Example-4 displays the three new loopback addresses on R1, ranging from 131.108.2.0 to 131.108.4.0.


Example-4. Loopback Configuration on R1


R1(config)#interface loopback 0
R1(config-if)#ip address 131.108.2.1 255.255.255.0
R1(config-if)#interface loopback 1
R1(config-if)#ip address 131.108.3.1 255.255.255.0
R1(config-if)#interface loopback 2
R1(config-if)#ip address 131.108.4.1 255.255.255.0
 
We must next advertise these loopbacks with the network command. Because these networks are local to R1 and present in R1's IP routing table as connected routes, you can apply the network command as displayed in Example-5.


Example-5. network Command on R1


R1(config)#router bgp 1
R1(config-router)#network 131.108.2.0 mask 255.255.255.0
R1(config-router)#network 131.108.3.0 mask 255.255.255.0
R1(config-router)#network 131.108.4.0 mask 255.255.255.0



Example-6 displays the BGP table on R1, using the command show ip bgp.
 

Example-6. show ip bgp on R1

R1#show ip bgp
BGP table version is 4, local router ID is 131.108.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
 
   Network          Next Hop            Metric LocPrf Weight Path
*> 131.108.2.0/24   0.0.0.0                  0         32768 i
*> 131.108.3.0/24   0.0.0.0                  0         32768 i
*> 131.108.4.0/24   0.0.0.0                  0         32768 i

The BGP table on R1 displays three local networks (next hop is 0.0.0.0 or local interfaces). Example-6 also displays the path as i, or advertised through BGP. The local router ID is 131.108.1.1.




Example-7 displays the BGP table on R2.


Example-7. show ip bgp on R2
 
R2#show ip bgp 
BGP table version is 7, local router ID is 171.109.3.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
 
   Network          Next Hop            Metric LocPrf Weight Path
* i131.108.2.0/24   131.108.1.1              0    100      0 i
* i131.108.3.0/24   131.108.1.1              0    100      0 i
* i131.108.4.0/24   131.108.1.1              0    100      0 i
 
R2's local router is 131.108.1.2, and it learns the remote loopbacks on R1 through the next hop address 131.108.1.1, or R1's Ethernet interface. Notice that R2 has set the local preference to 100 (default value); the origin attribute is set to i or IGP.

Because R1 and R2 are running only IBGP and no other interior gateway protocol, R2's IP routing table does not have the BGP entries inserted because of synchronization. 

Example-8 confirms this with only the locally connected routes visible on R2.


Example-8. show ip route on R2


R2#show ip route
        131.108.0.0/24 is subnetted, 2 subnets
C       131.108.255.0 is directly connected, Serial1/0
C       131.108.1.0 is directly connected, Ethernet0/0
 
To enable BGP to insert the routes, you must disable synchronization or configure an IGP routing protocol. R2, in turn, does not propagate the loopbacks to R3; therefore, R3 does not have any entries at all, either in the BGP table or IP routing table.
Disable synchronization on R1 and R2.

 Example-9 displays the no synchronization command on R1 and R2.


Example-9. Disabling Synchronization on R1/R2


R1(config)#router bgp 1
R1(config-router)#no synchronization
 
R2(config)#router bgp 1
R2(config-router)#no synchronization
 
Example-10 displays R2's routing table.

 

Example-10. R2's Routing Table


R2#sh ip route
        131.108.0.0/24 is subnetted, 5 subnets
C       131.108.255.0 is directly connected, Serial1/0
B       131.108.4.0 [200/0] via 131.108.1.1, 00:00:43
B       131.108.3.0 [200/0] via 131.108.1.1, 00:00:43
B       131.108.2.0 [200/0] via 131.108.1.1, 00:00:43
C       131.108.1.0 is directly connected, Ethernet0/0
 
The three remote networks are inserted into the IP routing tables as BGP-learned networks.

Example-11 displays R3's BGP and IP routing table.


Example-11. R3's BGP and IP Tables


R3>show ip bgp
BGP table version is 10, local router ID is 131.108.255.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP,? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*> 131.108.2.0/24   131.108.255.1                          0 1 i
*> 131.108.3.0/24   131.108.255.1                          0 1 i
*> 131.108.4.0/24   131.108.255.1                          0 1 i
 
R3>show ip route
     131.108.0.0/24 is subnetted, 5 subnets
C       131.108.255.0 is directly connected, Serial0
B       131.108.4.0 [20/0] via 131.108.255.1, 00:02:09
B       131.108.3.0 [20/0] via 131.108.255.1, 00:02:09
B       131.108.2.0 [20/0] via 131.108.255.1, 00:02:09
C       131.108.1.0 is directly connected, Ethernet0
 
Notice that the next hop address on R3 is R2. The AS path on R3 indicates that the remote networks, 131.108.2.0 to 131.108.4.0/24, transverse autonomous system number 1, as displayed in the BGP table in Example-11.


Verifying BGP Operation

This final scenario looks at Cisco IOS mechanisms for monitoring and verifying BGP routing within a Cisco router network.
Refer to Figure and the BGP topology to see how to use some common show commands to verify that BGP is operating correctly.
Show and debug commands can be valuable, not only in the real-life networks you come across, but also during your certification exams.
This scenario covers the following commands:

  • show ip bgp summary— Displays BGP neighbors in summary mode
  • show ip bgp— Displays the BGP topology table
  • clear ip bgp *— Clears all BGP TCP sessions
  • show tcp brief— Displays all TCP sessions (BGP uses TCP)
  • debug ip bgp events— Displays any BGP events, such as neighbor state changes
  •  
Example 1 displays a sample output taken from R1 in Figure using the IOS show ip bgp summary command.

Example 1. show ip bgp summary on R1

R1#show ip bgp summary
BGP router identifier 131.108.255.13, local AS number 1
BGP table version is 11, main routing table version 11
6 network entries and 10 paths using 854 bytes of memory
3 BGP path attribute entries using 280 bytes of memory
BGP activity 50/44 prefixes, 73/63 paths
 
Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
131.108.1.2     4     1     194     195       11    0    0 00:03:22        2
131.108.255.6   4     1      84      83       11    0    0 00:03:23        3
131.108.255.14  4     1     152     152       11    0    0 00:03:23        3
141.199.1.1     4  1001       0       0        0    0    0 never    Idle
 
Example 1 displays a lot of useful information, including the local router identifier 131.108.255.13, the local AS of 1, and the BGP table version of 11. (An increasing version number indicates a network change is occurring; if no changes occur, this number remains the same.) It also shows six network paths on R1, using 854 bytes of memory.

Memory is important in BGP because in a large network, such as the Internet, memory can be a limiting factor. As more BGP entries populate the IP routing table, more memory is required.

Example 2 show ip bgp



R1>show ip bgp
 
BGP table version is 11, local router ID is 131.108.255.13
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*>i131.108.255.0/30 131.108.1.2              0    100   1000 i
* i                 131.108.255.14           0    100   1000 i
*> 131.108.255.4/30 0.0.0.0                  0         32768 i
* i                 131.108.255.6            0    100   1000 i
*>i131.108.255.8/30 131.108.1.2              0    100   1000 i
* i                 131.108.255.6            0    100   1000 i
*> 131.108.255.12/30 0.0.0.0                 0         32768 i
* i                 131.108.255.14           0    100   1000 i
*>i141.108.1.0/24   131.108.255.6            0    100   1000 i
*>i151.108.1.0/24   131.108.255.14           0    100   1000 i
 
Again, the BGP table version is displayed as 11 and the local router ID is 131.108.255.13. The various networks are listed along with the next hop address, metric (MED), local preference (Locpref), weight, and the path. On the left side (part of the status codes) indicates an internal BGP route and the i on the right side of Example 2 indicates the origin. (i is for IGP, part of the origin codes.)

If a BGP configuration change is completed on Cisco IOS routers, the BGP peer session must be cleared. The command to clear all sessions is clear ip bgp *. To clear a single peer router, use the clear ip bgp peer-ip-address command.


Example 3. clear ip bgp * and show ip bgp on R1



R1#clear ip bgp ?
  
   *                Clear all connections
  <1-65535>        AS number of the peers
  A.B.C.D          BGP neighbor address to clear
  dampening        Clear route flap dampening information
  flap-statistics  Clear route flap statistics
  peer-group       Clear BGP connections of peer-group
 
R1#clear ip bgp *
R1#show ip bgp
 
BGP table version is 11, local router ID is 131.108.255.13
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
 
   Network          Next Hop            Metric LocPrf Weight Path
*>i131.108.255.0/30 131.108.1.2              0    1000   1000 i
* i                 131.108.255.14           0    1000  1000 i
*> 131.108.255.4/30 0.0.0.0                  0         32768 i
* i                 131.108.255.6            0    1000   1000 i
*>i131.108.255.8/30 131.108.1.2              0    1000   1000 i
* i                 131.108.255.6            0    1000   1000 i
*> 131.108.255.12/30 0.0.0.0                 0         32768 i
* i                 131.108.255.14           0    1000   1000 i
*>i141.108.1.0/24   131.108.255.6            0    1000   1000 i
*>i151.108.1.0/24   131.108.255.14           0    100   1000 i
 
This tool displays a number of options, including clearing BGP sessions based on AS numbers or remote peer address. On Cisco IOS routers, you must clear the BGP sessions if you want a change to take place because BGP does not update changes after a BGP session is established. You can, however, configure soft configurations with the neighbor peer address soft-reconfiguration inbound command, which enables you to make changes and not have to clear the TCP peer, resulting in no downtime.


Example 4 show tcp brief



R1#show tcp brief
 
TCB       Local Address           Foreign Address        (state)
812CC228  131.108.255.5.11040     131.108.255.6.179      ESTAB
812CF508  131.108.1.1.11039       131.108.1.2.179        ESTAB
812D0054  131.108.255.13.11041    131.108.255.14.179     ESTAB
 
Router R1, as displayed in Example 4, has three TCP sessions in an established state. The TCP port numbers are also listed. This command is useful because you need to be certain that TCP is active at Layer 4 of the OSI model when troubleshooting to discover why two BGP peers are not sending updates, for example. The foreign addresses list the TCP port as 179, and the local address is a number TCP generates. This tells you that R1 has three TCP sessions active, and you can expect BGP to send updates and keepalives across each TCP session.


Example 5. show tcp brief on R1



R1#sh tcp brief
 
TCB       Local Address           Foreign Address        (state)
812CF984  131.108.255.5.11042     131.108.255.6.179      ESTAB
812CCB20  131.108.1.1.11044       131.108.1.2.179        ESTAB
812CC6A4  131.108.255.13.11043    131.108.255.14.179     ESTAB

 


Example 6 debug ip bgp keepalives on R1



R1#debug ip bgp keepalives
 
BGP keepalives debugging is on
4d01h: BGP: 131.108.255.6 sending KEEPALIVE
4d01h: BGP: 131.108.255.6 KEEPALIVE rcvd
4d01h: BGP: 131.108.255.14 sending KEEPALIVE
4d01h: BGP: 131.108.255.14 KEEPALIVE rcvd
4d01h: BGP: 131.108.1.2 sending KEEPALIVE
4d01h: BGP: 131.108.1.2 KEEPALIVE rcvd
 
R1 is sending and receiving keepalives to the three remote peers to ensure that the remote routers are still active. Assume that R1 is reloaded.

BGP Attributes At a Glance