Layer by Layer Troubleshooting with a Cisco Router

OSI Model - Bottom Up Troubleshooting

If you will recall, the OSI model starts with the physical layer (layer 1) and goes up to layer 7 (application). When troubleshooting with a Cisco router, much of your time will be spent working in layers 1-3. They are:

• Layer 3 - Network
• Layer 2 - Data Link
• Layer 1 - Physical

Because these layers build on each other, Layer 1 is most critical, without layer 1, layer 2 will not function. Without layer 1 & 2, layer 3 will not function, and so on. For this reason, I start troubleshooting at layer 1, physical, and move on up from there.

Here is what you look for:

• Is the interface UP?
• Is the line protocol UP?
• If both the interface and line protocol are NOT up, your connection is never going to work.
• To resolve a line down, I look at the cable or the keepalives
• To resolve a line protocol down, check to make sure that the protocols match on each side of the connection(notice the "line protocol" on each of the interfaces above).
• Are you taking input, CRC, framing, or other errors on the line (notice how the serial interface above does show errors)? If so, check your cable or contact your provider.

In general, verify that you have a good cable on each side, verify that line protocols match, and that clocking settings are correct.

If this is an Ethernet connection, is there a link light on the switch?

If this is a serial connection, do you have an external CSU/DSU? If it is an external CSU, check that the Carrier Detect (CD) light & data terminal ready (DTR) lights are on. If not, contact your provider. This also applies if you have an internal Cisco WIC CSU card. If that is the case, take a look at this Cisco link on understanding the lights on that card.

You can, of course, use the Cisco IOS test commands to test your network interfaces with internal staff and with your telecommunications providers.

Do not proceed to upper level layers until your Physical interface on the router shows as being UP and your line protocol is UP. Until then, don't worry about IP addressing, pinging, access-lists or anything like that.

I would recommend taking this interface configuration and comparing it, side by side, with the remote WAN connection to ensure they are the same. Ask yourself questions like:

• Are these interfaces on the same IP network?
• Do these interfaces have the same subnet mask?
• Are there any access-lists (ACL) that are blocking your traffic?
• Can you remove all optional IP features to make sure that the basic configuration works before adding additional features that could be causing trouble?

Router Troubleshooting at OSI Layer 3 - Network

Once you have Layers 1 & 2 working (your show interface command shows the line is "UP & UP", it is time to move on to layer 3 - the OSI Network layer. The easiest thing to do here to see if layer 3 is working is to ping the remote side of the LAN or WAN link from this router. Make sure you ping as close as possible to the router you are trying to communication with - from one side across to the other side.

Router Troubleshooting at OSI Layer 1 & 2 - Physical & Data link

Remember, if Layer 1 isn't up, nothing else will work so make sure you start here. Examples of layer 1 are your T1 circuit or your Ethernet cable - physical connectivity. I usually troubleshoot layer 1 and layer 2 in union because they are so closely paired. Examples of layer 2 - data link - are your line protocol (such as Ethernet, ATM, 802.11, PPP, frame-relay, HDLC, or PPP).

To troubleshoot at these layers, the first thing I would do on your router is a show interface. 

Router Troubleshooting at OSI Layers 4 - 7

Now, let's say that you have made it to the point where you can ping from LAN to LAN, through your WAN. Congratulations - that is a very good sign. If you are still having trouble, it must be in OSI Layers4-7. Here are those layers listed out and possible issues you might experience in each layer:

• Layer 4 - Transport - in the transport layer are TCP and UDP - you could be have an ACL or QoS feature blocking or slowing this traffic. Your TCP traffic could also be fragmented to the point that it could not be reassembled. Another option is that you may not be receiving an ACK back from your traffic that was successfully sent.

• Layer 5 - Session - in the session layer are protocols like SQL, NFS, SMB, or RPC - you could be taking errors on any one of these session protocols. I would recommend using a protocol analyzer like Wireshark to analyze your session data.

• Layer 6 - Presentation - in the Presentation layer are data encryption, compression, and formatting - your VPN tunnel could be failing or perhaps you are sending one type of data (like a MPEG) and the receiver is trying to view it as a WMV file.

• Layer 7 - Application - in the Application layer are, of course, your applications like FTP, HTTP, SCP, TFTP, TELNET, SSH, and more - you could be trying to connect to a telnet server with the SSH protocol, for example.

• Layer 8 - End User - the standing joke is that "Layer 8" is the user - the user could be just mistyping their username or password or you, the network admin, could have been troubleshooting the wrong IP address all along.

Summary

In summary, using the OSI model to troubleshoot connectivity issues is the fastest and most efficient way to troubleshoot any network issue. Even if someone calls you to work on a Windows share problem, all of the same principles in this article apply to that troublesooting process. So remember, the next time you work on a network issue - remember the OSI model and how to use the bottom-up approach to troubleshooting! It could same you a while lot of time!

What is Spanning Tree? What is PortFast?

Question
What is Spanning Tree and Portfast ?

Answer

The Spanning-Tree Protocol (STP) was created to overcome the problems of transparent bridging in redundant networks. The purpose of STP is to avoid and eliminate loops in the network by negotiating a loop-free path through a root bridge. This is done by determining where there are loops in the network and blocking links that are redundant.

Spanning-Tree Protocol executes an algorithm called the Spanning-Tree Algorithm (STA). In order to find redundant links, STA will choose a reference point called a Root Bridge, and then determines all the available paths to that reference point. If it finds a redundant path, it chooses for the best path to forward and for all other redundant paths to block. This effectively severs the redundant links within the network.

All switches participating in STP gather information on other switches in the network through an exchange of data messages. These messages are referred to as Bridge Protocol Data Units (BPDUs). The exchange of BPDUs in a switched environment will result in the election of a root switch for the stable spanning-tree network topology, election of designated switch for every switched segment, and the removal of loops in the switched network by placing redundant switch ports in a backup state.

During the execution of the Spanning-Tree Algorithm, Spanning-Tree will force the ports to go into five different states:

• Blocked
• Listen
• Learn
• Forward
• Disabled

A Description of each state follows:

• Blocked—All ports start in the blocked mode in order to prevent the switch from creating a loop.
• Listen—The port transitions from the blocked state to the listen state. It uses this time to attempt to learn whether there are any other paths to the root bridge. This state is really used to indicate that the port is getting ready to transmit but it would like to listen for a little longer to make sure it does not create a loop.
• Learn—When in this state, the switch will add information it has learned through the listening process to its address table. It is still not allowed to send data.
• Forward—This state means the port can send and receive data.
• Disabled—The switch can disable a port for a variety of reasons including: hardware failure, deletion of the ports native VLAN, and being administratively disabled.

The transitioning period from state to state takes the following times by default:

• From blocking to listening: 20 seconds
• From listening to learning: 15 seconds
• From learning to forwarding: 15 seconds

Spanning-Tree protocol is running by default on all ports of the switch. The spanning-tree protocol makes each port wait up to 50 seconds before data is allowed to be sent on the port. This Delay in turn can cause problems with some applications/protocols (PXE, Bootworks, etc.). To alleviate the problem, Porfast was implemented on Cisco devices, the terminology might differ between different vendor devices.

PortFast causes a port to enter the forwarding state almost immediately by dramatically decreasing the time of the listening and learning states. Portfast minimizes the time it takes for the server or workstation to come online, thus preventing problems with applications such as DHCP, DNS, Novell IPX, PXE, BootWorks, etc.

The spanning-tree protocol is always running even when the port is in forwarding state so that it can still detect loops, however the use of Portfast should only be implemented when the port on the switch is directly connected to a server/workstation and never to another hub/switch.

To enable portfast on Cisco switches you must enter the following command:

On set based switches: at the Switch>(enable)" prompt: 

Set Spantree portfast <mod_number/port_number>

on IOS based switches: at the "switch(config-if)#" prompt: 

spantree start-fowarding