When trying to diagnose network problems one of the questions I always ask is “What is the status of the switch port that is connected to the module’s interface?” The typical answer is “I need to ask the network people”. Now using the magic of Simple Network Management Protocol (SNMP) and the attached command macro it may not be necessary to ask.

 

All manageable switches (and this includes routers that also act as switches) respond to SNMP requests. The SNMP protocol is protected by a password, known as the community string. There is typically a password for reading and another for writing. I can certainly understand your network people not wanting to give you the write community string but if you ask nicely they may give you the read string. After all, there should be no harm in reading and if you can do it yourself, you do not have to bother them. However, before asking you might want to try the standard default string which is “public”. If it works, no need to even ask.

 

SNMP data is organized by MIBs (Management Information Base). The “Interfaces” table is in the MIB-II MIB which is supported by, dare I say, all devices that support SNMP. The table includes error, byte and packet counters. It is not a lot of information but it is enough to determine the basic health of the connection between your module’s interface and the network. The interfaces MIB is documented in RFC-1213 “Management Information Base for Network Management of TCP/IP-based internets: MIB-II” You can find a copy at http://www.ietf.org/rfc/rfc1213.txt.

 

The macro gives you three ways to identify the individual port on the switch to query. The first is by interface name. Using the name it uses the netstat command to determine the MAC address of the interface and queries the switch’s forwarding table to determine which port is associated with the MAC address.

 

get_switch_interface_stats 172.16.1.222 #sdlmux.m16.11-3
The IP address that precedes the name is the IP address of the switch.
You can also provide the MAC address in the form XX-XX-XX-XX-XX-XX. This is one way to look at the standby adapter’s switch connection.

 

get_switch_interface_stats 172.16.1.222 00-00-a8-44-52-22 -type mac
Finally you can just provide the port index. Depending on the switch this may be the actual port number or something based on the port number. In the following example the port number is 48 but the index is 1048.

 

get_switch_interface_stats.cm 172.16.1.222 1048 -type port
In the following example output you will notice that the timestamps between the first and second passes are slightly more than 60 seconds apart. This is because the timestamp is printed before the queries are made and the queries take time.

 

Why are there two passes? The counters are not normally cleared so a single value doesn’t give you any idea of how fast the counters are changing. The macro displays both the first and second values and does the subtraction for you.

get_switch_interface_stats.cm 172.16.1.222 1048 -type port
172.16.1.222 1048 -type port public -sleep 60

Collecting results pass 1 10-01-20.11:32:21
sleeping for 60 seconds
Collecting results pass 2 10-01-20.11:33:23


Port Index :                1048
Port name :                 "X350-48t Port 48"
Speed :                     1000
Administrative Status :     up(1) up(1)
Operation Status :          up(1) up(1)
In Octets :                 205663121 - 205568072 = 95049
In Unicast Packets :        1336981429 - 1336981157 = 272
In non-Unicast Packets :    330419202 - 330418397 = 805
In Discards :               0 - 0 = 0
In Errors :                 0 - 0 = 0
In Unknown Protocols :      0 - 0 = 0
Out Octets :                290040991 - 290003159 = 37832
Out Unicast Packets :       1570859994 - 1570859741 = 253
Out non-Unicast Packets :   4386374 - 4386362 = 12
Out Discards :              0 - 0 = 0
Out Errors :                0 - 0 = 0
Out Q Length :              0 - 0 = 0

 

Under ideal circumstances there are never any errors or discards so if these counters are non-zero the cause should be investigated. The packet counts give you an idea of volume. If there is an unusually low volume of outgoing packets (packets going from the switch to the interface) you know there may be an issue with the switch’s connection to the rest of the network. An unusually high non-unicast outgoing packet count may indicate that some host on the network is flooding the network with broadcasts or multicasts. If the incoming packet count is low you have to ask why the application(s) on the module are not transmitting at the expected rate. The key here is knowing what is unusual and that requires creating a baseline.

 

The only unambiguous values are the administrative and operational status. If operational status shows down you know there is a problem with the connection between the switch and the interface. If administrative status shows down you know that someone disabled your network connection.

 

If the link is down how can you query the switch port to know that the link is down? You can query the switch using either the mac address or port index from another module. However, if the link has been down for some time it is possible that the MAC address has been purged from the switch’s forwarding table. If that is the case you will need to use the port index form of the command.

 

The SNMP commands are found in the >system>maint_library. They are a port of the NET-SNMP commands and documentation can be found at http://www.net-snmp.org/. Note that it is not necessary to be running the SNMP server (snmpd) on the module to run these commands against the switch.

 

& get_switch_interface_stats.cm begins here
&
& get_switch_interface_stats.cm
& version 1.0 10-01-20
& noah.davids@stratus.com
&
&
&begin_parameters
SWITCH switch:string,req
ID id:string,req
TYPE option(-type),name,allow(int,mac,port),=int
COMMUNITY community:string=public
SLEEP option(-sleep),number,=60
&end_parameters
&
&
&if (process_type) = 'batch' &then &do
set_ready -format off
&echo no_command_lines no_macro_lines no_input_lines
&end
&
&
& display input arguments
display_line &SWITCH& &ID& -type &TYPE& &COMMUNITY& -sleep &SLEEP&
&
&
&set_string INTERFACE_NAME ''
&set_string MAC_ADDR ''
&set_string IDX ''
&if &TYPE& = int &then &set_string INTERFACE_NAME &ID&
&if &TYPE& = mac &then &set_string MAC_ADDR &ID&
&if &TYPE& = port &then &set_string IDX &ID&
&
&
&set_string TEST (process_dir)>test
&set_string INTERFACE (process_dir)>interface
&set_string MAC (process_dir)>mac
&set_string INDEX (process_dir)>index
&set_string R1 (process_dir)>r1
&set_string R2 (process_dir)>r2
&
&
& make sure we can get a response from the switch
attach_default_output &TEST&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::sysName.0
detach_default_output
&if (file_info &TEST& blocks_used) = 0 &then &goto NOSWITCH
&
&if (length &IDX&) = 0 &then &do
&if (length &INTERFACE_NAME&) > 0 &then &do
attach_default_output &INTERFACE&
netstat -interface &INTERFACE_NAME&
detach_default_output
&if (file_info &INTERFACE& blocks_used) = 0 &then &goto NOINTERFACE
display &INTERFACE& -match 'MAC Address' -no_header -output_path &MAC&
&set_string MAC_ADDR (substr (contents &MAC& 1) 14)
&end
&if (length &MAC_ADDR&) < 17 &then &goto BADMAC
&set_string OCT1 (decimal (substr &MAC_ADDR& 1 2)x)
&set_string OCT2 (decimal (substr &MAC_ADDR& 4 2)x)
&set_string OCT3 (decimal (substr &MAC_ADDR& 7 2)x)
&set_string OCT4 (decimal (substr &MAC_ADDR& 10 2)x)
&set_string OCT5 (decimal (substr &MAC_ADDR& 13 2)x)
&set_string OCT6 (decimal (substr &MAC_ADDR& 16 2)x)
&
&
& query the forwarding table using the MAC adress to get the switch
& port number. The forwarding table is in the BRIDGE-MIB and is the
& dot1dTpFdbPort
attach_default_output &INDEX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& &+
.1.3.6.1.2.1.17.4.3.1.2.&OCT1&.&OCT2&.&OCT3&.&OCT4&.&OCT5&.&OCT6&
detach_default_output
&set_string TEMP (reverse (contents &INDEX& 1))
&set_string IDX (index (string &TEMP&) ' ')
&set_string IDX (reverse (substr (string &TEMP&) 1 (calc &IDX& - 1)))
& if after all that manipulation IDX is "OID" it means that the
& forwarding table did not contain the MAC address
&if &IDX& = OID &then &goto NOTFOUND
&
& dump the entire interface index table and use the port number from above
& to index into the table to get the port index
attach_default_output &INDEX&
snmpwalk -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifIndex
detach_default_output
&set_string IDX (substr (contents &INDEX& &IDX&) 22)
&set_string IDX (substr (string &IDX&) 1 &+
(calc (index (string &IDX&) ' ') - 1))
&end
&
&
display_line
display_line Collecting results pass 1 (date).(time)
&
&
attach_default_output &R1&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifDescr.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifSpeed.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifAdminStatus.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOperStatus.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifLastChange.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInOctets.&IDX&
<snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInUcastPkts.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInNUcastPkts.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInDiscards.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInErrors.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInUnknownProtos.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutOctets.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutUcastPkts.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutNUcastPkts.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutDiscards.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutErrors.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutQLen.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifSpecific.&IDX&
<detach_default_output
&
&
& if the last 3 characters of the first line are OID it means that the
& switch does not suppor the query - probably the index is index is wrong.
&if (substr (reverse (contents &R1& 1)) 1 3) = DIO &then &goto NOIDX
&
&
display_line sleeping for &SLEEP& seconds
sleep -seconds &SLEEP&
&
&
display_line Collecting results pass 2 (date).(time)
attach_default_output &R2&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifDescr.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifSpeed.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifAdminStatus.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOperStatus.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifLastChange.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInOctets.&IDX&
<snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInUcastPkts.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInNUcastPkts.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInDiscards.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInErrors.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifInUnknownProtos.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutOctets.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutUcastPkts.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutNUcastPkts.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutDiscards.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutErrors.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifOutQLen.&IDX&
snmpget -v 2c -c &COMMUNITY& &SWITCH& RFC1213-MIB::ifSpecific.&IDX&
detach_default_output
&
&
display_line
display_line
display_line Port Index : '              ' &IDX&

&set LIDX (length &IDX&)

&set START (calc 33 + &LIDX&)
display_line Port name : '               ' (substr (contents &R1& 1) &START&)

&set START (calc 34 + &LIDX&)
display_line Speed : '                   ' &+
(calc (substr (contents &R1& 2) &START&) / 1000000)

&set START (calc 40 + &LIDX&)
display_line Administrative Status : '   ' &+
(substr (contents &R2& 3) &START&) (substr (contents &R1& 3) &START&)

&set START (calc 39 + &LIDX&)
display_line Operation Status : '        ' &+
(substr (contents &R2& 4) &START&) (substr (contents &R1& 4) &START&)

&set START (calc 39 + &LIDX&)
&set V1 (substr (contents &R1& 6) &START&)
&set V2 (substr (contents &R2& 6) &START&)
display_line In Octets : '               ' &V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 42 + &LIDX&)
&set V1 (substr (contents &R1& 7) &START&)
&set V2 (substr (contents &R2& 7) &START&)
display_line In Unicast Packets : '       '&V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 43 + &LIDX&)
&set V1 (substr (contents &R1& 8) &START&)
&set V2 (substr (contents &R2& 8) &START&)
display_line In non-Unicast Packets : '  ' &V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 41 + &LIDX&)
&set V1 (substr (contents &R1& 9) &START&)
&set V2 (substr (contents &R2& 9) &START&)
display_line In Discards : '             ' &V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 39 + &LIDX&)
&set V1 (substr (contents &R1& 10) &START&)
&set V2 (substr (contents &R2& 10) &START&)
display_line In Errors : '               ' &V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 46 + &LIDX&)
< &set V1 (substr (contents &R1& 11) &START&)
&set V2 (substr (contents &R2& 11) &START&)
display_line In Unknown Protocols : '    ' &V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 40 + &LIDX&)
&set V1 (substr (contents &R1& 12) &START&)
&set V2 (substr (contents &R2& 12) &START&)
display_line Out Octets : '              ' &V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 43 + &LIDX&)
&set V1 (substr (contents &R1& 13) &START&)
&set V2 (substr (contents &R2& 13) &START&)
display_line Out Unicast Packets : '     ' &V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 44 + &LIDX&)
&set V1 (substr (contents &R1& 14) &START&)
&set V2 (substr (contents &R2& 14) &START&)
display_line Out non-Unicast Packets : ' ' &V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 42 + &LIDX&)
&set V1 (substr (contents &R1& 15) &START&)
&set V2 (substr (contents &R2& 15) &START&)
display_line Out Discards : '            ' &V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 40 + &LIDX&)
&set V1 (substr (contents &R1& 16) &START&)
&set V2 (substr (contents &R2& 16) &START&)
display_line Out Errors : '              ' &V2& - &V1& = (calc &V2& - &V1&)

&set START (calc 36 + &LIDX&)
&set V1 (substr (contents &R1& 17) &START&)
&set V2 (substr (contents &R2& 17) &START&)
display_line Out Q Length : '            ' &V2& - &V1& = (calc &V2& - &V1&)
&return
&
&label NOSWITCH
display_line
display_line
<span style="font-family: Courier New,monospace;"display_line &SWITCH& is not responding to SNMP queries using &+
the community string &COMMUNITY&
&return
&
&
&label NOINTERFACE
display_line
display_line
display_line &INTERFACE_NAME& is not an STCP interface on this module
&return
&
&
&label BADMAC
display_line
display_line
display_line MAC address &MAC_ADDR& is not in the format of XX:XX:XX:XX:XX:XX
display_line each octet must be 2 characters
&return
&
&
&label NOTFOUND
display_line
display_line
&if (length &INTERFACE_NAME&) > 0 &then &do
display_line MAC address &MAC_ADDR& for interface &INTERFACE_NAME&
display_line not found in switch &SWITCH&
&end
&else &do
display_line MAC address &MAC_ADDR& not found in switch &SWITCH&
&end
&return
&
&
&label NOIDX
display_line
display_line
display_line No data found for switch port &IDX& in switch &SWITCH&
&
&
& get_switch_interface_stats.cm ends here