Apr 12
14
Recently I dealt with a site that appeared to be having an issue with the link between their module and one of their Ethernet switches. As a test the system administrator, call him Fred, pinged the switch and since all pings got a reply Fred concluded the problem was elsewhere.
The problem with this conclusion is that the pings were not going over the module’s link to the switch.
Let me describe the networking configuration, I’ve changed the IP addresses to private networks but the relationship between interfaces and networks is similar.
The module has two interfaces, #enet10 and #enet192. The IP address of #enet10 is 10.1.1.1, this is the interface that is connected to the switch over the problematic link. The IP address of #enet192 is 192.168.1.1, and is connected to a different switch over a different link. The default gateway is 192.168.1.254 and no other routes are configured. The switch’s management IP address is 172.16.100.100.
To reach 172.16.100.100 the module must use the default gateway and to reach the default gateway it must send the packets out of the #enet192 interface. The switch sees the source of the ping packets as #enet192′s address, 192.168.1.1, and so replies to that address. At no time do any of the packets leave from or arrive at #enet10. The fact that #enet10 is directly connected to the switch doesn’t matter; the only thing that matters is the packet’s destination address and the module’s routing table.
So is there any way for Fred to test the link between #enet10 and the switch? No, the best that Fred can do is to ping some other host on the 10 subnet. That would cause the packets to be transmitted from #enet10 and go over the link to the target host and for replies to come back the same way but it would be testing both the link between #enet10 and the switch and any other links between the switch and the target host.
Fred could use the “netstat -interface #enet10″ command to see if the hardware or the driver are reporting any low level errors on the link. The switch also has commands that will report error counts on the port connected to the link connected to #enet10. But there is no way to test the link between #enet10 and the switch.
The performance of applications that use cross module or cross system communication is dependent on having the correct number of OSL server processes running. Too few processes can dramatically slow down your applications. This Webinar will explain how the default number of OSL server processes is determined, how to tell if that is enough, and how to start more. In addition it will cover how to select an appropriate value for the max_open_servers parameter and its relationship to number of OSL server processes.
View the slidedeck on VOS Server Processes here!
The performance of applications that use cross module or cross system communication is dependent on having the correct number of OSL server processes running. Too few processes can dramatically slow down your applications. This Webinar will explain how the default number of OSL server processes is determined, how to tell if that is enough, and how to start more. In addition it will cover how to select an appropriate value for the max_open_servers parameter and its relationship to number of OSL server processes.
The “netstat –interval X” command in releases prior to 17.0 will display a list of network connections every X seconds but you have no idea which if any of those connections are active or how active. You might get a clue if the Recv-Q or Send-Q values change but only if you are lucky. Starting in OpenVOS 17.1 there is a new improved netstat that you can access by adding the “-64” argument. This gives you access to a set of 64 bit counters but it also changes the behavior of some of the displays. What I want to highlight here is the list of connections displayed by “netstat -64 –interval X”
First lets review the old netstat display, which is still the default in 17.1. Note the “Send-Q” value in the second to last connection. Since it is changing you know that the connection is active but the state of the other connections is unknown.
Adding the “-64” argument to the command line (netstat –numeric –protocol tcp –interval 15 -64) gives you an interactive display that highlights changes so when the Recvd or Sent byte counts or the “Recv Q” or “Send Q” values change they are displayed in “bold”.
By default the byte counts are displayed to 3 significant digits with a “K”, “M”. or “G” suffix; so you know that bold counters have changed but have no idea of the magnitude of the change. Pressing the “9” key toggles the display to the actual number.
When the Recvd and/or Sent columns have large numbers they can be difficult to read, pressing the “c” key toggles between the actual number and a delta value.
Note that while the old netstat just wrote to the screen a line at a time this new -64 netstat is dynamically updating a screen image, like list_users –interval. If there are more lines than will fit on the screen you can use the “p”, “b”, “n”, “f”, space or any of the standard cursor control keys to move the display up or down.
Noah Davids, Support Engineer at Stratus discusses how application performance is affected by the network, the TCP stack and the application’s design. Covering common issues, how to test for them before the application is placed into production, how to diagnose them when they occur, and how to tune the system to reduce their impact.
The slides can be found on Noah David’s website, here. Please feel free to ask questions in the comments below!