I’m pleased to announce that OpenVOS Release 17.2.0, GNU Tools Release 4.0.0 and Kona for OpenVOS Release 1.1.0 are now generally available. All VOS V Series customers with a maintenance contract are eligible for these releases. Plus, all of them are available on the www.openvos.com web site, so you no longer need to order release tapes.
StrataDoc (stratadoc.stratus.com) has been updated with the manuals for these products as well.
What’s in OpenVOS 17.2, you ask? Here is a brief summary:
- Support for a new LTO-5 tape drive (up to 1.5TB capacity)
- Support for Solid State Disk Drives in ftScalable Gen2
- Support for 64-bit Stream Files
- Support for Extended Names Version 2
- Phase 1 of Improved Password Encryption
- Increased Software Limits
- Larger Syserr and Accounting Logs
- Sort Command Performance Boost (sorts more records in memory)
Here is what is new in GNU Tools 4.0:
- Support for 64-bit Stream Files and Extended Names V2
- New Porting Bases for Many Open-Source Packages
- Man Pages for Most Packages
- The wget command
- The xz command
- GDB can follow forks
Here is what is new in Kona 1.1:
- Support for 64-bit Stream Files and Extended Names V2
- All Relevant OpenJDK Tests Pass
OpenVOS Release 17.2.0 supports all current V Series hardware platforms (V100 to V6512). It will be the final release to support the V100, V150, V200, V250, V300, V400, and V500 platforms.
See the Software Release Bulletins for these products, available on StrataDOC, for more details.
Apr 13
24
Remote Command Execution
Last week (well it was last week when I started writing this) we had an issue come into the CAC where someone wanted to write a command macro that ran telnet to log into a system and execute some commands, for example the command macro test.cm logins into a remote host and executes the foo.cm command macro.
&attach_input
telnet 192.168.77.128
password
foo.cm
|
|
Figure 1 – telnet command macro test.cm |
display_line display_system_usage
display_system_usage
display_line netstat -statistics -protocol tcp
netstat -statistics
display_line analyze_system -request_line 'stcp_meters -all -long' -quit
analyze_system -request_line 'stcp_meters' -quit
|
|
Figure 2 – foo.cm |
Unfortunately telnet is not designed to execute in a command macro and it fails.
test
telnet: fatal error - tcgetattr failed
Error on line 3 of test.
command_processor: Object not found. password. In command macro
%azvos#m17_mas>SysAdmin>Noah_Davids>test.cm
|
|
Figure 3 – execution of test.cm fails |
There are however several ways that automagic login into a remote host followed by command execution can be done.
send_cmds.c:
I wrote a program a while back that makes a connection to a telnet server, sends strings, and parses the output. The scripting language is simple but for simple things it works fine. The source code with more extensive examples can be found here
The script is basically a send string followed by a wait string. The script advances to the next line when the wait string is seen, as I said very simple. The script does have arguments, in figure 4 I use arg0 for the password so my password is not stored in the script. Execution of the script follows in figure 5. I truncated the output of each command in the interest of saving space. You will notice that the non-control characters from the character positioning strings are displayed until I set the terminal type to ascii.
//login
/login nd/Password?
/arg0/ready
/set_terminal_parameters -terminal_type ascii -pause_lines 0/ready
/foo.cm/ready
/quit/quit
|
|
Figure 4 – send_cmds script to login and execute the foo command macro |
send_cmds -IPAddress 192.168.77.128 -InputFilePath test.send_cmds -no_Verbose -a
+rg0 password
PHOENIX CAC VOS TEST SYSTEM (%phx_vos#m16)
*** To boot on a different VOS Release see ***
*** >Overseer>COMMON>cfg>reconfig_instructions ***
OpenVOS Release 17.1.0ax, Module %phx_vos#m16
Please login 08:15:01
login nd
Password?
Noah_Davids.CAC logged in on %phx_vos#m16 at 13-04-21 08:15:01 mst.
[f[J[?7l[20l=[1m[1;24r[f[J[J[24;80f
[0;1mready 08:15:01
[0mset_terminal_parameters -terminal_type ascii -pause_lines 0
[1mset_terminal_parameters -terminal_type ascii -pause_lines 0
ready 08:15:01
foo.cm
display_system_usage
Usage statistics for module %phx_vos#m16, G94330, OpenVOS Release 17.1.0ax
All 762.6 hours (31.7 days) up.
----CPU----- Last Min Last 5 Min Last Hour All Up Time
. . . . .
Int. Time 0.00 0.1% 0.02 0.1% 0.23 0.1% 191.65 0.1%
netstat -statistics
tcp:
. . . .
201766 tcpOutRsts
analyze_system -request_line stcp_meters -all -long -quit
OpenVOS Release 17.1.0ax, analyze_system Release 17.1.0ax
Current process is 884, ptep 8D3E6000, Noah_Davids.CAC
stcp_meters %phx_vos#m16 762:36:19 13-04-21 08:15:02
. . . .
bufdat executed 678 (avg 0.0002/sec)
ready 08:15:02
ready 08:15:03
|
|
Figure 5 – send_cmds execution |
expect:
There is a program called expect in the gnu_library. The syntax for an expect script can be very complex, it allows for branching based on different return strings so you can intelligently handle errors instead of just timing out. You can check out the web for examples, just search for "expect script examples". I will present a simple script that just logs in and executes the foo.cm macro. Note that there were some bugs in early versions of gnu_tools (which is why I wrote send_cmds) but as of release 3.4.0a you should be OK. While I do not show it, expect can also use arguments so the password does not have to be part of the script or the script can prompt you for a password. Again I truncated the command output from the script and you can see that the non-control characters from the character positioning strings are displayed until I set the terminal type to ascii. Expect however filters the string differently so not all the characters are shown.
spawn "telnet"
expect "telnet>"
send "open 192.168.77.128\r"
expect "login"
send "login nd\r"
expect "Password"
send "password\r"
expect "ready"
send "set_terminal_parameters -terminal_type ascii -pause_lines 0\r"
expect "ready"
send "foo.cm\r"
expect "ready"
|
|
Figure 6 – expect script to login and execute the foo command macro |
>bin>expect test.expect
spawn telnet
telnet> open 192.168.77.128
Trying...
Connected to 192.168.77.128.
Escape character is '^]'.
PHOENIX CAC VOS TEST SYSTEM (%phx_vos#m16)
*** To boot on a different VOS Release see ***
*** >Overseer>COMMON>cfg>reconfig_instructions ***
OpenVOS Release 17.1.0ax, Module %phx_vos#m16
Please login 08:51:28
login nd
Password?
Noah_Davids.CAC logged in on %phx_vos#m16 at 13-04-21 08:51:28 mst.
fJ?7l20l
0;1mready 08:51:28
1mset_terminal_parameters -terminal_type ascii -pause_lines 0
ready 08:51:28
foo.cm
display_system_usage
Usage statistics for module %phx_vos#m16, G94330, OpenVOS Release 17.1.0ax
All 763.2 hours (31.8 days) up.
----CPU----- Last Min Last 5 Min Last Hour All Up Time
. . . .
Int. Time 0.00 0.1% 0.02 0.1% 0.24 0.1% 191.80 0.1%
netstat -statistics
tcp:
. . . .
201926 tcpOutRsts
analyze_system -request_line stcp_meters -all -long -quit
OpenVOS Release 17.1.0ax, analyze_system Release 17.1.0ax
Current process is 895, ptep 8D2E1200, Noah_Davids.CAC
stcp_meters %phx_vos#m16 763:12:46 13-04-21 08:51:29
. . . .
bufdat executed 678 (avg 0.0002/sec)
ready 08:51:29
ready 08:51:29
|
|
Figure 7 – expect script execution |
SSH:
Finally you can use SSH to execute a command on a remote system. If you have also set up public key authentication you are not prompted for a password at the time of execution.
There are a number of interesting wrinkles when using SSH. First abbreviations do not work, don’t try to execute dps, use display_print_status.
ssh nd@192.168.77.128 dbs
sh: dbs: command not found
ready 12:26:25
|
|
Figure 8 – Using SSH to remotely execute a command abbreviation doesn’t work |
ssh nd@192.168.77.128 display_batch_status
Batch queues for %phx_vos#m16
QUEUE STATE MAX RUNNING JOBS
normal run 5
ready 12:26:43
|
|
Figure 9 – Using SSH to remotely execute a command does work |
Second, executing a command macro directly doesn’t work; note there is no output and it appears that only the first command in the macro was even processed.
ssh nd@192.168.77.128 'foo.cm'
display_system_usage
ready 12:08:40
|
|
Figure 10 – Using SSH to remotely execute a command macro doesn’t always work |
Instead you have to execute the shell command and provide the command macro as its argument.
ssh nd@192.168.77.128 '/bin/sh foo.cm'
display_system_usage
Usage statistics for module %phx_vos#m16, G94330, OpenVOS Release 17.1.0ax
All 766.5 hours (31.9 days) up.
----CPU----- Last Min Last 5 Min Last Hour All Up Time
CPU minutes 0.02 0.5% 0.09 0.4% 0.91 0.4% 724.76 0.4%
. . . .
Int. Time 0.00 0.1% 0.02 0.1% 0.23 0.1% 192.57 0.1%
netstat -statistics
tcp:
. . . .
202796 tcpOutRsts
analyze_system -request_line stcp_meters -all -long -quit
OpenVOS Release 17.1.0ax, analyze_system Release 17.1.0ax
Current process is 922, ptep 8D3C2780, Noah_Davids.CAC
stcp_meters %phx_vos#m16 766:30:14 13-04-21 12:08:57
. . . .
bufdat executed 678 (avg 0.0002/sec)
ready 12:08:59
|
|
Figure 11 – Using SSH to remotely execute a command macro by executing it as a shell script |
Complex macros will not work. For example each line of the command macro is forked as a separate process so macros that use the process_dir to hold intermediate results will not work. Note that each invocation of analyze_system is reporting a different process number.
ssh nd@192.168.77.128 '/bin/sh foo3.cm'
%phx_vos#m16_mas>SysAdmin>Noah_Davids>foo3.cm 13-04-21 12:52:20 mst
display foo3.cm
display_line
analyze_system -request_line '' -quit
analyze_system -request_line '' -quit
analyze_system -request_line '' -quit
OpenVOS Release 17.1.0ax, analyze_system Release 17.1.0ax
Current process is 1028, ptep 8D3C3000, Noah_Davids.CAC
OpenVOS Release 17.1.0ax, analyze_system Release 17.1.0ax
Current process is 1029, ptep 8D3C3000, Noah_Davids.CAC
OpenVOS Release 17.1.0ax, analyze_system Release 17.1.0ax
Current process is 1030, ptep 8D3C3000, Noah_Davids.CAC
ready 12:52:21
|
|
Figure 12 – Each command in the macro is forked as a separate process |
Using command macros with VOS style variables will generate syntax errors since the shell doesn’t know what they are. The foo3.cm command macro displays itself and then counts to 9. The display command is executed and the “&set” generates a syntax error.
ssh nd@192.168.77.128 '/bin/sh foo3.cm'
%phx_vos#m16_mas>SysAdmin>Noah_Davids>foo3.cm 13-04-22 07:33:18 mst
display foo3.cm
&set COUNT 1
&label again
display_line &COUNT&
&set COUNT (calc &COUNT& + 1)
&if &COUNT& < 10 &then &goto again
foo3.cm: line 2: syntax error near unexpected token `&s'
foo3.cm: line 2: `&set COUNT 1'
ready 07:33:18
|
|
Figure 13 – Command macros with VOS style variables will not work |
But using shell variables will work. That of course may require extensive rewrites of existing macros. Here is an example of the same macro as a shell script.
ssh nd@192.168.77.128 '/bin/sh foo3u'
cat foo3u
count=1
while [[ $count -le 9 ]]
do
echo "$count"
(( count++ ))
done
1
2
3
4
5
6
7
8
9
ready 07:44:41
|
|
Figure 14 – Shell script with shell stype variables does work |
However, you can run your VOS style command macro as a started process and then display the outfile that is produced. The macro start_foo3.cm runs start_process, waits for the process to terminate and then displays the output file without a header. I have modified the foo3.cm to turn off the command and macro lines and also the ready prompt to more fully mimic the output from a command macro run interactively.
start_process foo3.cm -wait
display foo3.out -no_header
|
|
Figure 15 – Command macro to run foo3.cm as a started process |
With the exception of the login header and the command macro name on the first 2 lines and the “Process finished” line at the end the results are the same as if you ran the macro from the VOS command line.
ssh nd@192.168.77.128 '/bin/sh start_foo3.cm'
Noah_Davids.CAC logged in on %phx_vos#m16 at 13-04-22 10:45:23 mst.
foo3.cm
%phx_vos#m16_mas>SysAdmin>Noah_Davids>foo3.cm 13-04-22 10:45:23 mst
&echo no_input_lines no_command_lines no_macro_lines
set_ready -format off
display foo3.cm
&set COUNT 1
&label again
display_line &COUNT&
&set COUNT (calc &COUNT& + 1)
&if &COUNT& < 10 &then &goto again
1
2
3
4
5
6
7
8
9
Process finished.
ready 10:45:25
|
|
Figure 16 – Results of running foo.cm in a started process |
So, as you can see while you cannot use telnet that are other ways to automagically login and execute commands.
About 18 months ago I wrote a blog talking about changes in how accept works in OpenVOS release 17.1. What I neglected to indicate was that these changes only take effect if the application code is POSIX based. To recap, prior to 17.1 STCP would only reply to a client connection request if the server application code had called the accept routine. If the code had not called accept the client’s connection request would go into a queue but would not be answered. Once the application called accept, STCP would send a connection reply. If the client is still waiting, the connection completes at that time. If the client had already given up, it would respond to the connection reply with a reset, STCP would throw the connection away and the accept would hang (assuming blocking mode) and wait for another connection.
Figure 1 shows this scenario; the left justified text lines are status messages from the server application, the indented packet_monitor lines (slightly modified) are the result of client activity. The application (acceptTestnoPosix) is started and listens on port 12345, and once listening goes to sleep for 300 seconds. The client then makes a connection to port 12345 by sending a TCP segment to port 12345 with the Syn (S) flag set. It tries three times before giving up. After 300 seconds the application wakes up and calls accept. The STCP stack replies to the client with a TCP segment with the Syn and Ack (SA) flags set. Since the client no longer has a record of the connection if sends a Reset (R), STCP blocks at this point. If the socket were in non-blocking mode accept would have returned with an EAGAIN error.
acceptTestnoPosix 12345 Executing acceptTestnoPosix 12345 Sleeping for 300 seconds 11:04:45.923 R TCP 164.152.77.50 164.152.77.217 11071 12345 S 11:04:48.925 R TCP 164.152.77.50 164.152.77.217 11071 12345 S 11:04:54.937 R TCP 164.152.77.50 164.152.77.217 11071 12345 S Ready to accept a connection on port number 12345 11:09:41.852 T TCP 164.152.77.217 164.152.77.50 12345 11071 SA 11:09:41.854 R TCP 164.152.77.50 164.152.77.217 11071 12345 R |
Figure 1 – accept pre 17.1 or non-POSIX post 17.1
If the application is POSIX based, things are different (Figure 2). STCP replies to the connection request (S) immediately (SA). Then after 300 seconds the application wakes up, calls accept and STCP returns the accepted socket.
acceptTestwithPosix 12345 Executing acceptTestnoPosix 12345 Sleeping for 300 seconds 10:58:43.962 R TCP 164.152.77.50 164.152.77.217 11024 12345 S 10:58:43.964 T TCP 164.152.77.217 164.152.77.50 12345 11024 SA 10:58:43.964 R TCP 164.152.77.50 164.152.77.217 11024 12345 A Ready to accept a connection on port number 12345 Connection accepted |
Figure 2 – accept POSIX based post 17.1
The behavior difference can be critical. If the client is expecting a response from the server application after making a connection it may time out and close the connection before the application actually calls accept.
How do you build an application with POSIX? First the line
#define _POSIX_C_SOURCE 200112L |
should be the first line of the application source. Second the library path
(master_disk)>system>posix_object_library |
needs to be in the object library paths after the STCP libraries and before the C library.
How can you tell if this has been done? Well you can look at the object libraries that the application was bound with using the display_program_module -object_dirs command
display_program_module acceptTestwithPosix -object_dirs
%azvos#m17_mas>SysAdmin>Noah_Davids>temp>acceptTestwithPosix.pm
Object Directory Map:
11 search directories
0 non-search directories
11 directories in all
DTC Directory Path
1 %azvos#m17_mas>SysAdmin>Noah_Davids>temp
2 %azvos#m17_mas>system>stcp>object_library
3 %azvos#m17_mas>system>stcp>object_library>socket
4 %azvos#m17_mas>system>stcp>object_library>net
5 %azvos#m17_mas>system>stcp>object_library>common
6 %azvos#m17_mas>system>posix_object_library
7 %azvos#m17_mas>system>c_object_library
8 %azvos#m17_mas>system>object_library
9 %azvos#m17_mas>opt>apache>lib
10 %azvos#m17_mas>opt>openssl>lib
11 %azvos#m17_mas>opt>mysql>lib>mysql
|
But this doesn’t tell you if the #define is part of the source code.
If you can run the code you can look at the socket flags, a POSIX based program will have the SF_POSIX socket flag set. For example, I have three programs acceptTestnoPosix, acceptTestwithPosix and acceptTestwithPosixPathOnly. This last one had the >system>posix_object_library directory in the library paths but did not include the #define in the source code. Note that this is considered an invalid combination. Only the application that was bound with the POSIX library and had the POSIX_SOURCE #define statement in the code created a socket with the SF_POSIX flag.
acceptTestnoPosix.pm 12345
Executing acceptTestnoPosix 12345
Sleeping for 300 seconds
as: match posix; dump_stcbq -full -lport 12345
as:
|
acceptTestwithPosix.pm 12345
Executing acceptTestnoPosix 12345
Sleeping for 300 seconds
as: match posix; dump_stcbq -full -lport 12345
SF_POSIX
as:
|
acceptTestwithPosixPathOnly.pm 12345
Executing acceptTestnoPosix 12345
Sleeping for 300 seconds
as: match posix; dump_stcbq -full -lport 12345
as:
|
There is a fourth possibility, the #define was included in the source but the posix_object_library was not used. In this case the socket routine will return an error.
acceptTestwithPosixDefineOnly 12345 Executing acceptTestnoPosix 12345 acceptTestnoPosix: can't create listening socket: Application built + incorrectly: POSIX runtime must be searched before C runtime. |
All of the STCP commands and utilities are built with POSIX and I strongly recommend that any applications you build also use POSIX.
The maintenance network is used by OpenVOS for monitoring several devices including the RAID and SAN disk arrays. This network also allows you access for making configuration changes, debugging problems and general monitoring. However, when accessing these devices from the OpenVOS module you can only use the command line interface (CLI); the friendlier, graphical user interface (GUI), used via a web browser is not available.
The lack of a GUI interface has led some of you to connect the maintenance network to another externally available network. This requires that all the devices on the network and the Stratus interface be renumbered. It also means that the devices on the maintenance network will be accessible to anyone with access to the network and the monitoring functions done over the network will be subject to the problems of a much larger network environment. This is something that Stratus strongly urges you not to do.
This post will discuss three ways of allowing you to use the web browser GUI interface to manage these devices while still keeping the maintenance network restricted to just the Stratus module’s environment. The first assumes that you want to gain access from a browser located on the same subnet as one of the Stratus module’s IP interfaces. The next two approaches use SSH tunneling.
