Add/Update
(OTADD)
Otadd is the executable called from the HP
OpenView OverTime menu when an “Add/Update” is requested. It is passed the list
of currently selected objects from the OpenView map/s. If no objects are
selected, the menu item will not be available. Each object is then SNMP queried
for additional information not available within the HP OpenView database. Otadd
firstly resolves the SNMP SysName and selects an IP address from each device to
use to communicate with that device. OverTime determines which IP address
OpenView will use to communicate with the device and uses this address. The HP
OpenView snmpCol.conf file is then searched to see what, if any, collections
may already be in place for this device. Two classes of collection are
considered. I/O collections involve ifInOctets, ifOutOctets, ifInErrors,
ifOutErrors, CiscoAvgBusy5 and sysuptime are in the I/O class. All other
collections are considered as special user defined collections. If no
collections exist then otadd will setup collections based purely on I/O. If a
user defined collection is setup and no I/O collection is active, then only the
user-defined collection will be setup within OverTime. If both I/O and user
defined collections are setup, then both will be setup within OverTime. OverTime
will work with any numeric collection including EXPRESSION collections.
For I/O collections (from routers and switches)
1 instance = 1 interface's InOctets, OutOctets,
InErrors and OutErrors.
1 instance = Cisco CPU Average Busy 5
For all other collections 1 instance = 1 SNMP
instance.
For I/O collections, the device is queried for
all available interfaces that meet certain criteria. I.E. the interface must be
administratively up, not be any of: a ds1, a loopback, a Null, a Dialer, a CPU
and have a reasonable interface speed >0 and <400Mb/sec. The acceptable
interfaces are then queried for their speed, their names setup in the device,
their maximum bandwidth and any associated IP address all of which then
automatically appear in the OverTime master configuration file for this device
and will ultimately appear on the web page and its graphs.
If the master configuration file for the device
already existed and the legends, y axis label or graph title were changed for
user-defined collections, then these changes are preserved by otadd.
Otadd writes an snmpCollect merge file that is
then merged with the existing collections so that HP OpenView will perform the
collections.
Otadd also writes the master configuration file
in OverTime/cfgs. This file’s name is the device’s SNMP sysName (or its IP
address if no name is available either of which will be referred to as sysName
within this document) with an extension of “.mst”. (See above for the
definition of this file).
Otadd then updates overtime.csv (see later) and
the master HTML file, index.html.
Otadd creates the directories for the RRD
databases to be stored in OverTime/data/ under the devices sysName directory.
Otadd creates the directories for the graphics
to be held in the web server’s area OverTime/ under the devices sysName
directory.
Otadd sets up the RRDs based on what it finds
and other user setting within overtime.cfg.
User-defined SNMP collections cause otadd to
create RRDs based on the collection name. These databases are also configured
based on the collection frequency setup when the RRD is initially created. IT
IS NOT CHANGED while the RRD database exists. The RRDs are given names that
commence with sysName and then followed by the SNMP interface number for I/O
collections. For user-defined collections, the RRD is called sysName and then
the MIB name (as found in snmpCol.conf) followed by the instance number
discovered by “snmpwalking” the device for this OID.
Otadd checks to see if the collections being
requested will alter the work being performed by snmpCollect and only if it
considers there is a change will it request snmpCollect to re-initialise. When
snmpCollect re-intialises it takes a little time for the collections to
re-commence.
External/Non-SNMP Collections
OverTime can be used to record and display
collections gathered by your own application or script and not just by
OpenView's snmpCollect daemon. These types of collections are termed
"external or non-SNMP collections" in OverTime. You use non-SNMP
collections to monitor from your server any kind of statistical value. Maybe
you want to monitor your web server’s response time or the number of cars going
past your window. If your server can count them, then OverTime can help you
record them. This is what a graph of web server response times looks like:
Setting Up a new TCP port Collection.
An example of this type of collection will help
to make the process clear. Let's assume that you would like to monitor the
response times for loading the main web page from two web sites (www.myco.com
and www.myotherco.com) and you want to see the results on your OverTime page
under the label ‘Web_Responses”.
You can use this time2open.pl script as
a starting point.
1) Copy the check_port.pl script and give it a name like http_time.pl on
your OverTime server.
2) Edit the new script and change the settings at the top. The settings
that you need to modify are the TCP port number, the string to send when the
port opens and the string that you expect to be returned if the test is
successful. Leaving the ‘testing’ value set to 1 will help you test the
process. For our example, we need to make these changes:
$port=80; # 80 for http
- User chooses what they want!
$send = "GET index.html\n"; #
What do you want sent
$expect="\<\/html\>";
#
What you expect to be returned if it worked.
$testing=1; #
1 if we're testing, 0 if we're really collecting
$threshold = 0.5 #
A threshold of 0.5 seconds to respond
$runcmd = "echo Threshold breached."; #
Command to run when threshold is breached
$script_name = "http_time"; #
The name of this script
3) Test your new script with a command like ‘http_time.pl Web-Responses
www.myco.com www.myotherco.com”
4) When you have this working correctly, remember to edit the script and
set $testing=0
5) To add your new collection to OverTime you run the otadd command:
“otadd --external G
/var/opt/OverTime/cfgs/http_time.pl Web_Responses www.myco.com
www.myotherco.com”.
This instructs OverTime to create the appropriate configuration files and also
update the PingTime-xxx command file.
The “--external” tells otadd to create an external/non-SNMP collection configuration.
The “G” indicates if the numbers collected are a gauge. Alternatively, it could
have been a "C" for counter. Counters are for values that
continuously increase like the count of packets through an interface. Gauge is
for values that are absolute like the current temperature or the current
response time.
The "/var/opt/OverTime/cfgs/http_time.pl"
(or on Windows "c:\OverTime\bin\http_time.pl") is
the command, script or executable to run, including its full path.
“Web_Responses” is the name that will be used on the main OverTime HTML page
and must also be a valid directory name. It may be a new name or the name of an
existing OverTime device.
“www.myco.com
www.myotherco.com” is the list of targets to be tested.
6) Once this has completed, you will be able to browse to results via the
OverTime index.html page. (Remember, you won't see any results until the first
collection has completed)
If "sysName" does not already appear
in the index.html page for OverTime, then it will be added by otadd.
RRD datafiles will are created and the
configuration of the regularly run (via 'at' or 'cron') command
"PingTime-300" (see PingTime) is modified.
When PingTime-300 is next run, it then calls this
command:
“/var/opt/OverTime/cfgs/http_time.pl Web_Responses www.myco.com
www.myotherco.com”.
You can download the sample Perl script, check_port.pl and it looks like this:
__________________________________________________________________________________________
#!/usr/local/bin/perl
# Web
server checking looks like this:
$port=80; #
80 for http - User chooses what they want!
$send =
"GET /index.html\n"; #
What do you want sent
$expect="\<\/html\>"; # What you expect to be returned if
it worked.
$testing=1; # 1 if we are
testing , 0 if we're really collecting
$threshold
= 0.5; # A
threshold of 0.5 seconds to respond
$runcmd =
"echo Threshold breached."; #
Command to run when threshold is breached
$script_name
= "http_time"; #
The name of this script
########################
Probably won't need to change anything below here ##################
use
IO::Socket;
use
Time::HiRes qw(gettimeofday);
for($ind=1;
$ind< @ARGV; ++$ind){
$resptime = "U";
# Start
the timer here!
# Open the
socket
$start = now();
$soket = IO::Socket::INET->new(Proto
=> "tcp", PeerAddr => $ARGV[$ind] ,PeerPort => $port,
Timeout => 2, );
$resptime="U"; # Failed by default
if ( $! eq '' ) { # Everything OK so far
$soket->autoflush(1);
# Send
some protocol dependant data to the server
print $soket "$send"; # Send something to the port for it to
respond
# Read
back the results
while(<$soket>){
if($testing > 0){ print; }#Print all
the responses if testing is set greater than 1
# Check
for something in the result?
if(/$expect/) {
$resptime = now() - $start;
if ($testing) { print "Expected
response of '$expect' was seen in $resptime seconds for $ARGV[$ind]\n"; }
} # Success
}
close $soket;
} else {
# Tell
someone we've had a problem
print ("Port $port to $ARGV[$ind] did
not open\n");
}
# End the
timer here
# if
$resptime=="U" then the time is not valid so don't check the
thresholds.
# If the
threshold has been breached, run a command!
if($resptime ne "U") {
if($resptime > $threshold) {`$runcmd`;
}
} elsif ($testing) { print "Expected
response of '$expect' was NOT seen for $ARGV[$ind]\n"; }
if ($testing > 0) {
`"/opt/OV/bin/devupd"`; # Just
run an update with no arguments - should see the usage
} else {
`"/opt/OV/bin/devupd" $ARGV[0]
$script_name $ARGV[$ind] Now $resptime`;
}
}
sub now {
my(@tmp) = gettimeofday();
return $tmp[0] + ($tmp[1] / 1000000.0);
}
__________________________________________________________________________________________
You
can download the sample Perl script, time2open.pl
and it looks like this:
__________________________________________________________________________________________
#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Std;
use IO::Socket::INET;
use Time::HiRes qw(gettimeofday);
my
($start,$resptime,$port,$host,$expect,$testing,$threshold,$runcmd);
my($portn,$script_name,$devupdcmd,$ind);
$| = 1;
$port=53;
# 80 for
http, 25 for SMTp, 53 for DNS - User chooses what they want!
$testing=0; #
1 if we are testing , 0 if we're really collecting
$threshold = 0.5; # A threshold of 0.5 seconds to respond
$runcmd = "cmd /c echo Threshold
breached."; # Command to run
when threshold is breached
$script_name = "time2open"; # The name of this script
$devupdcmd = "\"\\Program Files\\HP
OpenView\\bin\\devupd\""; #
The update command
######################## Probably won't need
to change anything below here ##################
for($ind=1; $ind< @ARGV; ++$ind){
($host,$portn)=split /\./,$ARGV[$ind];
$resptime="U"; # Failed by default
$port = ($portn eq "dns" ) ? 53 : $portn; # The name 'dns'
does not resolve
# Start the timer here!
# Open the socket
$start = now();
my
$soket = new IO::Socket::INET->new( PeerAddr=> $host, PeerPort=>
$port, Proto=> 'tcp', Timeout=> 30, );
if (
$soket ) { # Everything OK so far
$soket->autoflush(1);
$resptime = now() - $start; # End the timer here
## while(<$soket>){ print; last;}
shutdown($soket,2);
close $soket;
}
# if $resptime=="U" then the time
is not valid so don't check the thresholds.
# If the threshold has been breached, run a
command!
if($resptime ne "U") {
if($resptime > $threshold) {print `$runcmd`; }
}
if
($testing > 0) {
if($resptime ne "U") {
print "Response time of $resptime observered for $host on
$portn\n";
### print`$devupdcmd`; # Just
run an update with no arguments - should see the usage
} else {
# Tell someone we've had a problem
print ("Port $portn to $host did not open\n");
}
}
else {
print `$devupdcmd $ARGV[0] $script_name $host.$portn Now $resptime`;
}
}
sub now {
my(@tmp)
= gettimeofday();
return
$tmp[0] + ($tmp[1] / 1000000.0);
}
Need to install Hi-Res on
Windows?
Download nmake15 (http://download.microsoft.com/download/vc15/Patch/1.52/W95/EN-US/Nmake15.exe)
and
HiRes
can be downloaded from http://ppm.activestate.com/PPMPackages/zips/6xx-builds-only/Time-HiRes.zip
Make sure your Perl\bin is in your PATH. If
not, set it up in your environment.
unzip
Time-HiRes.zip into a temporary directory and then cd to that directory.
Finally, run “ppm install Time-HiRes.ppd” and your Hi-Res
will be installed.