Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Graphical System Statistics with rrd and some scripts
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
bernjuer
n00b
n00b


Joined: 20 Nov 2002
Posts: 22

PostPosted: Fri Jun 06, 2003 3:11 pm    Post subject: Graphical System Statistics with rrd and some scripts Reply with quote

Requirements

Apache, rrd, perl, bash, lmsensors, metalog,snmp
Optional: SpamAssassin and Amavis


Download

http://www.weihenstephan.org/~bernjuer/rrd/

Remarks

I wanted to monitor some new systems and decided
to use rrd instead of mrtg. Soon I found out that
the setup of rrd ist quite complex, so I decided
to hack some scripts together. Oh, I did this
while having a bad flue.
You have been warned.

Description

A set of scripts to set up a system monitor.

Tracked informations are:
* system load
* trafic on two net devices
* system temperature
* mail trafic

Data is shown in 4 time frames:
* actual (last 3 hours)
* daily (last 24 hours)
* montly (last 5 weeks)
* yearls (last 15 months)

Data is sample every 5 minutes for the net devices and
for load, temperature and email are gathered every 15 minutes.

The graphics are generated on the fly by using rrd's
cgi functionality.




Installation

Create a directory somewhere, I used /home/httpd/rrd.

Create a User called rrd and set his home directory to
the freshly created directory.

Add this user to the group "cron" by writing rrd after
the line starting with "cron" in /etc/group. The line
should look like "cron:x:16:cron,bernd,rrd".

Allow apache to enter and this directory
"chown .apache /home/httpd/rrd"
"chmod g+rwx /home/httpd/rrd"
Allow Apache to run CGIs in the chosen directory
by adding something like

Code:
<Directory /home/httpd/htdocs/rrd>
    AllowOverride All
    Options ExecCGI
    Order allow,deny
    Allow from all
</Directory>


to Apache's config file (/etc/apache/conf/commonapache.conf).
Restart Apache by calling "/etc/init.d/apache restart".

If you want to analyze mail logs, the user rrd must have
read access to /var/log/mail/current. To do this,
create a group called logger with the user rrd by adding a
line like"logger:x:82:rrd" to /etc/group.
Allow read access to the directory by calling
"chown .logger /var/log/mail"
"chmod g+rx /var/log/mail"

"su" to the new user ("su rrd").
Create the rrd databases by running createrrds.sh

createrrds.sh is:

Code:

#!/bin/bash

#
# bernds basic aproach to data gathering
# V0.1  GPL, no warranty, yadadada...
#

# data for eth0, interval 5 min (--step 300)
# UNKNOWN after 10min w/o data (600)
#
# Store every sample (1) for ten days
#         10*24*60*60 /300
#
#        2880 samples* 300s per sample / 60 / 60 / 24 = 10
#         
#
# Store average of every half hour (6) for 50 days
#         100*24*60*60 / 300 / 6 
#
#        2400 samples* 30min per avg / 60 / 24 = 50 
#     
# Store average of every 3 hours (36) for 500 days (1.5 years)
#
#        4000 samples* 3h per avg / 24 = 50

rrdtool create eth0.rrd  --step 300    \
            DS:input:COUNTER:600:0:U   \
            DS:output:COUNTER:600:0:U  \
            RRA:AVERAGE:0.5:1:2880      \
            RRA:AVERAGE:0.5:6:2400    \
            RRA:AVERAGE:0.5:36:4000   



#data for eth1, 5 min (300 s)

rrdtool create eth1.rrd  --step 300       \
            DS:input:COUNTER:600:0:U   \
            DS:output:COUNTER:600:0:U  \
            RRA:AVERAGE:0.5:1:2880      \
            RRA:AVERAGE:0.5:6:2400    \
            RRA:AVERAGE:0.5:36:4000   

#data for load, 5 min

rrdtool create load.rrd   --step 300      \
            DS:val:COUNTER:600:0:U   \
            RRA:AVERAGE:0.5:1:2880      \
            RRA:AVERAGE:0.5:6:2400    \
            RRA:AVERAGE:0.5:36:4000   
           
#data for temp, 15 min (900s )

rrdtool create temp.rrd  --step 900       \
            DS:temp1:GAUGE:1800:0:U   \
            DS:temp2:GAUGE:1800:0:U   \
            RRA:AVERAGE:0.5:1:2880      \
            RRA:AVERAGE:0.5:3:2400    \
            RRA:AVERAGE:0.5:12:4000   

#data for mail, 15 min

rrdtool create mail.rrd  --step 900       \
            DS:in:COUNTER:1800:0:U   \
            DS:out:COUNTER:1800:0:U   \
            DS:spam:COUNTER:1800:0:U   \
            DS:virus:COUNTER:1800:0:U   \
            RRA:AVERAGE:0.5:1:2880      \
            RRA:AVERAGE:0.5:3:2400    \
            RRA:AVERAGE:0.5:12:4000   




Create the CGI scripts by running createCGIs.sh

createCGIs.sh is:

Code:
#!/bin/bash
./createCGI.pl 3 > index.cgi
./createCGI.pl 0 > day.cgi
./createCGI.pl 1 > month.cgi
./createCGI.pl 2 > year.cgi
chmod +x *cgi
chown apache.apache *cgi


createCGI.pl is:

Code:
#!/usr/bin/perl
use strict;

my @temps =("Temperatur","temp.rrd","temp1","temp2");
my @load  =("Last","load.rrd","val");
my @mail  =("EMail","mail.rrd","in","out","spam","virus");
my @eth0  =("eth0","eth0.rrd","input","output");
my @eth1  =("eth1","eth1.rrd","input","output");

my @sysinf=(\@temps,\@load,\@mail,\@eth0,\@eth1);
my @colors=("#00c000","#0000c0","#404040","#700000");

#      <RRD::GRAPH tempa.gif --start -10800 -A --lazy --title="Temperatur"  -h 80 -w 150
#                DEF:cel1=temp.rrd:temp1:AVERAGE
#                DEF:cel2=temp.rrd:temp2:AVERAGE
#           LINE1:cel1#00a000:"CPU"
#           LINE1:cel2#0000a0:"Board"> 

my $OPTS="-A -i "; # --lazy

my $SIZELARGE="-h 80 -w 610";
my $SIZESMALL="-h 80 -w 140";

my $BACKSMALL=-1;
my $BACKLARGE=-1;
my $PAGENAME="";

my $par = $ARGV[0];

if ($par == "0") {
  $BACKSMALL=60*60*3;      #3h
  $BACKLARGE=60*60*24*1.5; #36h
  $PAGENAME ="Aktuell";
}

if ($par == "1") {
  $BACKSMALL=60*60*24*6;   #1w
  $BACKLARGE=60*60*24*7*5; #1m
  $PAGENAME ="Monat";
}

if ($par == "2") {
  $BACKSMALL=60*60*24*40;   #1m
  $BACKLARGE=60*60*24*500;  #1y
  $PAGENAME ="Jahr";

}


if ($par == "3") {
  $BACKSMALL=60*60*6;   #6h
  $PAGENAME ="Systeminfo";
  $SIZESMALL="-h 200 -w 300";
}




if ($BACKSMALL<0) { die "no para found, use {0,1,2,3}";}

printf("#!/usr/bin/rrdcgi\n");
printf("<HTML>\n".
 "<!-- page generated by createCGI by bj 2003-6-6 (para ".$ARGV[0].") -->\n".
 "<meta http-equiv=REFRESH content=120>\n".
 "<HEAD><TITLE>$PAGENAME</TITLE></HEAD>\n".
 "<BODY bgcolor=\"#a0a0a0\">\n<CENTER>\n");

if ($par != "3") {
  print(" <TABLE border=0>");
  for (my $i=0;$i<5;$i++) {
    print(" <TR>\n");
    print("  <TD>\n");
    printOnePic($i,$OPTS,$SIZELARGE,$BACKLARGE);
    print("  </TD>\n");
    print("  <TD>\n");
    printOnePic($i,$OPTS,$SIZESMALL,$BACKSMALL);
    print("  </TD>\n");
    print(" </TR>\n");
  }
  print(" </TABLE>\n");
  print("<p>\n");
  print("<a href=\"index.cgi\">Zurück</a><br>\n");

} else {
  my $sysname = `uname -n`;
  print(" <h2> ".$sysname." </h2>");
  print(" <TABLE border=0>");
  print("  <TR>\n");
  print("  <TD>\n");
  printOnePic(0,$OPTS,$SIZESMALL,$BACKSMALL);
  print("  </TD>\n");
  print("  <TD>\n");
  printOnePic(2,$OPTS,$SIZESMALL,$BACKSMALL);
  print("  </TD>\n");
  print("  </TR><TR>\n");
  print("  <TD>\n");
  printOnePic(3,$OPTS,$SIZESMALL,$BACKSMALL);
  print("  </TD>\n");
  print("  <TD>\n");
  printOnePic(4,$OPTS,$SIZESMALL,$BACKSMALL);
  print("  </TD>\n");
  print(" </TR>\n");
 
  print(" </TABLE>\n");
  print("<p>\n");
  print("<a href=\"day.cgi\">Heute</a><br>\n");
  print("<a href=\"month.cgi\">Monat</a><br>\n");
  print("<a href=\"year.cgi\">Jahr</a><br>\n");
}
printf("</CENTER>\n</BODY>\n</HTML>\n");




sub printOnePic {
  my $i = @_[0];
  my $opts= @_[1];
  my $size= @_[2];
  my $range= @_[3];

  my $title=$sysinf[$i][0];
  my $name= $sysinf[$i][1];

  printf("  <RRD::GRAPH $PAGENAME".$i.$range.".gif --start -$range $opts --title=\"$title\" $size \n");
  my $start=2;
  for (my $j=$start;$j<10+$start;$j++) {
    my $val =$sysinf[$i][$j];
    if ($val eq "") { next; }
    printf("   DEF:val$j=$name:$val:AVERAGE\n");
  }
  for (my $j=$start;$j<10+$start;$j++) {
    my $val =$sysinf[$i][$j];
    if ($val eq "") { next; }
    if ($j==$start) {
      printf("   LINE2:val$j".$colors[$j-$start].":$val\n");

    } else {
      printf("   LINE2:val$j".$colors[$j-$start].":$val\n");
    }
  }
  printf("  >\n\n");
}




So we have the databases and the scripts. Now we need to
populate the database.
Verify the two scripts getmailtemp.pl and getstat.pl.

getmailtemp.pl is

Code:
#!/usr/bin/perl
#
# bernds basic aproach to statistics gathering
# V0.1  GPL, no warranty, yadadada...
#
# this should run every 5 mins
# the line in crontab looks like this:
# */5  * * * * /home/httpd/htdocs/...../rrd/getstat.pl


use strict;

my $debug=0;
my $line,
my $t1=0;
my $t2=0;
my $dummy;
my $res1;

my $rrt="/usr/bin/rrdtool";
my $sns="/usr/bin/sensors";
my $mail="/var/log/mail/current";


if ($ARGV[0] ne "") {
  printf("Debug mode, no data is written\n");
  $debug=1;
}

# mail stats. requires postfix, metalog, spamassasin and amavis
my $msent= doexec("grep -c \"250 O\" $mail");       # sent mails, response 250
my $mrx=   doexec("grep -c \"relay=local\" $mail"); # received mails

my $mspm=  doexec("grep -c \"identified spam\" $mail");  # message from spamassasin
my $mvir=  doexec("grep -c \"Virus found - quarantined\" $mail"); # message from amavis

if ($debug==0) {
  $res1=system("$rrt update mail.rrd N:$mrx:$msent:$mspm:$mvir");
} else {
  printf("Mail: Sent $mrx Rxed $msent Spam $mspm  Virus $mvir\n");
}


# requires lm_sensors
#
# adapt to your needs: my board has two temp sensors

concat(FH,"$sns|");

while ($line=<FH>)  {

        # Apollo

        #start of line followed by "temp1:"
        if ($line =~ /^temp1:/) {
                ($dummy,$t1)=split(/ /,$line);
        }

        if ($line =~ /^temp2:/) {
                ($dummy,$t2)=split(/ /,$line);
        }


   # Via Eden

        #start of line followed by "MB1: Temp"
   if ($line =~ /^MB1 Temp:/) {
     $t1=0; # no actual temp
   }

   if ($line =~ /^Proc Temp:/) {
     $t2=substr $line,12,4;
   }
}

if ($debug==0) {
  $res1=system("$rrt update temp.rrd N:$t1:$t2");
} else {
  print ("Temps $t1 $t2\n");
}


# execute external program and return retval as integer
sub doexec {
   open(FH,"@_|") or die ("cant exec @_");
   my $line =<FH>;
   return $line+0;
}



getstat.pl is
Code:

#!/usr/bin/perl
#
# bernds basic aproach to data gathering
# V0.1  GPL, no warranty, yadadada...
#
# this should get called every 5 mins
# the line in crontab looks like this:
# */5  * * * * /home/httpd/htdocs/ ... /rrd/getstat.pl

use strict;
use Net::SNMP;

my $debug=0;
my $rrt="/usr/bin/rrdtool";
my $datapath="/usr/bin/rrdtool";
my $res1="";
my $res2="";
my $ld=0;
#
# Adjust to match your needs
#
my $s = Net::SNMP->session( -hostname  => "127.0.0.1",
                            -community => "public" );


if ($ARGV[0] ne "") {
  printf("Debug mode, no data is written\n");
  $debug=1;
}


#
#  Get OIDs by calling
#
#snmpwalk localhost public interfaces.ifTable.ifEntry.ifInOctets  -On
#snmpwalk localhost public interfaces.ifTable.ifEntry.ifOutOctets -On
#

my $eth0in  = '.1.3.6.1.2.1.2.2.1.10.2';
my $eth0out = '.1.3.6.1.2.1.2.2.1.16.2';
my $eth1in  = '.1.3.6.1.2.1.2.2.1.10.3';
my $eth1out = '.1.3.6.1.2.1.2.2.1.16.3';


# fetch data from uptime
my $up=`/usr/bin/uptime` or die "Can't get uptime $!\n";
($res1,$res2) = split("average: ",$up);
($ld) = split(",",$res2);


if ($debug==1) {
  printf("Load $ld ($res1) ($up)\n");
} else {
  $res1=system("$rrt update load.rrd N:$ld");
}



# fetch data
my $res = $s->get_request(-varbindlist => [$eth0in,$eth0out,$eth1in,$eth1out]);


# throw data at rrd

if ($debug==1) {

  printf("eth0 bytes in $res->{$eth0in} out $res->{$eth0out}\n");
  printf("eth1 bytes in $res->{$eth1in} out $res->{$eth1out}\n");

} else {

  $res1=system("$rrt update eth0.rrd N:$res->{$eth0in}:$res->{$eth0out}");
  $res1=system("$rrt update eth1.rrd N:$res->{$eth1in}:$res->{$eth1out}");
}



Test them by calling them with any parameter, thus enabling debug mode.
That is :

"./getmailtemp.pl -deb"

should return something like

Code:
Debug mode, no data is written
Mail: Sent 6 Rxed 0 Spam 0  Virus 0
Temps 0 49.6



"./getstat.pl -deb"
should return something like
Code:

Debug mode, no data is written
Load 0.05 (0.05,)
eth0 bytes in 68454004 out 5938416
eth1 bytes in 39313841 out 126012954



Now add these scripts to cron by calling
"crontab -e" and pasting these two lines:
Code:

*/15 * * * * /home/httpd/htdocs/rrd/getmailtemp.pl > /dev/null 2>&1
*/5  * * * * /home/httpd/htdocs/rrd/getstat.pl > /dev/null 2>&1



Go have a party or something and after a while you should
see some lines when pointing mozilla to http://yourhost/rrd/index.cgi[url][/url]
_________________
sigs are boring
Back to top
View user's profile Send private message
togge
n00b
n00b


Joined: 17 Mar 2003
Posts: 14

PostPosted: Thu Jun 19, 2003 9:21 pm    Post subject: Reply with quote

Nice post :wink:

But could you tell us more about the snmp part ?
Maybe give an snmpd.conf example
Back to top
View user's profile Send private message
smouge
n00b
n00b


Joined: 22 Jan 2003
Posts: 66
Location: Oosterhout, the Netherlands

PostPosted: Thu Jun 26, 2003 8:03 pm    Post subject: Reply with quote

THe following link might be interesting for you as well:

http://hotsanic.sourceforge.net/

THis provides some easy ready made scripts to use.
Back to top
View user's profile Send private message
RaymaN
Tux's lil' helper
Tux's lil' helper


Joined: 15 Aug 2002
Posts: 93
Location: Earth, Europe, Germany, Berlin

PostPosted: Wed Jul 23, 2003 9:22 am    Post subject: Reply with quote

@ bernjuer:

Very nice :) Works with the standard syslogd ??

ray
Back to top
View user's profile Send private message
RaymaN
Tux's lil' helper
Tux's lil' helper


Joined: 15 Aug 2002
Posts: 93
Location: Earth, Europe, Germany, Berlin

PostPosted: Wed Jul 23, 2003 9:22 am    Post subject: Reply with quote

@ bernjuer:

Very nice :) Works with the standard syslogd ??

ray
Back to top
View user's profile Send private message
smouge
n00b
n00b


Joined: 22 Jan 2003
Posts: 66
Location: Oosterhout, the Netherlands

PostPosted: Fri Jul 25, 2003 6:38 am    Post subject: Reply with quote

I'm not sure, but the ready made scripts/plugins are mostly in perl, so you can see what is happening.

A lot of info it gets from /proc , network traffic by snmp

I haven't been progrmming new plug-ins , but at leat this HotSaNIC will give you a nice frame to place more plugins, and you can use the included plugins like eth by snmp almost out from the box.

There is no ebuild, but you could install it from source
Back to top
View user's profile Send private message
swimmer
Veteran
Veteran


Joined: 15 Jul 2002
Posts: 1314
Location: Netherlands

PostPosted: Fri Jul 25, 2003 9:10 am    Post subject: Reply with quote

Nice work Bernd ...

Greetz
Stefan ;-)
Back to top
View user's profile Send private message
cmay4
Tux's lil' helper
Tux's lil' helper


Joined: 18 May 2002
Posts: 109

PostPosted: Fri Jul 25, 2003 4:01 pm    Post subject: Another consideration Reply with quote

Take a look at Cacti:

Code:
emerge cacti

Pretty full featured with an excellent interface for setting types of graphs and options. Unfortunately requires MySQL (no PostgreSQL support yet).
Back to top
View user's profile Send private message
Jevin Sweval
n00b
n00b


Joined: 19 Jun 2003
Posts: 64
Location: Indiana

PostPosted: Sat Jul 26, 2003 5:37 pm    Post subject: Re: Another consideration Reply with quote

cmay4 wrote:
Take a look at Cacti:

Code:
emerge cacti

Pretty full featured with an excellent interface for setting types of graphs and options. Unfortunately requires MySQL (no PostgreSQL support yet).
exactly!

i even have it monitoring the number of users on a vbulletin forum i visit via a webscrapping perl script that uses WWW::Mechanize. i have it monitor the number of pages printed on the home office printer via SNMP! :D

it is hosted on my Rackshack server. you can check it out at http://www.futagollc.com/cacti/static/


the interface to add new hosts, data sources, and graphs is very easy to use. I made a graph template so that I can monitor the number of users on ANY vbulletin forum with just a few clicks! :D :D :D
_________________
Help me! :(
Back to top
View user's profile Send private message
dylix
Apprentice
Apprentice


Joined: 20 Sep 2002
Posts: 261
Location: Atlanta

PostPosted: Sun Jul 27, 2003 5:51 am    Post subject: Reply with quote

mrtg 4life!!!

http://plutonic.homelinux.net/mrtg/
_________________

Rape is such a strong word, I prefer surprise sex.
http://plutonic.mine.nu/
Back to top
View user's profile Send private message
smouge
n00b
n00b


Joined: 22 Jan 2003
Posts: 66
Location: Oosterhout, the Netherlands

PostPosted: Sun Aug 03, 2003 6:40 pm    Post subject: Reply with quote

Just made an ebuild for HotSaNIC. You can try it out if you want. See:

https://forums.gentoo.org/viewtopic.php?t=72160
Back to top
View user's profile Send private message
Guezz-Who
Tux's lil' helper
Tux's lil' helper


Joined: 15 Mar 2003
Posts: 90
Location: Norway::Bergen

PostPosted: Sun Aug 03, 2003 7:14 pm    Post subject: LRRD Reply with quote

You might also want to take a look at lrrd, i run it here: http://lurium.guezz.net/lrrd/guezz.net/lurium.guezz.net.html

From lrrd's homepage ( http://lrrd.sf.net/ )

Quote:
Lrrd is a tool for graphing all sorts of information about one or more servers and displaying it in a web interface. It uses the execellent RRDTool (written by Tobi Oetiker) and is written in Perl. LRRD has a server/client architecture. The server connects to all the clients at regular intervals, and asks them for data. It then stores the data in RRD-files, and (if needed) updates the graphs. One of the main goals have been ease of creating own "plugins" (graphs).


Too bad there are no ebuilds yet.. and some of the perl modules arn't even in portage (at least not when I installed it).. You can get them via CPAN though..
_________________
If you do something right once, someone will ask you to do it again.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum