Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Managing Apache weblog!
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
elmie
Tux's lil' helper
Tux's lil' helper


Joined: 12 Sep 2002
Posts: 123
Location: Brisbane - Australia

PostPosted: Mon Apr 21, 2003 12:16 pm    Post subject: Managing Apache weblog! Reply with quote

For those who run Apache and over time had somehow gathered one very very very large access_log under your log folder. Well here is a way to make use of the log and also try to make some sense out of it! Plus save hard drive room.

So what log am I talking about?

Well you need Apache for a start. Everytime a HTTP request is made on an Apache webserver, a one line log entry is made to access_log somewhere under /var/log or /var/log/apache depending on your distro.

I got tired of reading this large log file, so I had to figure a simple but very effective way of make this file more readable.

For a start on default apache log all the entry in IP addresses and for us who just don't seems to remember every IP addresses out there, we will have to convert it to something much easier to remember.

Lucky for us Apache come with a neat little script which will convert the IP address into hostnames instead, called logresolve.
But doing that still won't solve my problem of a large file size. SO I had to come up with plan, How about a periodic day to day file or month to month or what ever to your liking.

Here is what I did..

To do this I had to somehow save the old log and restart with a fresh one.
I heard that in Red Hat and even some distro will already do this for you. but since I don't use RH or any other distro I had to make my own.

I think RH call it rotating log file or something along those lines....

Here is the fun part, you might want to customize to what ever suits you best.. but for now I just do a day day rotate of the log file.

Here I created a simple rotating log file, save it somewhere like /sbin and call it rotate-access_log and chmod +x to make it executable.

Code:

#!/bin/sh
                                                                               
cd /var/log/apache
                                                                               
# Get yesturday's ( or what ever you want to be) date in yyyy-mm format
DATE=`date -d '1 day ago' '+%Y-%B-%d'`
                                                                               
# Rotate the log
mv access_log access_log-$DATE
kill -USR1 `cat /var/run/apache.pid`
                                                                               
# Wait a couple of minutes
sleep 120
                                                                               
# Resolve hostnames and compress
# ( this here changes the IP numbered addresses into hostnames )
logresolve < access_log-$DATE | gzip > access_log-$DATE.gz

# Remove the unresolved version
rm -f access_log-$DATE



The script is pretty easy to understand, the only thing I really need to explain about the script is the "kill -USR1" command. what that actually does is that it tell apache to reload its configuration file and reopen any log files. If you don't send the -USR1 signal to Apache, after the renaming, it will continue to write on to the renamed log file, and thats not we want, we want to rotate it and start afresh.


It is best running this under crontab, you can set it when ever you want to rotate your log. If your server aren't that busy rotate it every month or so, if is very busy rotate it every day ot week, what ever the case is, you'll figure it out on your own.

Having done that, now you can save a much more periodic time line of your log file and save hard drive space.
If you want to read the file, just unzip it and read it, or use something like webalizer or analog, you should be able to emerge these in gentoo.

Good luck and have fun!
Back to top
View user's profile Send private message
Sanity1971
n00b
n00b


Joined: 13 Sep 2003
Posts: 19
Location: Holland

PostPosted: Sat Sep 13, 2003 9:21 pm    Post subject: ehm, rotate the logs? Reply with quote

It could be that I am missing something but there is a utility for this on every linux. It is called logrotate and does what the original poster wants and a lot more. Like compressing the old logs.
Back to top
View user's profile Send private message
nielchiano
Veteran
Veteran


Joined: 11 Nov 2003
Posts: 1287
Location: 50N 3E

PostPosted: Sun Nov 07, 2004 9:45 pm    Post subject: Re: ehm, rotate the logs? Reply with quote

Sanity1971 wrote:
It could be that I am missing something but there is a utility for this on every linux. It is called logrotate and does what the original poster wants and a lot more. Like compressing the old logs.

Can it also keep logs forever and use the date instead of the usual .0 .1 .2 ...?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum