Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
logrotate-3.8.7 deletes my apache logfiles
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
geeksheik
Tux's lil' helper
Tux's lil' helper


Joined: 07 Sep 2003
Posts: 95
Location: Zürich, Switzerland

PostPosted: Thu May 01, 2014 7:59 pm    Post subject: logrotate-3.8.7 deletes my apache logfiles Reply with quote

It's up-to-date with the nearly default config.

I may have changed 'rotate 4' to 'rotate 104' in logrotate.conf first, but this was the only setting modified. I changed it back and still have the same effect.

I've just created a repeatable script that tests the problem (note: PWD is /etc/logrotate.d and the config file writes to /var/log/apache_test, so as not to disrupt the installed version):

Code:

#!/bin/bash
set -o verbose
rm -rf /var/log/apache2_test/
cp -a /var/log/apache2 /var/log/apache2_test
ls -alh /var/log/apache2*
logrotate -f apache2
ls -alh /var/log/apache2*
set +o verbose


This produces the results:

Code:

# cat test_logrotate.sh | /bin/bash
rm -rf /var/log/apache2_test/
cp -a /var/log/apache2 /var/log/apache2_test
ls -alh /var/log/apache2*
/var/log/apache2:
total 48K
drwxr-x--- 2 apache apache 4.0K May  1 17:28 .
drwxr-xr-x 6 root   root   4.0K May  1 19:01 ..
-rw-r--r-- 1 root   root      0 Apr 29 01:14 .keep_www-servers_apache-2
-rw-r--r-- 1 root   root   6.2K May  1 18:58 access.log
-rw-r--r-- 1 root   root   1.1K May  1 18:58 agent.log
-rw-r--r-- 1 root   root    20K May  1 18:58 error.log
-rw-r--r-- 1 root   root    210 May  1 18:58 referer.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_access.log
-rw-r--r-- 1 root   root   1.6K May  1 18:58 ssl_error.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_request.log

/var/log/apache2_test:
total 48K
drwxr-x--- 2 apache apache 4.0K May  1 17:28 .
drwxr-xr-x 6 root   root   4.0K May  1 19:01 ..
-rw-r--r-- 1 root   root      0 Apr 29 01:14 .keep_www-servers_apache-2
-rw-r--r-- 1 root   root   6.2K May  1 18:58 access.log
-rw-r--r-- 1 root   root   1.1K May  1 18:58 agent.log
-rw-r--r-- 1 root   root    20K May  1 18:58 error.log
-rw-r--r-- 1 root   root    210 May  1 18:58 referer.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_access.log
-rw-r--r-- 1 root   root   1.6K May  1 18:58 ssl_error.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_request.log
logrotate -f apache2
ls -alh /var/log/apache2*
/var/log/apache2:
total 52K
drwxr-x--- 2 apache apache 4.0K May  1 17:28 .
drwxr-xr-x 6 root   root   4.0K May  1 19:01 ..
-rw-r--r-- 1 root   root      0 Apr 29 01:14 .keep_www-servers_apache-2
-rw-r--r-- 1 root   root   6.6K May  1 19:01 access.log
-rw-r--r-- 1 root   root   1.1K May  1 19:01 agent.log
-rw-r--r-- 1 root   root    23K May  1 19:01 error.log
-rw-r--r-- 1 root   root    224 May  1 19:01 referer.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_access.log
-rw-r--r-- 1 root   root   1.9K May  1 19:01 ssl_error.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_request.log

/var/log/apache2_test:
total 8.0K
drwxr-x--- 2 apache apache 4.0K May  1 19:01 .
drwxr-xr-x 6 root   root   4.0K May  1 19:01 ..
-rw-r--r-- 1 root   root      0 Apr 29 01:14 .keep_www-servers_apache-2
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_access.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_request.log
set +o verbose


The logfiles are deleted (zero'd)

...even if the 'create' is disabled in logrotate.conf, the problem still happens.

Here's my current logrotate.conf:

Code:

# $Header: /var/cvsroot/gentoo-x86/app-admin/logrotate/files/logrotate.conf,v 1.5 2013/05/18 09:41:04 nimiux Exp $
#
# Logrotate default configuration file for Gentoo Linux
#
# See "man logrotate" for details

# rotate log files weekly
weekly
#daily

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# use date as a suffix of the rotated file
dateext

# uncomment this if you want your log files compressed
compress

notifempty
nomail
noolddir

# packages can drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp and btmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    create 0664 root utmp
        minsize 1M
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0600 root utmp
    rotate 1
}

# system-specific logs may be also be configured here.


Which I think is exactly default, but it was edited.

Contents of /etc/logrotate.d/apache2 (default installed with apache-2.2.7):

Code:

-> cat /etc/logrotate.d/apache2
# Apache2 logrotate snipet for Gentoo Linux
# Contributes by Chuck Short
#
/var/log/apache2_test/*log {
  missingok
  notifempty
  sharedscripts
  postrotate
  /etc/init.d/apache2 reload > /dev/null 2>&1 || true
  endscript
}


...and version info:

Code:
-> equery list apache
 * Searching for apache ...
[IP-] [  ] www-servers/apache-2.2.27:2

-> equery list logrotate
 * Searching for logrotate ...
[IP-] [  ] app-admin/logrotate-3.8.7:0


Is this one for Bugzilla? I didn't see anything in either bugzilla or the forums about it yet.
Back to top
View user's profile Send private message
geeksheik
Tux's lil' helper
Tux's lil' helper


Joined: 07 Sep 2003
Posts: 95
Location: Zürich, Switzerland

PostPosted: Thu May 01, 2014 9:11 pm    Post subject: mask logrotate-3.8.7 Reply with quote

I've just masked 3.8.7, installed 3.8.6 (the latest available alternative) and the problem is solved.

Code:
-> cat test_logrotate.sh | /bin/bash
rm -rf /var/log/apache2_test/
cp -a /var/log/apache2 /var/log/apache2_test
ls -alh /var/log/apache2*
/var/log/apache2:
total 68K
drwxr-x--- 2 apache apache 4.0K May  1 17:28 .
drwxr-xr-x 6 root   root   4.0K May  1 19:56 ..
-rw-r--r-- 1 root   root      0 Apr 29 01:14 .keep_www-servers_apache-2
-rw-r--r-- 1 root   root   9.3K May  1 19:09 access.log
-rw-r--r-- 1 root   root   1.6K May  1 19:09 agent.log
-rw-r--r-- 1 root   root    33K May  1 19:09 error.log
-rw-r--r-- 1 root   root    315 May  1 19:09 referer.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_access.log
-rw-r--r-- 1 root   root   2.4K May  1 19:09 ssl_error.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_request.log

/var/log/apache2_test:
total 68K
drwxr-x--- 2 apache apache 4.0K May  1 17:28 .
drwxr-xr-x 6 root   root   4.0K May  1 19:56 ..
-rw-r--r-- 1 root   root      0 Apr 29 01:14 .keep_www-servers_apache-2
-rw-r--r-- 1 root   root   9.3K May  1 19:09 access.log
-rw-r--r-- 1 root   root   1.6K May  1 19:09 agent.log
-rw-r--r-- 1 root   root    33K May  1 19:09 error.log
-rw-r--r-- 1 root   root    315 May  1 19:09 referer.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_access.log
-rw-r--r-- 1 root   root   2.4K May  1 19:09 ssl_error.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_request.log
logrotate -f apache2
ls -alh /var/log/apache2*
/var/log/apache2:
total 68K
drwxr-x--- 2 apache apache 4.0K May  1 17:28 .
drwxr-xr-x 6 root   root   4.0K May  1 19:56 ..
-rw-r--r-- 1 root   root      0 Apr 29 01:14 .keep_www-servers_apache-2
-rw-r--r-- 1 root   root   9.3K May  1 19:09 access.log
-rw-r--r-- 1 root   root   1.6K May  1 19:09 agent.log
-rw-r--r-- 1 root   root    33K May  1 19:09 error.log
-rw-r--r-- 1 root   root    315 May  1 19:09 referer.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_access.log
-rw-r--r-- 1 root   root   2.4K May  1 19:09 ssl_error.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_request.log

/var/log/apache2_test:
total 68K
drwxr-x--- 2 apache apache 4.0K May  1 17:28 .
drwxr-xr-x 6 root   root   4.0K May  1 19:56 ..
-rw-r--r-- 1 root   root      0 Apr 29 01:14 .keep_www-servers_apache-2
-rw-r--r-- 1 root   root   9.3K May  1 19:09 access.log
-rw-r--r-- 1 root   root   1.6K May  1 19:09 agent.log
-rw-r--r-- 1 root   root    33K May  1 19:09 error.log
-rw-r--r-- 1 root   root    315 May  1 19:09 referer.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_access.log
-rw-r--r-- 1 root   root   2.4K May  1 19:09 ssl_error.log
-rw-r--r-- 1 root   root      0 May  1 17:28 ssl_request.log
set +o verbose

-> equery list logrotate
 * Searching for logrotate ...
[IP-] [  ] app-admin/logrotate-3.8.6:0


A switch back to 3.8.7 shows that it's broken again.

This looks like a very serious issue with the latest available version of logrotate. I'm just reinstalling on a second system to verify...

Bummer I lost the log files; I really needed them. I did a snapshot of the drive, hopefully I can find it in the freed space...


I recommend masking logrotate-3.8.7, and do so asap if you want to preserve your current log files

Depending on your configuration, it might be either of these lines:

Code:

mkdir -p /etc/portage/package.mask/ ; echo "=app-admin/logrotate-3.8.7" >> /etc/portage/package.mask/logrotate


If your portage.mask already exists as a file, do this:

Code:

echo "=app-admin/logrotate-3.8.7" >> /etc/portage/package.mask


Last edited by geeksheik on Thu May 01, 2014 9:33 pm; edited 1 time in total
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 10261
Location: Somewhere over Atlanta, Georgia

PostPosted: Thu May 01, 2014 9:17 pm    Post subject: Reply with quote

This would be worth a bug report on Gentoo Bugzilla if you don't mind.

- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 3959
Location: Dallas area

PostPosted: Thu May 01, 2014 10:06 pm    Post subject: Reply with quote

I haven't had a problem with it and I've been running it since Oct 2013.

Maybe it's an interaction with something else.

What does "/var/lib/logrotate.status" have in it?

Edit to add: I had turned off apache a while back, and just turned it on to test it. You're right it has a problem.

Edit to add2: The previous version (3.8.6) wouldn't do anything with the files even though it was forced.
Updated to 3.8.7 again and it "tried" to do something. ( I turned on verbose) and the output was

Code:
 /var/log/apache2 $ sudo logrotate -fv /etc/logrotate.d/apache2
reading config file /etc/logrotate.d/apache2

Handling 1 logs

rotating pattern: /var/log/apache2/*log  forced from command line (no old logs will be kept)
empty log files are not rotated, old logs are removed
considering log /var/log/apache2/access_log
  log does not need rotating
considering log /var/log/apache2/error_log
  log needs rotating
considering log /var/log/apache2/ssl_access_log
  log does not need rotating
considering log /var/log/apache2/ssl_error_log
  log does not need rotating
considering log /var/log/apache2/ssl_request_log
  log does not need rotating
rotating log /var/log/apache2/error_log, log->rotateCount is 0
dateext suffix '-20140501'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
renaming /var/log/apache2/error_log.1 to /var/log/apache2/error_log.2 (rotatecount 1, logstart 1, i 1),
old log /var/log/apache2/error_log.1 does not exist
renaming /var/log/apache2/error_log.0 to /var/log/apache2/error_log.1 (rotatecount 1, logstart 1, i 0),
old log /var/log/apache2/error_log.0 does not exist
log /var/log/apache2/error_log.2 doesn't exist -- won't try to dispose of it
renaming /var/log/apache2/error_log to /var/log/apache2/error_log.1
disposeName will be /var/log/apache2/error_log.1
running postrotate script
removing old log /var/log/apache2/error_log.1


And it did not move the old error_log file to error_log.1.gz
and it did not create a new empty error_log file.
_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.0 (no-pie) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon


Last edited by Anon-E-moose on Thu May 01, 2014 11:02 pm; edited 3 times in total
Back to top
View user's profile Send private message
geeksheik
Tux's lil' helper
Tux's lil' helper


Joined: 07 Sep 2003
Posts: 95
Location: Zürich, Switzerland

PostPosted: Thu May 01, 2014 10:23 pm    Post subject: Gentoo bugzilla entry Reply with quote

https://bugs.gentoo.org/show_bug.cgi?id=509328
Back to top
View user's profile Send private message
geeksheik
Tux's lil' helper
Tux's lil' helper


Joined: 07 Sep 2003
Posts: 95
Location: Zürich, Switzerland

PostPosted: Thu May 01, 2014 11:24 pm    Post subject: Reply with quote

Puzzling...on re-install on both systems, both seem to be working correctly with 3.8.7. At the moment, I don't know how to reproduce the issue.

I'm afraid that I didn't keep the logrotate.status from each run. The most recent runs were successful:

System #1 (with the 3.8.6 logrotate info):

Quote:
-> cat /var/lib/logrotate.status
logrotate state -- version 2
"/var/log/rsync.log" 2010-12-31-4:0:0
"/var/log/apache2/elektrogeddon_ssl_error.log" 2014-5-1-23:52:26
"/var/log/apache2/ssl_request.log" 2014-4-14-3:10:15
"/var/log/rc.log" 2014-2-24-1:10:9
"/var/log/apache2/ssl_request_log" 2013-10-5-13:0:0
"/var/log/apache2/referer.log" 2014-4-30-3:10:10
"/var/log/portage/elog/summary.log" 2014-4-28-3:10:11
"/var/log/apache2/access.log" 2014-4-30-3:10:10
"/var/log/hibernate.log" 2010-12-31-4:0:0
"/var/log/apache2/elektrogeddon_ssl_access.log" 2014-3-6-1:0:0
"/var/log/wtmp" 2013-10-6-13:0:0
"/var/log/apache2/access_log" 2011-8-26-13:0:0
"/var/log/mysql/mysqld.err" 2010-12-31-4:0:0
"/var/log/btmp" 2010-12-31-4:0:0
"/var/log/apache2/ssl_access.log" 2014-4-14-3:10:15
"/var/log/mysql/mysql.log" 2010-12-31-4:0:0
"/var/log/apache2/ssl_access_log" 2013-10-5-13:0:0
"/var/log/apache2/error.log" 2014-5-1-23:52:26
"/var/log/apache2/ssl_error.log" 2014-5-1-23:52:26
"/var/log/apache2/agent.log" 2014-4-30-3:10:10
"/var/log/apache2/error_log" 2013-10-5-13:0:0
"/var/log/mysql/mysql.err" 2010-12-31-4:0:0
"/var/log/apache2/ssl_error_log" 2013-10-5-13:0:0
"/var/log/kdm.log" 2011-5-25-13:0:0
"/var/log/messages" 2013-10-18-13:0:0
"/var/log/vsftpd.log" 2010-12-31-4:0:0


System #2 (3.8.7 with a successful rotate):
Quote:
-> cat /var/lib/logrotate.status
logrotate state -- version 2
"/var/log/apache2/ssl_request.log" 2014-5-1-17:28:10
"/var/log/apache2_test/ssl_error.log" 2014-5-1-21:1:34
"/var/log/apache2/ssl_request_log" 2014-5-1-17:28:10
"/var/log/apache2/referer.log" 2014-5-1-17:28:10
"/var/log/apache2_test/ssl_request.log" 2014-5-1-17:0:0
"/var/log/apache2_test/error.log" 2014-5-1-21:1:34
"/var/log/apache2/access.log" 2014-5-1-17:28:10
"/var/log/apache2_test/ssl_access.log" 2014-5-1-17:0:0
"/var/log/apache2_test/agent.log" 2014-5-1-21:1:34
"/var/log/apache2/access_log" 2014-5-1-17:28:10
"/var/log/apache2_test/access.log" 2014-5-1-21:1:34
"/var/log/apache2/ssl_access.log" 2014-5-1-17:28:10
"/var/log/apache2_test/referer.log" 2014-5-1-21:1:34
"/var/log/apache2/ssl_access_log" 2014-5-1-17:28:10
"/var/log/apache2/error.log" 2014-5-1-17:28:10
"/var/log/apache2/ssl_error.log" 2014-5-1-17:28:10
"/var/log/apache2/error_log" 2014-5-1-17:27:53
"/var/log/apache2/agent.log" 2014-5-1-17:28:10
"/var/log/apache2/ssl_error_log" 2014-5-1-17:28:10
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 3959
Location: Dallas area

PostPosted: Thu May 01, 2014 11:39 pm    Post subject: Reply with quote

To get 3.8.7 to handle apache logs properly, I had to add these three lines to logrotate.d/apache2
Code:
create
compress
rotate 4


It seemed to lose the understanding of those 3 from the logrotate.conf.

Peculiar.

Edit to add: It must have something to do with the name starting with "a" some type of sorting where the apache file is losing global parms.
_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.0 (no-pie) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum