rsyslog 8.33.0: "/etc/init.d/rsyslog rotate" fails

[atc0005]

On two clean Ubuntu VMs (one 14.04, the other 16.04) I enabled the adiscon/v8-stable PPA and installed the 8.33.0 version of rsyslog. I then proceeded to try out the 'rotate' argument via the init.d script. For Ubuntu 14.04 systems there is no issue. For 16.04, a "fail" message is given.

root@ubuntu-1404-virtual-machine:~# /etc/init.d/rsyslog rotate

 * Closing open files rsyslogd                                           [ OK ] 

root@ubuntu-1604-virtual-machine:~# /etc/init.d/rsyslog rotate

 * Closing open files rsyslogd                                           [fail] 

No changes were made to the conf files on either system. I've not yet dug into the root cause, but this is causing issues on all of our Ubuntu 16.04 systems (0 byte log files). A restart of rsyslog appears to be sufficient to work around the issue (current content is added to the log files).

[atc0005]

Relevant bits from the /etc/init.d/rsyslog script:

#
# Author: Michael Biebl 
#

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="enhanced syslogd"
NAME=rsyslog

RSYSLOGD=rsyslogd
RSYSLOGD_BIN=/usr/sbin/rsyslogd
RSYSLOGD_OPTIONS="-c5"
RSYSLOGD_PIDFILE=/var/run/rsyslogd.pid

SCRIPTNAME=/etc/init.d/$NAME

# Exit if the package is not installed
[ -x "$RSYSLOGD_BIN" ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Define LSB log_* functions.
. /lib/lsb/init-functions

#
# Tell rsyslogd to close all open files
#
do_rotate() {
        NAME="$RSYSLOGD"
        PIDFILE="$RSYSLOGD_PIDFILE"

        start-stop-daemon --stop --signal HUP --quiet --pidfile $PIDFILE --name $NAME
}

  rotate)
        log_daemon_msg "Closing open files" "$RSYSLOGD"
        do_rotate
        log_end_msg $?
        ;;

Looks like the pid file is not being created? Is that the default now with rsyslog 8.33.0?

root@ubuntu-1604-virtual-machine:~# echo $PIDFILE

/var/run/rsyslogd.pid

root@ubuntu-1604-virtual-machine:~# ls -l $PIDFILE

ls: cannot access '/var/run/rsyslogd.pid': No such file or directory

[atc0005]

Possibly related to rsyslog/rsyslog#2143 ?

[atc0005]

@mbiebl

At least for the Debian package, that is not true

What Debian repo are you pushing the latest rsyslog package versions into? I wanted to look at the files to see if the changes would be trivial to introduce as a PR against the repo here for Ubuntu 16.04+ packages (because my skills in this area are limited).

I found the init.d script in a gitlab repo here:

https://salsa.debian.org/debian/rsyslog/blob/master/debian/rsyslog.init

Is that still used in the Debian package?

[mbiebl]

it is

[mbiebl]

although, it's unclear to me why you are interested in the init script?

[atc0005]

@mbiebl I asked because I noticed that the logrotate config here appears to make a call to the script (specifically the do_rotate function) in order to handle log rotation.

rsyslog-pkg-ubuntu/rsyslog/xenial/v8-stable/debian/rsyslog.init

Line 68 in b5f29c8

do_rotate() {

Complete function:

do_rotate() {
	NAME="$RSYSLOGD"
	PIDFILE="$RSYSLOGD_PIDFILE"

	start-stop-daemon --stop --signal HUP --quiet --pidfile $PIDFILE --name $NAME
}

I suppose the fix is to just update the /etc/logrotate.d/rsyslog conf file to use a different command to send a SIGHUP to rsyslog, but I was hoping to make a change to the /etc/init.d/rsyslog script that would allow any other custom logrotate conf files to still work with the old invoke-rc.d rsyslog rotate >/dev/null call.

[atc0005]

Workaround for anyone landing here affected by the change in the current 8.33.0 package:

1. sudo mkdir -p /etc/systemd/system/rsyslog.service.d
2. sudo nano /etc/systemd/system/rsyslog.service.d/10-enable-creation-of-pid-file.conf
3. sudo systemctl daemon-reload
4. sudo systemctl restart rsyslog

Here is the content to put into the /etc/systemd/system/rsyslog.service.d/10-enable-creation-of-pid-file.conf file:

# /etc/systemd/system/rsyslog.service.d/10-enable-creation-of-pid-file.conf
[Service]

# Upstream package choice:
#ExecStart=/usr/sbin/rsyslogd -n -iNONE

# Our override (options prior to 8.33.0 release):
ExecStart=
ExecStart=/usr/sbin/rsyslogd -n

Once the issue is fixed in the upstream packages then this drop-in would no longer be needed.

[Ackis]

Alternative to this would be to edit /etc/init.d/rsyslog:

do_rotate() {
        NAME="$RSYSLOGD"
        PIDFILE="$RSYSLOGD_PIDFILE"
        #start-stop-daemon --stop --signal HUP --quiet --pidfile $PIDFILE --name $NAME
        systemctl kill -s HUP --kill-who=main rsyslog.service
}

[mbiebl]

no, that would not be a good idea

[mbiebl]

this makes your sysv init script dependent on a systemd service file.

[mbiebl]

rather fix your logrotate config to do the right thing depending on whether it runs under sysvinit or systemd. Well, just copy what the Debian package did and you'll be fine

[hrak]

What Debian did is add a small script called rsyslog-rotate that gets dropped in /usr/lib/rsyslog. Then they change the postrotate command in the logrotate conf to /usr/lib/rsyslog/rsyslog-rotate

Can this please be addressed soon? Log rotation is basically broken on 16.04+ now.

[UiP9AV6Y]

What Debian did is add a small script called rsyslog-rotate that gets dropped in /usr/lib/rsyslog. Then they change the postrotate command in the logrotate conf to /usr/lib/rsyslog/rsyslog-rotate

instead of adding another file, it would be more practical to use reload instead of rotate from the logrotate rule.

@@ -7,7 +7,7 @@
        delaycompress
        compress
        postrotate
-               invoke-rc.d rsyslog rotate >/dev/null
+               invoke-rc.d rsyslog reload >/dev/null
        endscript
 }

@@ -33,6 +33,6 @@
        delaycompress
        sharedscripts
        postrotate
-               invoke-rc.d rsyslog rotate >/dev/null
+               invoke-rc.d rsyslog reload >/dev/null
        endscript
 }

the systemd unit files would require the addition of ExecReload=/bin/kill -HUP $MAINPID while the SysV scripts could be adapted to act similar for rotate and reload instructions

@@ -116,7 +116,7 @@ case "$1" in
        esac

        ;;
-  rotate)
+  rotate|reload)
        log_daemon_msg "Closing open files" "$RSYSLOGD"
        do_rotate
        log_end_msg $?
@@ -129,7 +129,7 @@ case "$1" in
        status_of_proc -p $RSYSLOGD_PIDFILE $RSYSLOGD_BIN $RSYSLOGD && exit 0 $
        ;;
   *)
-       echo "Usage: $SCRIPTNAME {start|stop|rotate|restart|force-reload|statu$
+       echo "Usage: $SCRIPTNAME {start|stop|rotate|reload|restart|force-reloa$
        exit 3
        ;;
 esac

[mbiebl]

No, reload has a very specific meaning: It is supposed to trigger a reload of the configuration
Sending SIGHUP to rsyslog does not do that. Use re-using reload for that would be confusing.

[rgerhards]

@atc0005 Is this fixed as of your PoV? Can we close?

[atc0005]

@atc0005 Is this fixed as of your PoV? Can we close?

Yes, thanks for checking back!