[gentoo-dev] Inotify and (f)crontabs

["Ryan Reich" <ryan.reich@gmail.com>]

My apologies for triple-posting this.  I can't tell which list would
be most appropriate, since it is a user, development, and performance
issue (albeit a minor performance issue).

This is a small essay on Gentoo's setup for fcron.

My issue:
I just installed fcron and I have to say, I'm a little disappointed
with the kludgy mechanism for implmenting:
1. easy configuration, meaning I don't have to run fcrontab personally
2. /etc/cron.{hourly,daily,weekly,monthly}
These are implmented by some very silly-looking polling tricks which,
even in principle, will necessarily waste 83% (that's 5/6) of their
efforts and clutter the logs with useless and uninformative messages.

The facts:
1. is implmented by putting the following rule in /etc/fcrontab:
    @mail(false),nolog(true) 10 /usr/sbin/check_system_crontabs -s 0
whose effect is to run a script, every ten minutes, to check whether
I've changed any of the various configuration files /etc/{,f}crontab,
/etc/cron.d/* and then add them all to the system crontab.

2. is implmented by putting the following rules in /etc/crontab:
    0  *  * * *      rm -f /var/spool/cron/lastrun/cron.hourly
    1  3  * * *      rm -f /var/spool/cron/lastrun/cron.daily
    15 4  * * 6      rm -f /var/spool/cron/lastrun/cron.weekly
    30 5  1 * *      rm -f /var/spool/cron/lastrun/cron.monthly
    */10  *  * * *      /usr/bin/test -x /usr/sbin/run-crons &&
/usr/sbin/run-crons
whose effect is, at intevals of one hour, day, week, and month, to
remove some state files for the script run-crons, and also to run said
script every ten minutes.  The purpose of run-crons is to run the
scripts in /etc/cron.{hourly,...} at the appropriate intervals, thus
saving me the effort of adding a lot of lines looking like
    1 3 * * * * some-command
to my crontab.

Criticism:
Both of these ideas are, in principle and in practice, inefficient and
irritating.
1.  Since I will only ever update my crontabs rarely, checking every
ten minutes is likely to yield a negative result virtually 100% of the
time; however, since one wants the system to respond quickly when a
change is made, it is necessary to check frequently.  This would seem
to be a contradiction.

2.  The problem is even worse: though it is conceivable that a real
cron demon would be updating his crontabs willy-nilly, and thus that 1
is not actually inefficient for him, it is not possible, even in
principle, for 2 to be worthwhile.  run-crons runs commands at
intervals of no less than one hour, but is itself run every ten
minutes.  Furthermore, the files /var/spool/cron/lastrun/cron.* are
ALREADY handled in the run-crons script itself, so that most of the
above commands would seem to be redundant.  This one also has the
additional unpleasant property of filling the logs with useless
messages:
    [fcron] Job /usr/bin/test -x /usr/sbin/run-crons &&
/usr/sbin/run-crons started for user
    systab
and NOT actually logging the commands which are run (since they are
run by run-crons, which doesn't log)!  It seems that the reason for
this is that the option nolog(true) is invalid syntax for cron daemons
other than fcron, and /etc/crontab is not fcron-specific so the option
can't be passed there.  The reason that run-crons doesn't log is
unclear to me.

My solution:
It seems odd to me that Gentoo isn't following the advice in its own
scripts, since check_system_crontabs has a big comment on top advising
the user to run it with dnotify.  This would be a lovely solution,
since dnotify asks the kernel to inform it of directory changes and
thus, will only run the script when it needs to be run.  The only
problem is that you have to monitor the whole directory, and as the
comment notes, that will generate a lot of problems with /etc.

Dnotify, however, is obsoleted by inotify, which allows you to monitor
_individual_file_changes_.  Inotify is already in the kernel, so there
are no API changes to be made, just configuration changes to employ
inotify.  Furthermore, there exists a nice cron-like utility, based on
inotify, which performs exactly the tasks we want with fcron.  Its
name is incron and its page is here: http://inotify.aiken.cz.  I
hardly even need to explain how it would fit in, but here it is.
Rather than having /etc/fcrontab, one would simply have the following
in /etc/icron.d/fcron (say):
    /etc/crontab    IN_CLOSE_WRITE    /usr/sbin/check_system_crontabs -s 0
    /etc/fcrontab    IN_CLOSE_WRITE    /usr/sbin/check_system_crontabs -s 0
    /etc/cron.d    IN_CLOSE_WRITE,IN_CREATE
/usr/sbin/check_system_crontabs -s 0
Rather than having /etc/crontab, one would modify
check_system_crontabs to add lines like
    1 3 * * * * some-command
to the system crontab whenever /etc/cron.daily/some-command is
modified, and then add four more lines to /etc/icron.d/fcron:
    /etc/cron.hourly    IN_CLOSE_WRITE,IN_CREATE
/usr/sbin/check_system_crontabs -s 0
    /etc/cron.daily ....
The above problems are then all solved: the system crontab is updated
exactly when changes are made, wasteful polling is avoided, useless
syslogs are elided, and useful ones replace them.

I'm wondering now whether I should copy this letter to
fcrondev@ml.free.fr and ask them if they would like to include inotify
support in fcron itself, which would make configuration somewhat
easier and more flexible (also make incron unnecessary).  In any case,
I think the changes to Gentoo would be extremely easy to do and
improve life a little.

Thanks,
Ryan Reich
-- 
gentoo-dev@gentoo.org mailing list