[gentoo-dev] Inotify and (f)crontabs

[gentoo-dev] Inotify and (f)crontabs

[Ryan Reich]

My apologies for triple-posting this.  I can't tell which list would
be most appropriate, since it is a user, development, and performance
issue (albeit a minor performance issue).

This is a small essay on Gentoo's setup for fcron.

My issue:
I just installed fcron and I have to say, I'm a little disappointed
with the kludgy mechanism for implmenting:
1. easy configuration, meaning I don't have to run fcrontab personally
2. /etc/cron.{hourly,daily,weekly,monthly}
These are implmented by some very silly-looking polling tricks which,
even in principle, will necessarily waste 83% (that's 5/6) of their
efforts and clutter the logs with useless and uninformative messages.

The facts:
1. is implmented by putting the following rule in /etc/fcrontab:
    @mail(false),nolog(true) 10 /usr/sbin/check_system_crontabs -s 0
whose effect is to run a script, every ten minutes, to check whether
I've changed any of the various configuration files /etc/{,f}crontab,
/etc/cron.d/* and then add them all to the system crontab.

2. is implmented by putting the following rules in /etc/crontab:
    0  *  * * *      rm -f /var/spool/cron/lastrun/cron.hourly
    1  3  * * *      rm -f /var/spool/cron/lastrun/cron.daily
    15 4  * * 6      rm -f /var/spool/cron/lastrun/cron.weekly
    30 5  1 * *      rm -f /var/spool/cron/lastrun/cron.monthly
    */10  *  * * *      /usr/bin/test -x /usr/sbin/run-crons &&
/usr/sbin/run-crons
whose effect is, at intevals of one hour, day, week, and month, to
remove some state files for the script run-crons, and also to run said
script every ten minutes.  The purpose of run-crons is to run the
scripts in /etc/cron.{hourly,...} at the appropriate intervals, thus
saving me the effort of adding a lot of lines looking like
    1 3 * * * * some-command
to my crontab.

Criticism:
Both of these ideas are, in principle and in practice, inefficient and
irritating.
1.  Since I will only ever update my crontabs rarely, checking every
ten minutes is likely to yield a negative result virtually 100% of the
time; however, since one wants the system to respond quickly when a
change is made, it is necessary to check frequently.  This would seem
to be a contradiction.

2.  The problem is even worse: though it is conceivable that a real
cron demon would be updating his crontabs willy-nilly, and thus that 1
is not actually inefficient for him, it is not possible, even in
principle, for 2 to be worthwhile.  run-crons runs commands at
intervals of no less than one hour, but is itself run every ten
minutes.  Furthermore, the files /var/spool/cron/lastrun/cron.* are
ALREADY handled in the run-crons script itself, so that most of the
above commands would seem to be redundant.  This one also has the
additional unpleasant property of filling the logs with useless
messages:
    [fcron] Job /usr/bin/test -x /usr/sbin/run-crons &&
/usr/sbin/run-crons started for user
    systab
and NOT actually logging the commands which are run (since they are
run by run-crons, which doesn't log)!  It seems that the reason for
this is that the option nolog(true) is invalid syntax for cron daemons
other than fcron, and /etc/crontab is not fcron-specific so the option
can't be passed there.  The reason that run-crons doesn't log is
unclear to me.

My solution:
It seems odd to me that Gentoo isn't following the advice in its own
scripts, since check_system_crontabs has a big comment on top advising
the user to run it with dnotify.  This would be a lovely solution,
since dnotify asks the kernel to inform it of directory changes and
thus, will only run the script when it needs to be run.  The only
problem is that you have to monitor the whole directory, and as the
comment notes, that will generate a lot of problems with /etc.

Dnotify, however, is obsoleted by inotify, which allows you to monitor
_individual_file_changes_.  Inotify is already in the kernel, so there
are no API changes to be made, just configuration changes to employ
inotify.  Furthermore, there exists a nice cron-like utility, based on
inotify, which performs exactly the tasks we want with fcron.  Its
name is incron and its page is here: http://inotify.aiken.cz.  I
hardly even need to explain how it would fit in, but here it is.
Rather than having /etc/fcrontab, one would simply have the following
in /etc/icron.d/fcron (say):
    /etc/crontab    IN_CLOSE_WRITE    /usr/sbin/check_system_crontabs -s 0
    /etc/fcrontab    IN_CLOSE_WRITE    /usr/sbin/check_system_crontabs -s 0
    /etc/cron.d    IN_CLOSE_WRITE,IN_CREATE
/usr/sbin/check_system_crontabs -s 0
Rather than having /etc/crontab, one would modify
check_system_crontabs to add lines like
    1 3 * * * * some-command
to the system crontab whenever /etc/cron.daily/some-command is
modified, and then add four more lines to /etc/icron.d/fcron:
    /etc/cron.hourly    IN_CLOSE_WRITE,IN_CREATE
/usr/sbin/check_system_crontabs -s 0
    /etc/cron.daily ....
The above problems are then all solved: the system crontab is updated
exactly when changes are made, wasteful polling is avoided, useless
syslogs are elided, and useful ones replace them.

I'm wondering now whether I should copy this letter to
fcrondev@ml.free.fr and ask them if they would like to include inotify
support in fcron itself, which would make configuration somewhat
easier and more flexible (also make incron unnecessary).  In any case,
I think the changes to Gentoo would be extremely easy to do and
improve life a little.

Thanks,
Ryan Reich
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Inotify and (f)crontabs

[Daniel Schömer]

Hi!

Ryan Reich wrote:
> [...]
> My issue: I just installed fcron and I have to say, I'm
> a little disappointed with the kludgy mechanism for
> implmenting:
>
> 1. easy configuration, meaning I don't have to run fcrontab
>    personally
>
> 2. /etc/cron.{hourly,daily,weekly,monthly} These are implmented
>    by some very silly-looking polling tricks which, even in
>    principle, will necessarily waste 83% (that's 5/6) of their
>    efforts and clutter the logs with useless and uninformative
>    messages.
> [...]

I just want to share my system-wide fcrontab:

  $ sudo fcrontab -l systab
  PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

  !nice(15)
  !noticenotrun(false)
  !serial(true)

  %hourly  0-30  run-parts --report /etc/cron.hourly
  %daily   * *   run-parts --report /etc/cron.daily
  %weekly  * *   run-parts --report /etc/cron.weekly
  %monthly * * * run-parts --report /etc/cron.monthly

I use fcron's capabilities to execute the files in
/etc/cron.{hourly,daily,weekly,monthly}/ once within each period.

Daniel Schömer

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Inotify and (f)crontabs

[Ryan Reich]

On 7/1/07, Daniel Schömer <daniel.schoemer@gmx.net> wrote:
> Hi!
>
> Ryan Reich wrote:
> > [...]
> > My issue: I just installed fcron and I have to say, I'm
> > a little disappointed with the kludgy mechanism for
> > implmenting:
> >
> > 1. easy configuration, meaning I don't have to run fcrontab
> >    personally
> >
> > 2. /etc/cron.{hourly,daily,weekly,monthly} These are implmented
> >    by some very silly-looking polling tricks which, even in
> >    principle, will necessarily waste 83% (that's 5/6) of their
> >    efforts and clutter the logs with useless and uninformative
> >    messages.
> > [...]
>
> I just want to share my system-wide fcrontab:
>
>   $ sudo fcrontab -l systab
>   PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
>
>   !nice(15)
>   !noticenotrun(false)
>   !serial(true)
>
>   %hourly  0-30  run-parts --report /etc/cron.hourly
>   %daily   * *   run-parts --report /etc/cron.daily
>   %weekly  * *   run-parts --report /etc/cron.weekly
>   %monthly * * * run-parts --report /etc/cron.monthly
>
> I use fcron's capabilities to execute the files in
> /etc/cron.{hourly,daily,weekly,monthly}/ once within each period.

Thanks for sharing this.  I'd never seen this run-parts utility
before, though I see at least that it's a Debian contribution rather
than a standard utility (thus I avert shame).  This is a more elegant
solution than the run-crons method, though it doesn't have quite the
same effect as instructing cron to actually run each command
separately (their outputs will be mailed in conglomerate, rather than
one mail per periodic program.  This is, according to your
preferences, either a bug or a feature.  Less ambiguously a bug,
though, is that the syslog will only record running run-parts and not
which parts were run).

Have you any thoughts on what to do with check_system_crontabs?  Its
schedule is inherently reactive and not periodic, so really,
administering it with (f)cron at all is a logical error.  That's why I
suggested inotify in the first place.

-- 
Ryan Reich
--
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Re: Inotify and (f)crontabs

[Daniel Schömer]

Hi!

Ryan Reich wrote:
>  On 7/1/07, Daniel Schömer <daniel.schoemer@gmx.net> wrote:
>> $ sudo fcrontab -l systab
>> [...]
>> %hourly  0-30  run-parts --report /etc/cron.hourly
>> %daily   * *   run-parts --report /etc/cron.daily
>> [...]
> Thanks for sharing this.  [...] This is a more elegant solution
> than the run-crons method, though it doesn't have quite the
> same effect as instructing cron to actually run each command
> separately (their outputs will be mailed in conglomerate,
> rather than one mail per periodic program.  This is, according
> to your preferences, either a bug or a feature.

I have never really thought about the combined mails.  I'm using
this on my desktop with only a few scripts in
/etc/cron.{h,d,w,m}*/.  Thinking about it now, I would prefer one
mail per script.  At least when there are more than just a few
scripts run by cron.

I can't see an elegant way implementing this directly in fcron.
A modified version of run-parts that's capable of mailing the
output of each script (or all in conglomerate if specified) to
the runnig user (or a specified account) may do.

> Less ambiguously a bug, though, is that the syslog will only
> record running run-parts and not which parts were run).

Would it be sufficient to let this also be done by run-parts?
Then there would be records of fcron starting run-parts and
records of run-parts starting each script.

> Have you any thoughts on what to do with check_system_crontabs?
> Its schedule is inherently reactive and not periodic, so
> really, administering it with (f)cron at all is a logical
> error.  That's why I suggested inotify in the first place.

I must say, I can't remember that I've seen check_system_crontabs
on my desktop; maybe I just suppressed it :-).  Now that I think
of it, I'm remembering an elog message from the fcron ebuild
telling me to use the fcrontab systab for system-wide jobs
instead of /etc/{f,}crontab.

Using inotify sounds more logical for me than using (f)cron for
this.

Daniel Schömer

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Re: Inotify and (f)crontabs

[Ryan Reich]

On 7/1/07, Daniel Schömer <daniel.schoemer@gmx.net> wrote:
> Hi!
>
> Ryan Reich wrote:
> >  On 7/1/07, Daniel Schömer <daniel.schoemer@gmx.net> wrote:
> >> $ sudo fcrontab -l systab
> >> [...]
> >> %hourly  0-30  run-parts --report /etc/cron.hourly
> >> %daily   * *   run-parts --report /etc/cron.daily
> >> [...]

> > Less ambiguously a bug, though, is that the syslog will only
> > record running run-parts and not which parts were run).
>
> Would it be sufficient to let this also be done by run-parts?
> Then there would be records of fcron starting run-parts and
> records of run-parts starting each script.

That works of course, though if you're at the point of modifying
run-parts you might as well modify the crontabs setup.  I mean, having
run-parts do the logging is silly since fcron already has a logging
capability, and since the only point of using run-parts is to assist
fcron.  Otherwise you're just creating further unnecessary complexity.

> > Have you any thoughts on what to do with check_system_crontabs?
> > Its schedule is inherently reactive and not periodic, so
> > really, administering it with (f)cron at all is a logical
> > error.  That's why I suggested inotify in the first place.
>
> I must say, I can't remember that I've seen check_system_crontabs
> on my desktop; maybe I just suppressed it :-).  Now that I think
> of it, I'm remembering an elog message from the fcron ebuild
> telling me to use the fcrontab systab for system-wide jobs
> instead of /etc/{f,}crontab.
>
> Using inotify sounds more logical for me than using (f)cron for
> this.

If there's interest in incorporating this, I wouldn't mind testing my
idea.  Once I get past the initial resistance to doing anything at
all, it's probably two minutes' compilation time plus two more writing
the config files to set up.

-- 
Ryan Reich
--
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Inotify and (f)crontabs

[Daniel Schömer]

Ryan Reich wrote:
>  On 7/1/07, Daniel Schömer <daniel.schoemer@gmx.net> wrote:
>> Ryan Reich wrote:
>>>  On 7/1/07, Daniel Schömer <daniel.schoemer@gmx.net> wrote:
>>>> [...]
>>>> %daily   * *   run-parts --report /etc/cron.daily
>>>> [...]
>>> Less ambiguously a bug, though, is that the syslog will only
>>> record running run-parts and not which parts were run).
>> Would it be sufficient to let this also be done by run-parts?
>> [...]
> That works of course, though if you're at the point of
> modifying run-parts you might as well modify the crontabs
> setup.

Your're right.  Since I have nearly zero practice creating
C-code, it would just be easier for me to create something
similiar to run-parts in shell or ruby.  That shell or ruby
run-parts-thingy would be easier for me to modify than fcron.

> [...]
>>> [...]
>> [...]
> If there's interest in incorporating this, I wouldn't mind
> testing my idea.  [...]

I'm currently happy with run-parts.  But please don't let me stop
you trying your idea.  I will definitely give it a try.

Daniel Schömer

-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Re: Re: Inotify and (f)crontabs

[Steve Long]

Ryan Reich wrote:
<snip useful discussion that went over my head>
> 
> If there's interest in incorporating this, I wouldn't mind testing my
> idea.  Once I get past the initial resistance to doing anything at
> all, it's probably two minutes' compilation time plus two more writing
> the config files to set up.
> 
I think I should point out that comments like that really get people's backs
up; if it's so quick and easy why didn't you implement it first and then
ask for comments on a working project? If you only wanted discussion then
cool, you got it ;)

Please do post a follow up with a link to the working code.

(BTW my 5 minute projects always take weeks.. ;)


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Inotify and (f)crontabs

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 3304 bytes --]

On Sunday 01 July 2007, Ryan Reich wrote:
> This is a small essay on Gentoo's setup for fcron.

which is troublesome because some of the things here are specific to fcron 
(which frankly dont interest me) while others are specific to the cronbase 
package which installs `run-crons` (which does interest me as it is a 
Gentooism) ... i'll try to pick out only the relevant pieces as you said 
yourself, the fcron things should go upstream.

> 2. is implmented by putting the following rules in /etc/crontab:
>     0  *  * * *      rm -f /var/spool/cron/lastrun/cron.hourly
>     1  3  * * *      rm -f /var/spool/cron/lastrun/cron.daily
>     15 4  * * 6      rm -f /var/spool/cron/lastrun/cron.weekly
>     30 5  1 * *      rm -f /var/spool/cron/lastrun/cron.monthly
>     */10  *  * * *      /usr/bin/test -x /usr/sbin/run-crons &&
> /usr/sbin/run-crons
> whose effect is, at intevals of one hour, day, week, and month, to
> remove some state files for the script run-crons, and also to run said
> script every ten minutes.  The purpose of run-crons is to run the
> scripts in /etc/cron.{hourly,...} at the appropriate intervals, thus
> saving me the effort of adding a lot of lines looking like
>     1 3 * * * * some-command
> to my crontab.

you missed a critical aspect: offline time.  the way run-crons is implemented, 
if you happen to routinely shut your machine off at the time that the cronjob 
is supposed to fire, then the standard you proposed will pretty much never 
fire.  the run-crons implementation however has a pretty good guarantee that 
the periodic crons will get fired at the next uptime opportunity.

> Criticism:
> Both of these ideas are, in principle and in practice, inefficient and
> irritating.
> 1.  Since I will only ever update my crontabs rarely, checking every
> ten minutes is likely to yield a negative result virtually 100% of the
> time; however, since one wants the system to respond quickly when a
> change is made, it is necessary to check frequently.  This would seem
> to be a contradiction.
>
> 2.  The problem is even worse: though it is conceivable that a real
> cron demon would be updating his crontabs willy-nilly, and thus that 1
> is not actually inefficient for him, it is not possible, even in
> principle, for 2 to be worthwhile.  run-crons runs commands at
> intervals of no less than one hour, but is itself run every ten
> minutes. 

these statements are incorrect as they are based on incorrect assumptions.  
the point of the 10 minute cycle is not to see if anything of the cron dirs 
have changed, but if any routine cronjob needs to fire (refer to my comment 
above).

> Furthermore, the files /var/spool/cron/lastrun/cron.* are 
> ALREADY handled in the run-crons script itself, so that most of the
> above commands would seem to be redundant.

this is most likely true.

> This one also has the 
> additional unpleasant property of filling the logs with useless
> messages:
>     [fcron] Job /usr/bin/test -x /usr/sbin/run-crons &&
> /usr/sbin/run-crons started for user
>     systab

yes, this sucks, but so it goes.

> The reason that run-crons doesn't log is unclear to me.

there probably isnt one; file a bug with a patch if you like or just file a 
bug.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] Re: Re: Re: Inotify and (f)crontabs

[Ryan Reich]

[-- Attachment #1: Type: text/plain, Size: 2305 bytes --]

On 7/6/07, Steve Long <slong@rathaus.eclipse.co.uk> wrote:
> Ryan Reich wrote:
> <snip useful discussion that went over my head>
> >
> > If there's interest in incorporating this, I wouldn't mind testing my
> > idea.  Once I get past the initial resistance to doing anything at
> > all, it's probably two minutes' compilation time plus two more writing
> > the config files to set up.
> >
> I think I should point out that comments like that really get people's backs
> up; if it's so quick and easy why didn't you implement it first and then
> ask for comments on a working project? If you only wanted discussion then
> cool, you got it ;)
>
> Please do post a follow up with a link to the working code.
>
> (BTW my 5 minute projects always take weeks.. ;)

I don't know if I got anyone else's back up with this, but you got
mine up with your letter :)  So I wrote an ebuild for incron (though I
notice that there's an extremely minimal one pending on
bugs.gentoo.org).  It includes, in addition to the actual ebuild file,
a boot script, an incrontab similar to the one I conceptualized in one
of my earlier letters, and a little script that handles files in
/etc/cron.{hourly,...}.  I've tested it on my machine (indeed, I'm
running it now) and it does work.  Basically, the incrontab (called
"fcrontabs" and which goes in /etc/incron.d) runs my little script to
generate a one-line fcrontab which it puts in /etc/cron, whenever it
sees something change in one of the cron.timely directories; the
one-liner just runs the script which changed.  It also makes
check_system_crontabs the responsibility of incrond.  The net result
is that adding anything to any /etc/cron.timely/ results in a periodic
fcron job, like

@ 1d makewhatis -u

being added to the fcrontab for "systab" immediately.  The contents of
/etc/crontab and /etc/fcrontab are redundant under this scheme and the
files should be emptied if this is used.

I've tarred and bzipped the whole ebuild directory and attached it;
it's only four kilobytes so I hope no one minds (this letter is
probably longer).  I don't know where the correct forum to submit this
sort of thing for comment is, so if this isn't it I'd be happy to take
it somewhere else.  But I thought anyone following this discussion
would like to see it here.

-- 
Ryan Reich

[-- Attachment #2: incron-ebuild.tar.bz2 --]
[-- Type: application/x-bzip2, Size: 3131 bytes --]

[gentoo-dev] Inotify and (f)crontabs

[Ryan Reich]

On 7/7/07, Mike Frysinger <vapier@gentoo.org> wrote:
> On Sunday 01 July 2007, Ryan Reich wrote:
> > This is a small essay on Gentoo's setup for fcron.
>
> which is troublesome because some of the things here are specific to fcron
> (which frankly dont interest me) while others are specific to the cronbase
> package which installs `run-crons` (which does interest me as it is a
> Gentooism) ... i'll try to pick out only the relevant pieces as you said
> yourself, the fcron things should go upstream.

I'm sorry you don't care about my fcron criticisms, but that's the
cron I use so it's the one I picked on.  The problems with
check_system_crontabs were only half the point, anyway.

> > 2. is implmented by putting the following rules in /etc/crontab:
> >     0  *  * * *      rm -f /var/spool/cron/lastrun/cron.hourly
> >     1  3  * * *      rm -f /var/spool/cron/lastrun/cron.daily
> >     15 4  * * 6      rm -f /var/spool/cron/lastrun/cron.weekly
> >     30 5  1 * *      rm -f /var/spool/cron/lastrun/cron.monthly
> >     */10  *  * * *      /usr/bin/test -x /usr/sbin/run-crons &&
> > /usr/sbin/run-crons
> > whose effect is, at intevals of one hour, day, week, and month, to
> > remove some state files for the script run-crons, and also to run said
> > script every ten minutes.  The purpose of run-crons is to run the
> > scripts in /etc/cron.{hourly,...} at the appropriate intervals, thus
> > saving me the effort of adding a lot of lines looking like
> >     1 3 * * * * some-command
> > to my crontab.
>
> you missed a critical aspect: offline time.  the way run-crons is implemented,
> if you happen to routinely shut your machine off at the time that the cronjob
> is supposed to fire, then the standard you proposed will pretty much never
> fire.  the run-crons implementation however has a pretty good guarantee that
> the periodic crons will get fired at the next uptime opportunity.

This point is where your uninterest in fcron puts us at odds.  See,
fcron allows you to have such commands run at boot, and allows you to
schedule commands that run at periods of uptime rather than wall time,
so this is not an issue for fcron.  And for other crons, there exists
anacron, which is recommended precisely for this purpose by Gentoo.
The only thing that's accomplished by putting this functionality in
run-crons is to duplicate it, awkwardly.

> > Furthermore, the files /var/spool/cron/lastrun/cron.* are
> > ALREADY handled in the run-crons script itself, so that most of the
> > above commands would seem to be redundant.
>
> this is most likely true.
>
> > This one also has the
> > additional unpleasant property of filling the logs with useless
> > messages:
> >     [fcron] Job /usr/bin/test -x /usr/sbin/run-crons &&
> > /usr/sbin/run-crons started for user
> >     systab
>
> yes, this sucks, but so it goes.

So if it sucks, you would approve of an alternative that doesn't
exhibit this behavior?

--
Ryan Reich
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Inotify and (f)crontabs

[Peter Gordon]

[-- Attachment #1: Type: text/plain, Size: 714 bytes --]

On Sat, 2007-07-07 at 04:23 -0400, Mike Frysinger wrote:
> you missed a critical aspect: offline time.  the way run-crons is implemented, 
> if you happen to routinely shut your machine off at the time that the cronjob 
> is supposed to fire, then the standard you proposed will pretty much never 
> fire.  the run-crons implementation however has a pretty good guarantee that 
> the periodic crons will get fired at the next uptime opportunity.

Isn't this perfectly what anacron is intended for?
-- 
Peter Gordon (codergeek42)
Gentoo Forums Global Moderator
GnuPG Public Key ID: 0xFFC19479 / Fingerprint:
  DD68 A414 56BD 6368 D957 9666 4268 CB7A FFC1 9479
My Blog: http://thecodergeek.com/blog/


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Inotify and (f)crontabs

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 772 bytes --]

On Saturday 07 July 2007, Peter Gordon wrote:
> On Sat, 2007-07-07 at 04:23 -0400, Mike Frysinger wrote:
> > you missed a critical aspect: offline time.  the way run-crons is
> > implemented, if you happen to routinely shut your machine off at the time
> > that the cronjob is supposed to fire, then the standard you proposed will
> > pretty much never fire.  the run-crons implementation however has a
> > pretty good guarantee that the periodic crons will get fired at the next
> > uptime opportunity.
>
> Isn't this perfectly what anacron is intended for?

yes and no ... anacron is designed with this issue in mind, but as the 
homepage of anacron explains, it does not replace the normal cron 
functionality and as such cannot be used on its own
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] Inotify and (f)crontabs

[Ryan Reich]

On 7/7/07, Mike Frysinger <vapier@gentoo.org> wrote:
> On Saturday 07 July 2007, Peter Gordon wrote:
> > On Sat, 2007-07-07 at 04:23 -0400, Mike Frysinger wrote:
> > > you missed a critical aspect: offline time.  the way run-crons is
> > > implemented, if you happen to routinely shut your machine off at the time
> > > that the cronjob is supposed to fire, then the standard you proposed will
> > > pretty much never fire.  the run-crons implementation however has a
> > > pretty good guarantee that the periodic crons will get fired at the next
> > > uptime opportunity.
> >
> > Isn't this perfectly what anacron is intended for?
>
> yes and no ... anacron is designed with this issue in mind, but as the
> homepage of anacron explains, it does not replace the normal cron
> functionality and as such cannot be used on its own

I have to disagree in this particular case.  The anacron homepage,
anacron.sourceforge.net, gives this exact situation as its primary
example of what anacron is intended for.  Sure, it's not good for
handling more complex scheduling, but it seems to do what run-crons
tries to do: run jobs that should have been executed while the
computer was off, as soon as it comes back on.  Am I missing something
subtle?

-- 
Ryan Reich
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Inotify and (f)crontabs

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1695 bytes --]

On Sunday 08 July 2007, Ryan Reich wrote:
> On 7/7/07, Mike Frysinger <vapier@gentoo.org> wrote:
> > On Saturday 07 July 2007, Peter Gordon wrote:
> > > On Sat, 2007-07-07 at 04:23 -0400, Mike Frysinger wrote:
> > > > you missed a critical aspect: offline time.  the way run-crons is
> > > > implemented, if you happen to routinely shut your machine off at the
> > > > time that the cronjob is supposed to fire, then the standard you
> > > > proposed will pretty much never fire.  the run-crons implementation
> > > > however has a pretty good guarantee that the periodic crons will get
> > > > fired at the next uptime opportunity.
> > >
> > > Isn't this perfectly what anacron is intended for?
> >
> > yes and no ... anacron is designed with this issue in mind, but as the
> > homepage of anacron explains, it does not replace the normal cron
> > functionality and as such cannot be used on its own
>
> I have to disagree in this particular case.  The anacron homepage,
> anacron.sourceforge.net, gives this exact situation as its primary
> example of what anacron is intended for.  Sure, it's not good for
> handling more complex scheduling, but it seems to do what run-crons
> tries to do: run jobs that should have been executed while the
> computer was off, as soon as it comes back on.  Am I missing something
> subtle?

run-crons transparently gives all crons this behavior with very little 
overhead rather than making every user set up a dual system: a standard cron 
and anacron.

run-crons is a default helper for crons that just works.  if you want to not 
use it but opt for anacron instead, nothing is stopping you from doing 
exactly that.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

[gentoo-dev] Re: Inotify and (f)crontabs

[Steve Long]

Mike Frysinger wrote:
>> I have to disagree in this particular case.  The anacron homepage,
>> anacron.sourceforge.net, gives this exact situation as its primary
>> example of what anacron is intended for.  Sure, it's not good for
>> handling more complex scheduling, but it seems to do what run-crons
>> tries to do: run jobs that should have been executed while the
>> computer was off, as soon as it comes back on.  Am I missing something
>> subtle?
> 
> run-crons transparently gives all crons this behavior with very little
> overhead rather than making every user set up a dual system: a standard
> cron and anacron.
> 
> run-crons is a default helper for crons that just works.  if you want to
> not use it but opt for anacron instead, nothing is stopping you from doing
> exactly that.

I think Mr Frysinger is grudgingly conceding the point, so can we have some
stats eg on CPU time saved blah blah blah? But it'd be really sweet if you
could post em on the forums, as the technical discussion seems over for
now. (At least to this friendly-coder ;-))

ie: market it to the user base please, not the devs ;)

Please be sure that this works from a clean install and test it on a live
box as the only system-- for a period of at least a week, as you collect
sample data. A write up of how to make it work would be ideal for
Documentation, Tips & Tricks imo.

"2 of 5 - recall to pub" *bzzt*.. click.


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Inotify and (f)crontabs

[Ryan Reich]

On 7/8/07, Steve Long <slong@rathaus.eclipse.co.uk> wrote:
> Mike Frysinger wrote:
> > run-crons is a default helper for crons that just works.  if you want to
> > not use it but opt for anacron instead, nothing is stopping you from doing
> > exactly that.
>
> I think Mr Frysinger is grudgingly conceding the point, so can we have some
> stats eg on CPU time saved blah blah blah? But it'd be really sweet if you
> could post em on the forums, as the technical discussion seems over for
> now. (At least to this friendly-coder ;-))
>
> ie: market it to the user base please, not the devs ;)
>
> Please be sure that this works from a clean install and test it on a live
> box as the only system-- for a period of at least a week, as you collect
> sample data. A write up of how to make it work would be ideal for
> Documentation, Tips & Tricks imo.
>
> "2 of 5 - recall to pub" *bzzt*.. click.

Well, as you can tell from the fact that I use fcron, this point is of
academic interest to me.  It's also secondary to my main concern in
this thread, which is getting Gentoo to use incron; right now I'm just
waiting for people to comment on the ebuild I posted yesterday.

In my opinion, this is really an issue for the developers, and indeed
I think Mike Frysinger agrees with that since he views the periodic
scripts (now handled by run-crons) to be something that should "just
work", i.e. be beneath the notice of the user.  Replacing it with an
anacron setup that "just works" should be equivalent from the user's
perspective.

After all, how much of Gentoo is carefully preconfigured to "just
work" out of the box?  Until I installed fcron, the file I saw most
often in /etc was make.conf.  It's one thing to have to configure cron
to do your daily chores; that's necessary, of course, since only you
can know what you want done (but note that Gentoo already includes
daily makewhatis and updatedb jobs, which are the two big ones).  It's
another to have anacron set up just to do the generalized task of
handling this; the user doesn't even need to know it's there.  Just
like they don't know that run-crons is there.

As for CPU savings: are you kidding?  Right now, run-crons is run
every ten minutes, and anacron would be run on boot and every 24 hours
thereafter.  The advantages are clear.  I don't think the users are
invested in the particular implementation at all; since run-crons is,
as Mr. Frysinger wrote in his original response to me, a "Gentooism",
that question is really one for the developers.

To be more pointed about it, it is not even my problem to justify
using anacron, since this is the canonical answer to the question
which Gentoo answers by using a home-grown script "run-crons".
Whoever implemented run-crons should justify reinventing the wheel and
explain how anacron's failings prevent it from working as intended
(and why, at the same time, Gentoo also recommends installing it, or
using fcron).  I'm just here to ask them why.

-- 
Ryan Reich
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Inotify and (f)crontabs

[Ryan Reich]

On 7/8/07, Mike Frysinger <vapier@gentoo.org> wrote:
> On Sunday 08 July 2007, Ryan Reich wrote:
> > I have to disagree in this particular case.  The anacron homepage,
> > anacron.sourceforge.net, gives this exact situation as its primary
> > example of what anacron is intended for.  Sure, it's not good for
> > handling more complex scheduling, but it seems to do what run-crons
> > tries to do: run jobs that should have been executed while the
> > computer was off, as soon as it comes back on.  Am I missing something
> > subtle?
>
> run-crons transparently gives all crons this behavior with very little
> overhead rather than making every user set up a dual system: a standard cron
> and anacron.
>
> run-crons is a default helper for crons that just works.  if you want to not
> use it but opt for anacron instead, nothing is stopping you from doing
> exactly that.

What is the additional overhead of using cron+anacron as compared to
using cron+run-crons?  The README in anacron's tarball indicates that
the net difference is one bootscript.  Otherwise, you (by which I mean
"the developers" as opposed to "the person using anacron") just take
most of the existing /etc/crontab and put it (or its anacron
equivalent) in /etc/anacrontab, and with the rest you have cron run
anacron once a night.  The user wouldn't have to do any more setup
than currently; it would just work.

-- 
Ryan Reich
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Re: Re: Inotify and (f)crontabs

[Ryan Reich]

On 7/7/07, Ryan Reich <ryan.reich@gmail.com> wrote:
> ...
> I've tarred and bzipped the whole ebuild directory and attached it;
> it's only four kilobytes so I hope no one minds (this letter is
> probably longer).  I don't know where the correct forum to submit this
> sort of thing for comment is, so if this isn't it I'd be happy to take
> it somewhere else.  But I thought anyone following this discussion
> would like to see it here.

There hasn't been a single post to this end of the thread since I
wrote this letter.  Is it just not that interesting an idea?

-- 
Ryan Reich
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Re: Re: Inotify and (f)crontabs

[Robert Buchholz]

On Monday, 9. July 2007 23:43, Ryan Reich wrote:
> On 7/7/07, Ryan Reich <ryan.reich@gmail.com> wrote:
> > ...
> > I've tarred and bzipped the whole ebuild directory and attached it;
> > it's only four kilobytes so I hope no one minds (this letter is
> > probably longer).  I don't know where the correct forum to submit
> > this sort of thing for comment is, so if this isn't it I'd be happy
> > to take it somewhere else.  But I thought anyone following this
> > discussion would like to see it here.
>
> There hasn't been a single post to this end of the thread since I
> wrote this letter.  Is it just not that interesting an idea?

Thanks for following up on the issue.
Please attach the ebuilds to the bug if one exists (as plain text, not 
compressed), or open a new one.

Regards,
Robert
-- 
gentoo-dev@gentoo.org mailing list