[gentoo-user] weird cron mail problem

[gentoo-user] weird cron mail problem

[Philip Webb]

Starting yesterday IIRC I've been getting one of the messages below
every time my Fetchmail cron job polls my  2  ISPs for new mail.
The content of the msg clearly comes from my own Fetchmail,
as it refers to both ISPs, but the msg itself originates at Uniserve,
as is shown by the list of routings in the full header (not shown).
Perhaps Uniserve has changed its procedures with this result,
but before I phone their tech folks, I'ld like to be sure
that it's not caused by something in my own machine.
I'm not aware of having changed anything to do with mail recently.

Does anyone have any suggestions based on experience ?
Is there any way to tell Procmail to dump the msgs in  /dev/null ?
I have read the various man pages several times & checked Gentoo Forum,
but there's no sign of anything relevant.

-- start of message --

X-Sieve: CMU Sieve 2.3
Envelope-to: purslow@ca.inter.net
Delivery-date: Mon, 05 Jan 2009 07:30:08 -0800
Date: Mon, 05 Jan 2009 10:30:03 -0500
From: Cron Daemon <root@ca-inter-net.uniserve.ca>
To: purslow@ca-inter-net.uniserve.ca
Subject: Cron <purslow@localhost> /usr/bin/fetchmail
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/home/purslow>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=purslow>
X-Cron-Env: <USER=purslow>
X-Scanner: OK. Scanned.
X-Uniserve-Spam-Score: 2.2 22 (++)
X-Uniserve-Spam-Report: Content analysis details:   (2.2 points)
        pts rule name              description
        ---- ---------------------- --------------------------------------------------
        0.9 TO_MALFORMED           To: has a malformed address
        1.3 ADDRESS_IN_SUBJECT     To: address appears in Subject
X-Scanner: OK. Scanned.
X-Uniserve-Spam-Score: 2.2 22 (++)
X-Uniserve-Spam-Report: Spam detection software, running on this mail system has
        identified this incoming email as possible spam.  An
        analysis of the message is below.
        Content analysis details:   (2.2 points)
        pts rule name              description
        ---- ---------------------- --------------------------------------------------
        0.9 TO_MALFORMED           To: has a malformed address
        1.3 ADDRESS_IN_SUBJECT     To: address appears in Subject

3 messages (3 seen) for purslow at pop.chass.utoronto.ca (22378 octets).
skipping message purslow@artemis.chass.utoronto.ca:1 not flushed
skipping message purslow@artemis.chass.utoronto.ca:2 not flushed
skipping message purslow@artemis.chass.utoronto.ca:3 not flushed
1 message for purslow at pop.ca.inter.net (2763 octets).
reading message purslow@ca-inter-net.uniserve.ca:1 of 1 (2763 octets).. flushed

-- end of message --

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca

Re: [gentoo-user] weird cron mail problem

[Willie Wong]

On Mon, Jan 05, 2009 at 11:42:40AM -0500, Philip Webb wrote:
> Starting yesterday IIRC I've been getting one of the messages below
> every time my Fetchmail cron job polls my  2  ISPs for new mail.
> The content of the msg clearly comes from my own Fetchmail,
> as it refers to both ISPs, but the msg itself originates at Uniserve,
> as is shown by the list of routings in the full header (not shown).
> Perhaps Uniserve has changed its procedures with this result,
> but before I phone their tech folks, I'ld like to be sure
> that it's not caused by something in my own machine.
> I'm not aware of having changed anything to do with mail recently.

What is Uniserve?

W
-- 
Willie W. Wong                                      wwong@math.princeton.edu
408 Fine Hall,  Department of Mathematics,  Princeton University,  Princeton
A mathematician's reputation rests on the number of bad proofs he has given.

Re: [gentoo-user] weird cron mail problem

[Philip Webb]

090105 Willie Wong wrote:
> What is Uniserve?

Sorry, I assumed the context made it clear: Uniserve is my ISP.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca

Re: [gentoo-user] weird cron mail problem

[Willie Wong]

On Mon, Jan 05, 2009 at 01:04:45PM -0500, Penguin Lover Philip Webb squawked:
> 090105 Willie Wong wrote:
> > What is Uniserve?
> 
> Sorry, I assumed the context made it clear: Uniserve is my ISP.

Okay... hum, have you checked your outgoing mail logs to see if cron
attempts to send mail when fetchmail is called? 

Actually, what is your cron recipe anyway? 

Also, have you updated either cron or fetchmail recently?

W
-- 
A nice box of chocolates can provide your total daily intake of calories
in one place.  Now, isn't that handy?
Sortir en Pantoufles: up 759 days, 16:59

Re: [gentoo-user] weird cron mail problem

[Philip Webb]

090105 Willie Wong wrote:
> On Mon, Jan 05, 2009 at 01:04:45PM -0500, Penguin Lover Philip Webb squawked:
> have you checked your outgoing mail logs
> to see if cron attempts to send mail when fetchmail is called? 

Thanks for the suggestion: the last few lines of  /var/log/syslog  are :

  Jan  5 17:10:01 localhost cron[4753]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons )
  Jan  5 17:10:01 localhost cron[4754]: (purslow) CMD (/usr/bin/fetchmail)
  Jan  5 17:10:11 localhost sSMTP[4765]: Sent mail for root@ca.inter.net (221 smtp-relay1.uniserve.ca closing connection) uid=1000 username=purslow outbytes=1049
  Jan  5 17:15:01 localhost cron[4908]: (purslow) CMD (/usr/bin/fetchmail)
  Jan  5 17:15:09 localhost sSMTP[4909]: Sent mail for root@ca.inter.net (221 smtp-relay2.uniserve.ca closing connection) uid=1000 username=purslow outbytes=952

> Actually, what is your cron recipe anyway? 

 /var/log/spool/cron/crontabs/<userid>  is :

  # DO NOT EDIT THIS FILE - edit the master and reinstall.
  # (/tmp/crontab.XXXXwsshN7 installed on Sun Nov  4 23:46:22 2007)
  # (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
  */5 * * * * /usr/bin/fetchmail

I considered trying 'fetchmail -s', but I'm not sure
how to add a flag in a cron file (perhaps by using a script).

> Also, have you updated either cron or fetchmail recently?

The problem originated 090104 c0520 ,
when I edited  ~/.fetchmailrc  to delete the reference to a logfile.
However, attempts to restore the STATVS QVO ANTE have failed:
I've restored the previous version of  .fetchmailrc  without success
& I've remerged Fetchmail, rebooted & then run  fetchmailconf ,
but the crazy mails continue to appear every  5 min  in my inbox.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca

[gentoo-user] Re: weird cron mail problem

[Harry Putnam]

Philip Webb <purslow@ca.inter.net> writes:

>> Also, have you updated either cron or fetchmail recently?
>
> The problem originated 090104 c0520 ,
> when I edited  ~/.fetchmailrc  to delete the reference to a logfile.
> However, attempts to restore the STATVS QVO ANTE have failed:
> I've restored the previous version of  .fetchmailrc  without success
> & I've remerged Fetchmail, rebooted & then run  fetchmailconf ,
> but the crazy mails continue to appear every  5 min  in my inbox.

I think that looks like normal cron mail...

Looks like you may have once had a redirect to `/dev/null' in the
crontab line and inadvertently removed it.


My fetchmail line in crontab:
 
  */15 * * * *   /usr/bin/fetchmail -f /home/reader/.fetchmailrc   >/dev/null 2>&1

Note the difference with yours:
  */5 * * * * /usr/bin/fetchmail

Not redirect in yours but note that I dump any output to /dev/null

You may first want to just say `cmd >/dev/null'

So that any errors are still send to you but once its working smoothly
you can add 2>&1 like  `cmd >/dev/null 2>&1' so that both stderr and
stdout go to dev/null

I think you can also do the same thing like this  `cmd  &>/dev/null

Re: [gentoo-user] Re: weird cron mail problem

[Philip Webb]

090105 Harry Putnam wrote:
> Philip Webb <purslow@ca.inter.net> writes:
>> The problem originated 090104 c0520 ,
>> when I edited  ~/.fetchmailrc  to delete the reference to a logfile.
>> However, attempts to restore the STATVS QVO ANTE have failed:
>> I've restored the previous version of  .fetchmailrc  without success
>> & I've remerged Fetchmail, rebooted & then run  fetchmailconf ,
>> but the crazy mails continue to appear every  5 min  in my inbox.
> I think that looks like normal cron mail...
> Looks like you may have once had a redirect to `/dev/null'
> in the crontab line and inadvertently removed it.

No, I have never touched the cron part of it since installing 0710xx .
It looks to me as if Fetchmail reacts badly
if you change anything by hand in  ~/.fetchmailrc :
I did exactly that without any problem when I changed ISPs 0809xx,
but I also find it odd that Fetchmail is sending the mail to Uniserve,
but not to my U Toronto e-ddress, which is also in  .fetchmailrc .
I have a few ideas of things to try today & will try yours too (thanks).
Any suggestions from others are very welcome.
 
> My fetchmail line in crontab:
> */15 * * * * /usr/bin/fetchmail -f /home/reader/.fetchmailrc >/dev/null 2>&1
> Note the difference with yours:
>   */5 * * * * /usr/bin/fetchmail
> Not redirect in yours but note that I dump any output to /dev/null
> You may first want to just say `cmd >/dev/null' ,
> so that any errors are still send to you but once its working smoothly
> you can add 2>&1 like `cmd >/dev/null 2>&1',
> so that both stderr and stdout go to dev/null
> I think you can also do the same thing like this  `cmd  &>/dev/null

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca

Re: [gentoo-user] weird cron mail problem

[Willie Wong]

On Mon, Jan 05, 2009 at 05:28:41PM -0500, Penguin Lover Philip Webb squawked:
> 090105 Willie Wong wrote:
> > On Mon, Jan 05, 2009 at 01:04:45PM -0500, Penguin Lover Philip Webb squawked:
> > have you checked your outgoing mail logs
> > to see if cron attempts to send mail when fetchmail is called? 
> 
> Thanks for the suggestion: the last few lines of  /var/log/syslog  are :
> 
>   Jan  5 17:10:01 localhost cron[4753]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons )
>   Jan  5 17:10:01 localhost cron[4754]: (purslow) CMD (/usr/bin/fetchmail)
>   Jan  5 17:10:11 localhost sSMTP[4765]: Sent mail for root@ca.inter.net (221 smtp-relay1.uniserve.ca closing connection) uid=1000 username=purslow outbytes=1049
>   Jan  5 17:15:01 localhost cron[4908]: (purslow) CMD (/usr/bin/fetchmail)
>   Jan  5 17:15:09 localhost sSMTP[4909]: Sent mail for root@ca.inter.net (221 smtp-relay2.uniserve.ca closing connection) uid=1000 username=purslow outbytes=952

Okay, this suggests that either cron or your mailer is misconfigured.
From the logs, you are running ssmtp. Check the config files for it to
see why mail intended for root@localhost is delivered to
root@ca.inter.net. And check /etc/crontab to see whether the MAILTO
field is set properly to where you want the mail to be delivered. 

Also, I don't know which implementation of cron you are using, but if
your crontab is under <userid>, then usually cron will demand
notification e-mails sent to <userid>... unless, hum, do a 

  ls -l /var/spool/cron/crontabs/

Is your user's crontab owned by root for some reason?

> > Actually, what is your cron recipe anyway? 
> 
>  /var/log/spool/cron/crontabs/<userid>  is :
> 
>   # DO NOT EDIT THIS FILE - edit the master and reinstall.
>   # (/tmp/crontab.XXXXwsshN7 installed on Sun Nov  4 23:46:22 2007)
>   # (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
>   */5 * * * * /usr/bin/fetchmail
> 
> I considered trying 'fetchmail -s', but I'm not sure
> how to add a flag in a cron file (perhaps by using a script).

for what it's worth, I have my fetchmail line as 
  /usr/bin/fetchmail > /dev/null
so I only get e-mails when fetchmail writes to stderr (so when there's
an error). 

Writing "fetchmail -s" instead of what you have really shouldn't be a
problem. Why do you think it needs a script? 

> > Also, have you updated either cron or fetchmail recently?
> 
> The problem originated 090104 c0520 ,
> when I edited  ~/.fetchmailrc  to delete the reference to a logfile.
> However, attempts to restore the STATVS QVO ANTE have failed:
> I've restored the previous version of  .fetchmailrc  without success
> & I've remerged Fetchmail, rebooted & then run  fetchmailconf ,
> but the crazy mails continue to appear every  5 min  in my inbox.

Okay, running fetchmail -s (or redirecting fetchmail output to
/dev/null) will probably cure the problem of the deluge. But I still
think you MDA is misconfigured for local mail. 

W
-- 
Q: What's grey and proves the nondenumerability of the Reals? 
A: Cantor's Diagonal Elephant
Sortir en Pantoufles: up 760 days, 15:25

Re: [gentoo-user] weird cron mail problem

[Willie Wong]

On Tue, Jan 06, 2009 at 11:56:55AM -0500, Penguin Lover Willie Wong squawked:
> Okay, running fetchmail -s (or redirecting fetchmail output to
> /dev/null) will probably cure the problem of the deluge. But I still
> think you MDA is misconfigured for local mail. 

Oops, my mistake. I meant MTA, not MDA in the above. 

Sorry, 

W
-- 
Pintsize: Nooooooo! I'm lactose intolerant!
Sortir en Pantoufles: up 760 days, 15:48

Re: [gentoo-user] weird cron mail problem

[Philip Webb]

090106 Willie Wong wrote:
> this suggests that either cron or your mailer is misconfigured.
> From the logs, you are running ssmtp. Check the config files for it
> to see why mail intended for root@localhost
> is delivered to root@ca.inter.net. 

 /etc/ssmtp/ssmtp.conf  has the lines (the last refers to my ISP):

  ...
  # The person who gets all mail for userids < 1000
  # Make this empty to disable rewriting.
  root=postmaster
  ...
  # The full hostname
  hostname=ca.inter.net
  ...

It looks as if this combination causes it to send to root@ca.inter.net .
I can't find a man file for  ssmtp.conf , only for  ssmtp ,
tho' the latter says the former should exist,
so I can't check in any more detail just how the lines above work,
eg what "rewriting" refers to or how the hostname is used.

> And check /etc/crontab to see whether the MAILTO field is set properly
> to where you want the mail to be delivered.

  MAILTO=root

this is for system cron jobs, which are run by & for root.
 
> Also, I don't know which implementation of cron you are using,
> but if your crontab is under <userid>, ...

Yes, with mail that's necessary for security.

> ... then usually cron will demand notification e-mails sent to <userid>:
> Try 'ls -l /var/spool/cron/crontabs/' :
> is your user's crontab owned by root for some reason?

No :

  root:550 crontabs> ls -l
  -rw------- 1 purslow crontab 234 2007-11-04 23:46 purslow

> I have my fetchmail line as 
>   /usr/bin/fetchmail > /dev/null
> so I only get e-mails when it writes to stderr (when there's an error).

That looks like the quick fix.

> Writing "fetchmail -s" instead shouldn't be a problem.
> Why do you think it needs a script? 

I didn't know if cron will ignore stuff after the space otherwise,
but obviously it does, if your line above works.

> I still think you MTA is misconfigured for local mail. 

Like you, I'ld like to understand the real cause of the problem:
Gentoo is for people like us ... (smile)

Rewriting  .fetchmailrc  using  fetchmailconf  thro'out didn't help,
so that's one set of possibilities ruled out
& I will try the suggestions above (thanks) next.
Your or others' further comments are very welcome.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca

Re: [gentoo-user] weird cron mail problem

[Willie Wong]

On Tue, Jan 06, 2009 at 01:22:28PM -0500, Philip Webb wrote:
> 090106 Willie Wong wrote:
> > this suggests that either cron or your mailer is misconfigured.
> > From the logs, you are running ssmtp. Check the config files for it
> > to see why mail intended for root@localhost
> > is delivered to root@ca.inter.net. 
> 
>  /etc/ssmtp/ssmtp.conf  has the lines (the last refers to my ISP):
> 
>   ...
>   # The person who gets all mail for userids < 1000
>   # Make this empty to disable rewriting.
>   root=postmaster
>   ...
>   # The full hostname
>   hostname=ca.inter.net
>   ...
> 
> It looks as if this combination causes it to send to root@ca.inter.net .
> I can't find a man file for  ssmtp.conf , only for  ssmtp ,
> tho' the latter says the former should exist,
> so I can't check in any more detail just how the lines above work,
> eg what "rewriting" refers to or how the hostname is used.
> 

Okay, I am now a bit confused. What is "ca.inter.net"? A whois shows
no match. Does it refer to your ISP? or to your personal box? 

In any case, you may want to change the root line to 
"root=purslow" so the mail gets sent to purslow instead of postmaster
(which according to /etc/mail/aliases becomes root again). 

Also, just a personal preference: but on systems that are on for long
enough of periods to have daemon process that will be emailing stuff
to sys admins (processes like cron and such), I'd usually install an
actual MTA instead of something like ssmtp. 

W
-- 
Willie W. Wong                                      wwong@math.princeton.edu
408 Fine Hall,  Department of Mathematics,  Princeton University,  Princeton
A mathematician's reputation rests on the number of bad proofs he has given.

Re: [gentoo-user] weird cron mail problem

[Philip Webb]

090106 Willie Wong wrote:
> you may want to change the root line to "root=purslow",
> so the mail gets sent to purslow instead of postmaster
> (which according to /etc/mail/aliases becomes root again).

That doesn't work, but adding '> /dev/null' or '-s' in  crontab  does.
The latter seems simpler, so that's what I've done.

It doesn't explain why the problem suddenly arose last Sunday
after I made a simple editing change in  .fetchmailrc
& nothing like this had happened before
with the same  crontab  &  ssmtp.conf : perhaps there's an obscure bug,
but the irritating problem has been resolved & I have other jobs today.

Thanks for the helpful advice (smile).

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca

[gentoo-user] Re: weird cron mail problem

[Harry Putnam]

Philip Webb <purslow@ca.inter.net> writes:

> 090106 Willie Wong wrote:
>> you may want to change the root line to "root=purslow",
>> so the mail gets sent to purslow instead of postmaster
>> (which according to /etc/mail/aliases becomes root again).
>
> That doesn't work, but adding '> /dev/null' or '-s' in  crontab  does.
> The latter seems simpler, so that's what I've done.
>
> It doesn't explain why the problem suddenly arose last Sunday
> after I made a simple editing change in  .fetchmailrc
> & nothing like this had happened before
> with the same  crontab  &  ssmtp.conf : perhaps there's an obscure bug,
> but the irritating problem has been resolved & I have other jobs today.
>
> Thanks for the helpful advice (smile).

Looking back thru the thread I don't see the actual change made to
fetchmailrc.   Maybe just blind.

From my experience fetchmail is a very robust (non buggy) and easily
configured tool.  At least for my simple usage.  It's the only config
I can think of that is edited much like you might talk to friend while
walking along.

Something that happens from time to time is introducing an unprintable
CHAR into a *.rc file and not being able to see it.  I'm not sure if
fetchmail would respond poorly to that.

If there is any chance of that; you might want to use vim to check
each line. You can hit the el (l) lowercase, on each line to expose
most kinds of unprintable char.

It takes 3 key strokes to show the line.

1) :
2) l
3) enter

Then the line appears in the command area along with any unprintable
chars, 

As Willie mentioned the mail mta is capable of rewriting stuff in its
configurations.

Do you control this machine? Sorry if you've already covered that.

Another unlikely thing that can catch you ... happened to me on a
remote account I didn't control.  

The machine underwent some kind of mishap that required serious backup
effort replacing all us users files from backups.   Turned out the
backups were pretty old and further I missed the notification about
the mishap... the next thing I knew lots of strange things began to
happen.

Some of my older configurations were re-introduced in place of things
I had changed due to new circumstances.

Re: [gentoo-user] Re: weird cron mail problem: basically solved

[Philip Webb]

090106 Harry Putnam wrote:
> Philip Webb <purslow@ca.inter.net> writes:
>> 090106 Willie Wong wrote:
>>> you may want to change the root line to "root=purslow",
>>> so the mail gets sent to purslow instead of postmaster
>>> (which according to /etc/mail/aliases becomes root again).
>> That doesn't work, but adding '> /dev/null' or '-s' in  crontab  does.
>> The latter seems simpler, so that's what I've done.
>> It doesn't explain why the problem suddenly arose last Sunday
>> after I made a simple editing change in  .fetchmailrc
>> & nothing like this had happened before
>> with the same  crontab  &  ssmtp.conf : perhaps there's an obscure bug,
>> but the irritating problem has been resolved & I have other jobs today.
> I don't see the actual change made to fetchmailrc.

The problem started after I commented the line 
  'set logfile "/home/purslow/Mail/logfile"'.

> You can introduce an unprintable CHAR into a *.rc file
> and not be able to see it.  you might want to use vim to check each line.
> You can hit the el (l) lowercase on each line
> to expose most kinds of unprintable char:
>   1) :   2) l   3) enter
> Then the line appears in the command area with any unprintable chars, 

I tried that, but there's no sign of an unprintable character.

> Do you control this machine ?

Yes, I built it & no-one else ever gets near it.

Nearly all of the problem seems clear now, so for the record:
the basic problem arose because when I went over to downloading mail
via a user cronjob for security, I didn't add '-s' to 'fetchmail';
as a result, Fetchmail was writing several lines to  ~/Mail/logfile
every  5 min  with the obvious result that the file got very big;
frustrated by this, I took a quick look & the simple fix seemed to be
to remove the line in  ~/.fetchmailrc  which defined it (see above);
as soon as I did this (early Sun), msgs starting arriving in my spamtrap
mysteriously originating at 'root@myISP'; moreover, other lines continued
to be added to  logfile , which I realised came from Procmail,
so I altered  ~/.procmailrc  to suppress those (successfully);
to try to stop the mail msgs, I restored the original  ~/.fetchmailrc ,
but that had no effect, at which point I sought help via this list.

The correct procedure is to stop the msgs at source with 'fetchmail -s',
so having done that, the real-life irritation has gone away.
It's also clear how the rogue e-mails arose: when a cronjob ouputs msgs,
they are handled by Cron itself (see 'man cron'), not eg by Fetchmail,
so Cron sent them to 'root', which led all around the haystack.
What remains unclear is why restoring the original  ~/.fetchmailrc
didn't cause the msgs to be sent again to  ~/Mail/logfile ,
but that's not important enough to take more of my time.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca