[gentoo-dev] qmail and .keep files

[gentoo-dev] qmail and .keep files

[Vitaly Kushneriuk]

Hi!
Just wandering, what are those ".keep" files for?
Looks to me like a very ugly hack.
Anyway, ebuild for qmail should not install them in the 
queue directory, for qmail-qstat to work properly.
Also qmail rc file should be config-protected.
As there's no config-protection on per file basis (am I right?)
we should change it to:
---------------------
#!/bin/sh
exec env - PATH="/var/qmail/bin:$PATH" \
qmail-start "`cat /var/qmail/control/defaultdelivery`"
---------------------

with control/defaultdelivery containing;
-------------
|dot-forward .forward
./.maildir/
-------------
It will also fix the bug in current rc file: "dor-forward and
./.maildir/ should be on SEPARATE lines.

Gentoo rules :-)

Re: [gentoo-dev] qmail and .keep files

[Geert Bevin]

On Thu, 2001-12-13 at 12:33, Vitaly Kushneriuk wrote:
> 
> 
> Hi!
> Just wandering, what are those ".keep" files for?
> Looks to me like a very ugly hack.

The .keep files are necessary since otherwise when merging a new version
and unmerging the old one, required empty dirs would be removed too.
This is the way gentoo solves this problem standardly. Just look for
.keep files all over your hd.

> Anyway, ebuild for qmail should not install them in the 
> queue directory, for qmail-qstat to work properly.

Do you have real problems due to the .keep files?

> Also qmail rc file should be config-protected.
> As there's no config-protection on per file basis (am I right?)
> we should change it to:
> ---------------------
> #!/bin/sh
> exec env - PATH="/var/qmail/bin:$PATH" \
> qmail-start "`cat /var/qmail/control/defaultdelivery`"
> ---------------------
> 
> with control/defaultdelivery containing;
> -------------
> |dot-forward .forward
> ./.maildir/
> -------------
> It will also fix the bug in current rc file: "dor-forward and
> ./.maildir/ should be on SEPARATE lines.

Good idea ! I'll add that

> Gentoo rules :-)
> _______________________________________________
> gentoo-dev mailing list
> gentoo-dev@gentoo.org
> http://lists.gentoo.org/mailman/listinfo/gentoo-dev
> 
-- 
Geert Bevin
the Leaf sprl/bvba
"Use what you need"           Pierre Theunisstraat 1/47
http://www.theleaf.be         1030 Brussels
gbevin@theleaf.be             Tel & Fax +32 2 241 19 98

Re: [gentoo-dev] qmail and .keep files

[Vitaly Kushneriuk]

[-- Attachment #1: Type: text/plain, Size: 2990 bytes --]

On Thu, 2001-12-13 at 13:39, Geert Bevin wrote:
> 
> On Thu, 2001-12-13 at 12:33, Vitaly Kushneriuk wrote:
> > 
> > 
> > Hi!
> > Just wandering, what are those ".keep" files for?
> > Looks to me like a very ugly hack.
> 
> The .keep files are necessary since otherwise when merging a new version
> and unmerging the old one, required empty dirs would be removed too.
> This is the way gentoo solves this problem standardly. Just look for
> .keep files all over your hd.
I know, and as I said, looks like a VERY ugly way of doing that.

Yet, no package should "own", in any way, the standard directories. They
can be created form post_install of, say, base_layout, and left there
forever. Then if a package provides /usr/bin/program, only program
should be included in CONTENTS, not the /usr/bin. The .keep files
can be used during build process, to indicate directories, that
do bolong to the package, but should not be installed (better to wrap
 this by some "dokeepdir" func, to allow future change). 
During unmerge, only "owned" directories can be removed, 
and only if no other installed package uses them. A file with dir names
and usage counter can be used to track that.


> 
> > Anyway, ebuild for qmail should not install them in the 
> > queue directory, for qmail-qstat to work properly.
> 
> Do you have real problems due to the .keep files?
Yes. run /var/qmail/bin/qmail-qstat. You'll get
about 20 unprocessed messages in the queue, while it's
actualy empty.
> 
> > Also qmail rc file should be config-protected.
> > As there's no config-protection on per file basis (am I right?)
> > we should change it to:
> > ---------------------
> > #!/bin/sh
> > exec env - PATH="/var/qmail/bin:$PATH" \
> > qmail-start "`cat /var/qmail/control/defaultdelivery`"
> > ---------------------
> > 
> > with control/defaultdelivery containing;
> > -------------
> > |dot-forward .forward
> > ./.maildir/
> > -------------
> > It will also fix the bug in current rc file: "dor-forward and
> > ./.maildir/ should be on SEPARATE lines.
> 
> Good idea ! I'll add that
While you on it, consider attached qmail control script with many new
options.
Note: /init.d/svscan should be fixed not to use start-stop-daemon,
as it does not allow for more then one copy of svscan running.
It is better to run separate svscan in qmail/supervise dir then
linking qmail/supervise/* to /service, as this way you have better
control of when qmail will start.

> 
> > Gentoo rules :-)
> > _______________________________________________
> > gentoo-dev mailing list
> > gentoo-dev@gentoo.org
> > http://lists.gentoo.org/mailman/listinfo/gentoo-dev
> > 
> -- 
> Geert Bevin
> the Leaf sprl/bvba
> "Use what you need"           Pierre Theunisstraat 1/47
> http://www.theleaf.be         1030 Brussels
> gbevin@theleaf.be             Tel & Fax +32 2 241 19 98
> 
> _______________________________________________
> gentoo-dev mailing list
> gentoo-dev@gentoo.org
> http://lists.gentoo.org/mailman/listinfo/gentoo-dev


[-- Attachment #2: qmail --]
[-- Type: text/x-sh, Size: 2336 bytes --]

#!/bin/sh
# /etc/rc.d/init.d/qmail
#
# description: Start, stop, restart, reload, and otherwise signal qmail.
#
# script modified by Vitaly Kushneriuk<vitaly@jungo.com> to support Gentoo init model.
# script modified by Vincent Danen <vdanen@mandrakesoft.com> on July 17 2000
# revision: 10/12/00 - added qmail-pop3d support
#           02/25/01 - added qmail-qmqpd support
#                    - added support for split packages
#

PATH=/var/qmail/bin:/usr/local/bin:/usr/bin:/bin
export PATH

depend(){
	need net
}

start(){
    ebegin "Starting qmail supervise"
    cd /var/qmail/supervise
    env - PATH="$PATH" svscan &
    echo $! > /var/run/qmail-svscan.pid
    eend ${?}	
}

stop(){
    ebegin "Stopping qmail svscan"
    kill `cat /var/run/qmail-svscan.pid`
    eend 0
    ebegin "Stopping qmail"
    svc -dx /var/qmail/supervise/*
    eend 0
    ebegin "Stopping qmail logs"
    svc -dx /var/qmail/supervise/*/log
    eend 0
}

stat(){
    cd /var/qmail/supervise
    svstat * */log
}

alrm(){
    echo "Sending ALRM signal to qmail-send."
    svc -a /var/qmail/supervise/qmail-send
}

doqueue(){
	alrm
}

queue(){
    qmail-qstat
    qmail-qread
}

reload(){
    hup
}

hup(){
    echo "Sending HUP signal to qmail-send."
    svc -h /var/qmail/supervise/qmail-send
}

pause(){
    echo "Pausing qmail-send"
    svc -p /var/qmail/supervise/qmail-send
    echo "Pausing qmail-smtpd"
    svc -p /var/qmail/supervise/qmail-smtpd
    if [ -d /var/qmail/supervise/qmail-pop3d ]; then
      echo "Pausing qmail-pop3d"
      svc -p /var/qmail/supervise/qmail-pop3d
    fi
    if [ -d /var/qmail/supervise/qmail-qmqpd ]; then
      echo "Pausing qmail-qmqpd"
      svc -p /var/qmail/supervise/qmail-qmqpd
    fi
}

cont(){
    echo "Continuing qmail-send"
    svc -c /var/qmail/supervise/qmail-send
    echo "Continuing qmail-smtpd"
    svc -c /var/qmail/supervise/qmail-smtpd
    if [ -d /var/qmail/supervise/qmail-pop3d ]; then
      echo "Continuing qmail-pop3d"
      svc -c /var/qmail/supervise/qmail-pop3d
    fi
    if [ -d /var/qmail/supervise/qmail-qmqpd ]; then
      echo "Continuing qmail-qmqpd"
      svc -c /var/qmail/supervise/qmail-qmqpd
    fi
}

opts="start stop stat restart alrm doqueue queue reload hup pause cont"

Re: [gentoo-dev] qmail and .keep files

[Geert Bevin]

On Thu, 2001-12-13 at 13:18, Vitaly Kushneriuk wrote:
> I know, and as I said, looks like a VERY ugly way of doing that.

I agree that it's not pretty.

> Yet, no package should "own", in any way, the standard directories. They
> can be created form post_install of, say, base_layout, and left there
> forever. Then if a package provides /usr/bin/program, only program
> should be included in CONTENTS, not the /usr/bin. The .keep files
> can be used during build process, to indicate directories, that
> do bolong to the package, but should not be installed (better to wrap
>  this by some "dokeepdir" func, to allow future change). 
> During unmerge, only "owned" directories can be removed, 
> and only if no other installed package uses them. A file with dir names
> and usage counter can be used to track that.

I don't know if what you're proposing is a good alternative. I'll have
to think about this when I have the time. Anyway these kind of decisions
should be made by drobbins. Daniel, what are your thoughts on this
matter?

> > > Anyway, ebuild for qmail should not install them in the 
> > > queue directory, for qmail-qstat to work properly.

The problem is that when I remove them now for the qmail-qstat to
function properly, an unmerge of a previously installed package might
remove important queue dirs which are need for qmail to function. When
having to chose between the two I tend to prefer having a properly
running qmail system in all cases than an incorrectly reported number by
qstat.

> While you on it, consider attached qmail control script with many new
> options.
> Note: /init.d/svscan should be fixed not to use start-stop-daemon,
> as it does not allow for more then one copy of svscan running.
> It is better to run separate svscan in qmail/supervise dir then
> linking qmail/supervise/* to /service, as this way you have better
> control of when qmail will start.

I'll have a look at it when I get the time.

-- 
Geert Bevin
the Leaf sprl/bvba
"Use what you need"           Pierre Theunisstraat 1/47
http://www.theleaf.be         1030 Brussels
gbevin@theleaf.be             Tel & Fax +32 2 241 19 98

Re: [gentoo-dev] qmail and .keep files

[Jerry A!]

On Thu, Dec 13, 2001 at 01:54:11PM +0100, Geert Bevin wrote:
: On Thu, 2001-12-13 at 13:18, Vitaly Kushneriuk wrote:
:
: The problem is that when I remove them now for the qmail-qstat to
: function properly, an unmerge of a previously installed package might
: remove important queue dirs which are need for qmail to function. When
: having to chose between the two I tend to prefer having a properly
: running qmail system in all cases than an incorrectly reported number by
: qstat.

You know, qmail is the only MTA in the portage tree that does not
rebuild it's queue structure automagically.  Maybe we should look at
making an ebuild for one of the "qmail-queue-fixer programs" and just
have qmail-sumo (great name BTW) install that as well.  Heck, we could
even have it called from the init script -- so that it's seamless to the
end user.

For the record, I would prefer having the correct number of messages
reported by qstat.  Last thing we want to be is known as the distro that
couldn't get qmail right.  8)

        --Jerry

Open-Source software isn't a matter of life or death...
...It's much more important than that!

Re: [gentoo-dev] qmail and .keep files - admin consideration?

[jano]

I hope I got this right: the .keep files are there to prevent the qmail
queue directory from being clobbered during an unmerge, correct?  My take on
this discussion will be obviously OT, so any response to this would best
come directly to me (unless deemed proper for gentoo-dev)

>From an administration standpoint, however, how often is qmail, on a
production box, actually updated?  Some patches, e.g. smtp auth or
qmail-ldap, come out with a new version every.. say.. several months, but
that's about all I've seen.  The main patches (big todo, concurrency,
tarpiting, reiser sync) haven't changed much.  On a box where you're
constantly installing various qmail patches, i can see how the queue
directory issue might get in the way, but on a production box, for now, I
would simply remove those .keep files and not worry about it (either from
the ebuild before installing, or manually after install).  Of course this is
a no brainer, a moot point, i feel silly having brought it up.  *shrug*

Oh, but great thing about modifying an ebuild is that your modified version
is stored in /var/db/pkg :)  I love that :)))

Re: [gentoo-dev] qmail and .keep files - admin consideration?

[Jerry A!]

On Thu, Dec 13, 2001 at 09:45:39AM -0800, jano wrote:
:
:
: I hope I got this right: the .keep files are there to prevent the qmail
: queue directory from being clobbered during an unmerge, correct?  My take on
: this discussion will be obviously OT, so any response to this would best
: come directly to me (unless deemed proper for gentoo-dev)

Come to think of it, something isn't making sense to me.  I thought that
files and directories weren't unmerged if the mtimes were different from
the ones recorded when the portage was installed.  If that's the case
then, empty or not, these directories shouldn't get deleted b/c they
should all have been touched when the newer version of a portage is
installed.

Or am I missing something?

        --Jerry

Open-Source software isn't a matter of life or death...
...It's much more important than that!

Re: [gentoo-dev] qmail and .keep files

[Daniel Robbins]

On Thu, Dec 13, 2001 at 01:54:11PM +0100, Geert Bevin wrote:
> The problem is that when I remove them now for the qmail-qstat to
> function properly, an unmerge of a previously installed package might
> remove important queue dirs which are need for qmail to function. When
> having to chose between the two I tend to prefer having a properly
> running qmail system in all cases than an incorrectly reported number by
> qstat.

In this situation, the dir ${ROOT}/var/qmail... can be created in pkg_postinst.
Then it won't be autocleaned.  Making this "prettier" is of course do-able but
is not a high priority atm.

Best Regards,

-- 
Daniel Robbins                                  <drobbins@gentoo.org>
Chief Architect/President                       http://www.gentoo.org 
Gentoo Technologies, Inc.

Re: [gentoo-dev] qmail and .keep files - admin consideration?

[Daniel Robbins]

On Thu, Dec 13, 2001 at 12:47:11PM -0500, Jerry A! wrote:
> Come to think of it, something isn't making sense to me.  I thought that
> files and directories weren't unmerged if the mtimes were different from
> the ones recorded when the portage was installed.  If that's the case
> then, empty or not, these directories shouldn't get deleted b/c they
> should all have been touched when the newer version of a portage is
> installed.
> 
> Or am I missing something?

mtimes aren't reliable for directories and not used for determining to unmerge.

-- 
Daniel Robbins                                  <drobbins@gentoo.org>
Chief Architect/President                       http://www.gentoo.org 
Gentoo Technologies, Inc.

Re: [gentoo-dev] qmail and .keep files

[Geert Bevin]

Hello,

I made the changes to the qmail ebuild so that only .keep files are
created for directories that risk being empty after an update. Also the
queue dir hierarchy is created in pkg_postinstall now to fix the wrong
email count reported by qmail-qstat. I've also added the defaultdelivery
file and updated the qmail rc file for it. I didn't have time to look at
the init script so this hasn't been added yet. I'm not sure however that
I agree with you about having several svscan running. This seems a bit
overkill to me an not how the tools seems to have to be used.

Changes should be rsyncable is half an hour.

Geert

-- 
Geert Bevin
the Leaf sprl/bvba
"Use what you need"           Pierre Theunisstraat 1/47
http://www.theleaf.be         1030 Brussels
gbevin@theleaf.be             Tel & Fax +32 2 241 19 98