[gentoo-dev] rfc: logrotate and xinetd use flags

[gentoo-dev] rfc: logrotate and xinetd use flags

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1253 bytes --]

All,

I know a decision about this type of use flag was made in the past, but
especially now with the --newuse option in portage, that decision
bothers me, so I would like to re-open the discussion.

I will use logrotate as my example, but what I'm saying also applies for
xinetd.

I feel that the current approach (using INSTALL_MASK) to control whether
these configuration files are installed or not is not well documented.
We tell people about it on the mailing lists, but I do not know of a
place where it is documented.

Also, it seems to be an all or nothing arrangement. If I do not want
logrotate support, I have to set the INSTALL_MASK then if I decide later
I want it, I have to unset the INSTALL_MASK and run "emerge -e world" to
get the files installed.

If we use a "logrotate" or "xinetd" use flag, it gives the users better
control of which packages have this support, and the --newuse option in
portage can be used to rebuild only the affected packages.

I guess the argument against the use flag was that packages were being
rebuilt just to install configuration files. I can see how that could be
a pita for big packages. Did anyone ever bring up using pkg_config to
un/install these files based on the use flags?

Comments?

William


[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] rfc: logrotate and xinetd use flags

[Jeroen Roovers]

On Sun, 24 Apr 2011 16:58:23 -0500
William Hubbs <williamh@gentoo.org> wrote:

> I feel that the current approach (using INSTALL_MASK) to control
> whether these configuration files are installed or not is not well
> documented. We tell people about it on the mailing lists, but I do
> not know of a place where it is documented.

INSTALL_MASK is documented in make.conf(5).

> Also, it seems to be an all or nothing arrangement. If I do not want
> logrotate support, I have to set the INSTALL_MASK then if I decide
> later I want it, I have to unset the INSTALL_MASK and run "emerge -e
> world" to get the files installed.

That sounds like a design decision about your own special Gentoo based
distro - it is the same with most USE flags and with other build time
settings like USE flags.

Usually when you decide to use INSTALL_MASK, it's because you want an
as small as possible system image - you're probably developing an
embedded system with storage constraints. If you're really clever about
that and want to delay the INSTALL_MASK decision, you might as well
`mount --bind' some directory on the development system
to /etc/logrotate.d inside the chroot before you start `emerge world'.

Another option is to plan your distribution even better and set
FEATURES=buildpkg. Then, you can unset INSTALL_MASK in make.conf or on
the command line, and emerge the .tbz2 - no rebuilding needed. Maybe
*this* could be better documented, although it does belong in the tips
and tricks department.

(And if you really want to prevent portage from even putting certain
files in the .tbz2, then use PKG_INSTALL_MASK instead.)

> If we use a "logrotate" or "xinetd" use flag, it gives the users
> better control of which packages have this support, and the --newuse
> option in portage can be used to rebuild only the affected packages.

`ls /etc/logrotate.d' should give a clue in the current situation. :)

> I guess the argument against the use flag was that packages were being
> rebuilt just to install configuration files. I can see how that could
> be a pita for big packages. Did anyone ever bring up using pkg_config
> to un/install these files based on the use flags?

It isn't just that you have to rebuild a large package, perhaps
consuming a lot of time or other resources unrelated to actual storage,
but that a package with a large install image (many megabytes) does not
benefit from a USE flag that controls installing an extra kilobyte
(plus a couple more on a typical filesystem).


     jer

[gentoo-dev] Re: rfc: logrotate and xinetd use flags

[Duncan]

William Hubbs posted on Sun, 24 Apr 2011 16:58:23 -0500 as excerpted:

> I feel that the current approach (using INSTALL_MASK) to control whether
> these configuration files are installed or not is not well documented.
> We tell people about it on the mailing lists, but I do not know of a
> place where it is documented.

It's documented in the documentation, of course. =:^)

From the make.conf (5) manpage:

INSTALL_MASK = [space delimited list of file names]
    Use  this  variable if you want to selectively prevent certain files
    from being copied into your file system tree.  This does not work on
    symlinks, but only on actual  files.   Useful if  you wish to filter
    out files like HACKING.gz and TODO.gz. The INSTALL_MASK is processed
    just before a package is merged.   Also  supported  is  a
    PKG_INSTALL_MASK  variable  that behaves  exactly  like  INSTALL_MASK
    except that it is processed just before creation of a binary package.

But I agree that isn't as well documented as it might be, despite the 
manpage documentation.  Having it in the handbook's working with portage 
section would certainly add to its visibility. But I'm not sure that 
section, despite being dedicated to portage, is intended to be an 
exhaustive examination of the available settings.  I believe it's entirely 
appropriate to only have that in the manpage, as arguably, people to whom 
the near-first thought when looking for such a feature is NOT to go see 
what the manpage has to say about it, perhaps shouldn't be messing with 
the feature in the first place.

IOW, if it weren't in the manpage, I'd say that's a serious bug, the 
manpage needs fixed.  Since it /is/ in the manpage, and Gentoo's manpages 
are arguably much more accessible than most, additional documentation 
would be nice, but is arguably low priority TRIVIAL/ENHANCEMENT level 
stuff.

> Also, it seems to be an all or nothing arrangement. If I do not want
> logrotate support, I have to set the INSTALL_MASK then if I decide later
> I want it, I have to unset the INSTALL_MASK and run "emerge -e world" to
> get the files installed.

That's not really the case, either.  It's certainly possible to add 
absolute-path-specified names to INSTALL_MASK, and I've done just that in 
the past, tho I'm not doing so ATM.

It's also worth noting the /etc/portage/package.env file and /etc/portage/
env/ subdirs as covered in the portage (5) manpage.  Either of these 
methods can be used to set package-specific stuff like INSTALL_MASK, if 
the user doesn't want to make it global.  And the env/category/pkg feature 
can even be used for such shell logic as...

CFLAGS=${CFLAGS/ -combine/}

... which I routinely depend on for a number of packages as -combine is in 
my global CFLAGS (but not CXXFLAGS).

It's thus possible to use either absolute paths in the global INSTALL_MASK 
settings, or package-specific setting via two different methods, to 
control the installation of specific files.  And the documentation is 
there, in the usual and customary place one would expect to find it on a 
*ix system, the manpages.

I honestly don't see how a USE flag helps.  Yeah, it rubs the user's nose 
in it a bit more directly, but the users that really care either already 
know or can find out pretty fast when they need to, how to address the 
problem on their own systems, and for those that don't, it's yet more USE 
flag noise to mask the important stuff.  Plus, any USE flag added to a 
package is yet another bit of package maintenance for the gentoo 
maintainer to keep up on.  I simply don't see how the limited benefit can 
be considered worth the hassle, when both as explained here and as 
documented, the control is already there for those that actually care 
about it.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] rfc: logrotate and xinetd use flags

[Alec Warner]

On Sun, Apr 24, 2011 at 2:58 PM, William Hubbs <williamh@gentoo.org> wrote:
> All,
>
> I know a decision about this type of use flag was made in the past, but
> especially now with the --newuse option in portage, that decision
> bothers me, so I would like to re-open the discussion.
>
> I will use logrotate as my example, but what I'm saying also applies for
> xinetd.

In general having use-flags control the *config* that we ship with
packages is a silly idea.  Ideally we should ship whatever upstream
ships.  If they don't ship a logrotate config but we want to ship one,
awesome!  There is no way we can make enough unique packages to
support everyone's configuration...that is why users can configure
things to their liking themselves.

In the end there are really only a few ways this ends up:

1) We install logrotate but the user doesn't use it because they do
not have logrotate installed. This is not a big deal.  For example we
install headers all the time that the user *may* not never use to
compile against.  I don't see users complaining about that.  The
logrotate file is small and it only has 1 directory in /etc.
2) We install logrotate but the user does not like the defaults.  In
that case they should modify the defaults.  They have plenty of
options (puppet, cfengine, chef, bcgf2, a post install hook for the
package mangler...) to change the content of the logrotate file.
3) We install logrotate and the user uses the default.

>
> I feel that the current approach (using INSTALL_MASK) to control whether
> these configuration files are installed or not is not well documented.
> We tell people about it on the mailing lists, but I do not know of a
> place where it is documented.
>
> Also, it seems to be an all or nothing arrangement. If I do not want
> logrotate support, I have to set the INSTALL_MASK then if I decide later
> I want it, I have to unset the INSTALL_MASK and run "emerge -e world" to
> get the files installed.

I would argue that this is an edge case.  How many users actually use
install mask in this manner?  I imagine most users who don't care
about logrotate just don't have the package installed and leave the
files in /etc

>
> If we use a "logrotate" or "xinetd" use flag, it gives the users better
> control of which packages have this support, and the --newuse option in
> portage can be used to rebuild only the affected packages.

I don't want to get into the habit of having use flags for run-time
package configuration.

>
> I guess the argument against the use flag was that packages were being
> rebuilt just to install configuration files. I can see how that could be
> a pita for big packages. Did anyone ever bring up using pkg_config to
> un/install these files based on the use flags?
>
> Comments?
>
> William
>
>

Re: [gentoo-dev] rfc: logrotate and xinetd use flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1614 bytes --]

On Sun, 24 Apr 2011 16:58:23 -0500
William Hubbs <williamh@gentoo.org> wrote:

> I feel that the current approach (using INSTALL_MASK) to control
> whether these configuration files are installed or not is not well
> documented. We tell people about it on the mailing lists, but I do
> not know of a place where it is documented.
> 
> Also, it seems to be an all or nothing arrangement. If I do not want
> logrotate support, I have to set the INSTALL_MASK then if I decide
> later I want it, I have to unset the INSTALL_MASK and run "emerge -e
> world" to get the files installed.

I'm currently working on improving the situation. I'd like to create
a tool which would allow users to easily add and remove INSTALL_MASKs
(including a few common sets), and rebuild the packages as necessary.
This, however, would require some fixes in portage [1].

> I guess the argument against the use flag was that packages were being
> rebuilt just to install configuration files. I can see how that could
> be a pita for big packages. Did anyone ever bring up using pkg_config
> to un/install these files based on the use flags?

I don't think that would work. And if it does, then either PM or even
PMS is broken by design. The pkg_config() phase should have the same
flags set as the build had, or don't refer to flags at all.

In addition, upstream-installed config files often require running sed.
And this requires unpacking & configuring the package. Really don't
seem appropriate for pkg_config().

[1] http://bugs.gentoo.org/show_bug.cgi?id=364633

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] rfc: logrotate and xinetd use flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 347 bytes --]

On Sun, 24 Apr 2011 16:58:23 -0500
William Hubbs <williamh@gentoo.org> wrote:

> I will use logrotate as my example, but what I'm saying also applies
> for xinetd.

PS I think this does apply more or less to the following flags as well:
- bash-completion,
- zsh-completion,
- (possibly) vim-syntax.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] rfc: logrotate and xinetd use flags

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 420 bytes --]

On Mon, 25 Apr 2011 17:39:18 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> PS I think this does apply more or less to the following flags as
> well:
>
> - (possibly) vim-syntax.

Vim syntax packages are usually distributed elsewhere rather than as
part of the main project. Also, if they're installing any
documentation, they need Vim installed to get the help tags done
properly.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] rfc: logrotate and xinetd use flags

[Jeroen Roovers]

On Mon, 25 Apr 2011 09:44:24 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> I'm currently working on improving the situation. I'd like to create
> a tool which would allow users to easily add and remove INSTALL_MASKs
> (including a few common sets), and rebuild the packages as necessary.
> This, however, would require some fixes in portage [1].

As I said in a previous reply to this thread, no rebuilding is
necessary to switch your INSTALL_MASK preference if you already use
FEATURES=buildpkg.


     jer

Re: [gentoo-dev] rfc: logrotate and xinetd use flags

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 751 bytes --]

On Mon, 25 Apr 2011 22:57:46 +0200
Jeroen Roovers <jer@gentoo.org> wrote:

> On Mon, 25 Apr 2011 09:44:24 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > I'm currently working on improving the situation. I'd like to create
> > a tool which would allow users to easily add and remove
> > INSTALL_MASKs (including a few common sets), and rebuild the
> > packages as necessary. This, however, would require some fixes in
> > portage [1].
> 
> As I said in a previous reply to this thread, no rebuilding is
> necessary to switch your INSTALL_MASK preference if you already use
> FEATURES=buildpkg.

'Rebuild' sounded better than 'remerge' :P. In any case, 'emerge -1k'
is fine too here :P.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]