[gentoo-dev] rfc: add-on files handling improvements

[gentoo-dev] rfc: add-on files handling improvements

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1490 bytes --]

All,

I want to start a discussion about our add-on files practice and try to
improve it.

I agree it is reasonable to install bash completions
unconditionally, because bash is part of the base requirement for
Gentoo. However, I do not agree that we should continue installing
add-on files for everything under the sun unconditionally.

I believe, back in the day we started this practice, portage did not
support --newuse or --changed-use, so there was no way to only update
packages that had changed or new use flags. In that situation, I
understand why we installed all of these add-on files unconditionally
and told users to use INSTALL_MASK if they wanted them not to be on
their systems.

However, I feel that we should update our practice now since we have these
features available to us and to our users.

In my previous thread about zsh, it was suggested that I could use the
zsh-completion use flag to control zsh-completion installation, and not
rdepend on zsh. This is now how pybugz is set up.

The suggestion was made because zsh is not common, so let's try to
define what common means to get an idea of when use flags can be used
like this.

I think the most fair/objective way to define whether something is
common is whether or not it is part of the base requirement, like bash,
or whether it is the default provided by a virtual. In either of those
cases, I would say it makes sense to install add-on files that the
program might use unconditionally.

Thoughts?

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: add-on files handling improvements

[Matthias Maier]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]


> Thoughts?

One point in favor of the current practice (installing add-on files
unconditionally) is the fact that you can basically do it for free - you
neither have to depend on additional packages, nor is the presence of
the add-on files a penalty in download time or storage.

Further, a lot of packages install _small_ additional files
unconditionally - let it be examples, minimal documentation, example
configurations - unconditionally. And this is done with the very same
reasoning as above; the penalty is small enough to not warrant the
introduction of a use flag.

Personally, I would not introduce yet another set of global use flags
just for the sake of controlling everything with use flags. The
complexity this introduces (naming choice - enforcing the rule -
ensuring uniformity) is worse than the current behavior of just
installing small add-on files.

Best,
Matthias

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 820 bytes --]

Re: [gentoo-dev] rfc: add-on files handling improvements

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 2005 bytes --]

On Mon, Mar 30, 2015 at 12:11:34AM +0200, Matthias Maier wrote:
> 
> > Thoughts?
> 
> One point in favor of the current practice (installing add-on files
> unconditionally) is the fact that you can basically do it for free - you
> neither have to depend on additional packages, nor is the presence of
> the add-on files a penalty in download time or storage.

The add-on files i'm talking about are not specifically used by the
packages that install them. They are add-ons that hook the packages into
external functions, such as shell completions, logrotate files, xinetd
configurations, etc.

The penalty is cruft on the users's systems when they don't use the
programs that read these files, such as app-admin/logrotate,
sys-apps/xinetd, etc.

All I'm saying is if the application that processes these small
files is not installed, I don't think the files should be either, unless
the upstream build system forces it; I guess we shouldn't try to do
anything about that.

> Further, a lot of packages install _small_ additional files
> unconditionally - let it be examples, minimal documentation, example
> configurations - unconditionally. And this is done with the very same
> reasoning as above; the penalty is small enough to not warrant the
> introduction of a use flag.
 
 The small files I'm talking about, technically, aren't used by the
 package that installs them; they are used by another package, which may
 or may not be installed, independently from the package that installs
 the small files.

> Personally, I would not introduce yet another set of global use flags
> just for the sake of controlling everything with use flags. The
> complexity this introduces (naming choice - enforcing the rule -
> ensuring uniformity) is worse than the current behavior of just
> installing small add-on files.

Actually I'm not talking about introducing more use flags; the flags I'm
interested in adding this functionality to are already there.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: add-on files handling improvements

[Rich Freeman]

On Sun, Mar 29, 2015 at 7:28 PM, William Hubbs <williamh@gentoo.org> wrote:
> On Mon, Mar 30, 2015 at 12:11:34AM +0200, Matthias Maier wrote:
>>
>> > Thoughts?
>>
>> One point in favor of the current practice (installing add-on files
>> unconditionally) is the fact that you can basically do it for free - you
>> neither have to depend on additional packages, nor is the presence of
>> the add-on files a penalty in download time or storage.
>
> The add-on files i'm talking about are not specifically used by the
> packages that install them. They are add-ons that hook the packages into
> external functions, such as shell completions, logrotate files, xinetd
> configurations, etc.
>
> The penalty is cruft on the users's systems when they don't use the
> programs that read these files, such as app-admin/logrotate,
> sys-apps/xinetd, etc.

The problem is that if you don't install this stuff up-front you end
up rebuilding half your system to install it later.

I think the cleanest solution is to just install this stuff
unconditionally, and users who really object to having it around can
use INSTALL_MASK.  It is just a couple of inodes, on a distro that by
default sticks a dozen inodes for every package in the repository on
their root partition.

You suggested that the past policy was due to the lack of
--changed-use in portage at the time, but this is not the case.  That
option has been around for a very long time.  Maybe if it first came
up for manpages or docs that might have been the case, but certainly
not in more recent cases.

Not everybody uses logrotate, xinetd, cron.d, and so on.  It still
makes sense to just install the files, since they passively sit there
doing nothing in those cases.


-- 
Rich

Re: [gentoo-dev] rfc: add-on files handling improvements

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 2087 bytes --]

On Sun, Mar 29, 2015 at 07:49:32PM -0400, Rich Freeman wrote:
> On Sun, Mar 29, 2015 at 7:28 PM, William Hubbs <williamh@gentoo.org> wrote:
> > On Mon, Mar 30, 2015 at 12:11:34AM +0200, Matthias Maier wrote:
> >>
> >> > Thoughts?
> >>
> >> One point in favor of the current practice (installing add-on files
> >> unconditionally) is the fact that you can basically do it for free - you
> >> neither have to depend on additional packages, nor is the presence of
> >> the add-on files a penalty in download time or storage.
> >
> > The add-on files i'm talking about are not specifically used by the
> > packages that install them. They are add-ons that hook the packages into
> > external functions, such as shell completions, logrotate files, xinetd
> > configurations, etc.
> >
> > The penalty is cruft on the users's systems when they don't use the
> > programs that read these files, such as app-admin/logrotate,
> > sys-apps/xinetd, etc.
> 
> The problem is that if you don't install this stuff up-front you end
> up rebuilding half your system to install it later.
> 
> I think the cleanest solution is to just install this stuff
> unconditionally, and users who really object to having it around can
> use INSTALL_MASK.  It is just a couple of inodes, on a distro that by
> default sticks a dozen inodes for every package in the repository on
> their root partition.

*snip*

> Not everybody uses logrotate, xinetd, cron.d, and so on.  It still
> makes sense to just install the files, since they passively sit there
> doing nothing in those cases.

Rich,

Not everyone uses zsh either, but you just said in the other thread that
it is acceptable to put zsh completions behind a use flag [1], and a
couple of others agreed with us.

So, if we are going to do that for zsh, I'm just wanting to attempt
defining what is common vs what isn't, and no, I don't think we should
bug the council with this every time it comes up about a package.

William

[1]
https://archives.gentoo.org/gentoo-dev/message/d57b96bcfb1a91ee437f39410da00aad

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: add-on files handling improvements

[Rich Freeman]

On Sun, Mar 29, 2015 at 10:14 PM, William Hubbs <williamh@gentoo.org> wrote:
> On Sun, Mar 29, 2015 at 07:49:32PM -0400, Rich Freeman wrote:
>
>> Not everybody uses logrotate, xinetd, cron.d, and so on.  It still
>> makes sense to just install the files, since they passively sit there
>> doing nothing in those cases.
>
> Rich,
>
> Not everyone uses zsh either, but you just said in the other thread that
> it is acceptable to put zsh completions behind a use flag [1], and a
> couple of others agreed with us.

That wasn't what I meant at all.

I used the words "second choice" to describe this scenario.  My first
preference was to do what I quoted, which is to install the files
unconditionally as is done with bash.  If we did control it with a USE
flag then we shouldn't RDEPEND on zsh.

Apologies if my email was confusing.

> [1]
> https://archives.gentoo.org/gentoo-dev/message/d57b96bcfb1a91ee437f39410da00aad

-- 
Rich

Re: [gentoo-dev] rfc: add-on files handling improvements

[Thomas D.]

Hi,

William Hubbs wrote:
> I believe, back in the day we started this practice, portage did not
> support --newuse or --changed-use, so there was no way to only update
> packages that had changed or new use flags. In that situation, I
> understand why we installed all of these add-on files unconditionally
> and told users to use INSTALL_MASK if they wanted them not to be on
> their systems.
> 
> However, I feel that we should update our practice now since we have these
> features available to us and to our users.
> 
> In my previous thread about zsh, it was suggested that I could use the
> zsh-completion use flag to control zsh-completion installation, and not
> rdepend on zsh. This is now how pybugz is set up.

Are we talking about an actual problem?

Are these "add-on files" causing real problems?

How many "add-on files" on an average system are really useless (=cruft
files) for most people and how much disk space do they take up?


Do you remember the discussion about "USE flag per init system"? It was
decided to drop the systemd USE flag if it is only controlling the
installation of systemd service files and we didn't want to introduce a
USE flag per init system because it doesn't scale.

Also, image you are on OpenRC and decide to switch to systemd. If we
wouldn't install the service files on every system the user would have
to re-emerge his/her whole system to switch.

Following your argumentation we would add an exception for init systems.

So which add-on files are left? Logrotate! Doesn't the same argument
against USE flags for each init system applies to things like logrotate,
too? If not, at least the same argument "if you switch your init system"
applies: If you decide to start using logrotate you would have to
re-emerge your packages just for a 1kb file...

Add another exception for logrotate files? :)

I guess that's not what you want. But do you see the problem? How would
you decide for which files you want to add an exception?

Do you want to discuss with cron users if their cronjobs are add-on
files or not?

Some packages are providing files for logwatch. I don't expect that many
desktop user will use logwatch. But do you want to start a discussion
with non-desktop users if it is worth to re-emerge a whole package for
1kb additional files?

Coming back to my first question: Why do you want to change the current
situation? Are these "add-on files" causing any problems nowadays?

Introducing another USE flag to control what INSTALL_MASK already do
doesn't sound like a good way to go for me.

But maybe I am not aware of a real problem you have with these "add-on
files"...


-Thomas