[gentoo-dev] RFC: Package Manager Specification: configuration protection

[gentoo-dev] RFC: Package Manager Specification: configuration protection

[Ciaran McCreesh]

[-- Attachment #1.1: Type: text/plain, Size: 854 bytes --]

A while back it was agreed that it would be a good idea to standardise
certain aspects of package manager behaviour. We thought it'd be a good
idea to start with something easy so that we can iron out any kinks in
the process... So...

Attached is a first draft of an attempt at standardising how
configuration protection is handled. Although it's not strictly
speaking a core part of the ebuild API, it's none the less something
that should probably be handled consistently.

Comments both on the nature and the specifics of the specification
would be welcomed. In particular, I'd like to know if people think
we're mandating the appropriate degree of specificity and whether we're
providing sufficient generality to avoid overly restricting innovation.

Yours lovingly,
-- 
Ciaran McCreesh
Mail            : ciaranm at ciaranm.org


[-- Attachment #1.2: PMS-Configuration-Protection.txt --]
[-- Type: text/plain, Size: 3694 bytes --]

==================================================
Package Manager Standard: Configuration Protection
==================================================

Abstract
========

This document defines how a Package Manager should handle the filesystem aspect
of configuration protection.

Overview
========

Configuration protection is used by a Package Manager to avoid overwriting or
removing important configuration files (e.g. ``/etc/fstab``) when updating or
uninstalling a package. Rather than overwriting these files in the merge phase,
the file to be installed is renamed according to a defined set of rules; when
unmerging a package, these files are not removed.

File Merging Rules
==================

When merging a file to a protected location:

* If no existing file with the intended target name exists, or if the existing
  file has identical content to the file to be installed, the file to be installed
  is installed as normal.

* Otherwise, pretend that the filename of the file to be installed is
  ``._cfg0000_name``, where ``name`` is the real name. If no file with this name
  exists, or if the existing file with this name has identical content to the file
  to be installed, the file to be installed is merged with this new name.

* Otherwise, try again with ``._cfg0001_name``, then ``._cfg0002_name`` and so
  on (base ten is used for the number part) until a usable filename is found.

* Behaviour is undefined in the highly unlikely circumstance that ``9999`` is
  reached.

Configuration protection does not apply to non-files.

File Unmerging Rules
====================

Files in a protected location should not be unmerged.

Protected Locations
===================

Protected locations are determined by the ``CONFIG_PROTECT`` environment
variable, which is defined in the profiles and which may be augmented or
overridden by the current environment and user configuration files. This
variable contains a space separated list of values which are matched against the
beginning of a full file path and name of files to be installed.

Any item inside ``CONFIG_PROTECT`` that starts with a minus symbol
instead removes any previous item with the value following the minus from the
list. The special value ``-*`` can be used to remove *all* previous values.

The behaviour of special wildcard characters inside items (e.g.
``/foo?bar/*monkey*/baz``) is undefined.

The ``CONFIG_PROTECT_MASK`` environment variable, which has the same format and
origin as ``CONFIG_PROTECT``, can be used to unprotect locations. If any of its
items match the beginning of the full file path and name of a file to be
installed, that file is *not* considered protected.

For example, if ``CONFIG_PROTECT`` is ``/etc /usr/share/X11/xkb`` and
``CONFIG_PROTECT_MASK`` is ``/etc/init.d``, files to be installed to these
locations are considered protected:

* ``/etc/fstab``
* ``/etc/vim/vimrc``
* ``/etcetera``
* ``/usr/share/X11/xkb/keycodes/ibm``

And these are considered not protected:

* ``/usr/share/foo/foo.txt``
* ``/etc/init.d/sshd``
* ``/etc/init.donkey``
* ``/usr/local/etc/fstab``

If ``ROOT`` is in use, configuration protection should be applied to the
``ROOT`` directory rather than the normal filesystem root. For example, if
``ROOT`` is ``/image`` and ``CONFIG_PROTECT`` is ``/etc``, then ``/image/etc``
should be protected.

Integration with External Tools
===============================

The package manager should provide a way for external tools to obtain a list of
locations where protected files requiring action may be found.

.. vim: set tw=80 ft=glep spell spelllang=en : ..

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Chris White]

[-- Attachment #1: Type: text/plain, Size: 786 bytes --]

On Monday 11 September 2006 15:22, Ciaran McCreesh wrote:
> * Otherwise, try again with ``._cfg0001_name``, then ``._cfg0002_name`` and 
> so on (base ten is used for the number part) until a usable filename is
> found. 

For what purpose are the older cfg[number]_name files kept around?  I ask 
because I would anticipate the default behavior for replacing configuration 
files with their pending updates to be picking the newest update.  That said, 
the previous versions would not serve a purpose, or is there something I 
don't see?

-- 
Chris White
Gentoo Developer aka:
ChrisWhite		cpw
ChrisWhite|Work	WhiteChocolate
VanillaWhite		Whitey
WhiteLight		WhiteCheese
WhiteSugar		WhiteButter
WhiteWall		WhiteLemon
WhiteApple		WhiteBlanket
WhiteEnergy		WhiteWhite

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1305 bytes --]

On Mon, 11 Sep 2006 16:02:53 -0700 Chris White <chriswhite@gentoo.org>
wrote:
| On Monday 11 September 2006 15:22, Ciaran McCreesh wrote:
| > * Otherwise, try again with ``._cfg0001_name``, then
| > ``._cfg0002_name`` and so on (base ten is used for the number part)
| > until a usable filename is found. 
| 
| For what purpose are the older cfg[number]_name files kept around?  I
| ask because I would anticipate the default behavior for replacing
| configuration files with their pending updates to be picking the
| newest update.  That said, the previous versions would not serve a
| purpose, or is there something I don't see?

Existing tools ask the user which file they want to use when there's
more than one. It's possible that this is more useful behaviour,
especially if, say, someone is upgrading and downgrading the same
package repeatedly for testing purposes.

The purpose of these specifications isn't to change behaviour, except
for small things where obvious and clear bugs or deficiencies are found
(which I don't think is the case here). Rather, they're to document and
clarify what current behaviour should be considered reliable rather
than merely a coincidence of how Portage happens to work.

-- 
Ciaran McCreesh
Mail            : ciaranm at ciaranm.org


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1467 bytes --]

On Tue, Sep 12, 2006 at 12:15:34AM +0100, Ciaran McCreesh wrote:
> On Mon, 11 Sep 2006 16:02:53 -0700 Chris White <chriswhite@gentoo.org>
> wrote:
> | On Monday 11 September 2006 15:22, Ciaran McCreesh wrote:
> | > * Otherwise, try again with ``._cfg0001_name``, then
> | > ``._cfg0002_name`` and so on (base ten is used for the number part)
> | > until a usable filename is found. 
> | 
> | For what purpose are the older cfg[number]_name files kept around?  I
> | ask because I would anticipate the default behavior for replacing
> | configuration files with their pending updates to be picking the
> | newest update.  That said, the previous versions would not serve a
> | purpose, or is there something I don't see?
> 
> Existing tools ask the user which file they want to use when there's
> more than one. It's possible that this is more useful behaviour,
> especially if, say, someone is upgrading and downgrading the same
> package repeatedly for testing purposes.

Personally, think the behaviour should be that it ensures a copy of 
the file winds up config protected; in other words, if a preexisting 
copy is already sitting in the config protected file stack 
(essentially), don't see any point to adding yet another file.

Renaming to max + 1, or reusing the max if the match is the max is the 
match.

Pkgcore doesn't *quite* do this (reuses any match), but shifting the 
file in the 'stack' makes more sense imo.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1585 bytes --]

On Mon, Sep 11, 2006 at 04:02:53PM -0700, Chris White wrote:
> On Monday 11 September 2006 15:22, Ciaran McCreesh wrote:
> > * Otherwise, try again with ``._cfg0001_name``, then ``._cfg0002_name`` and 
> > so on (base ten is used for the number part) until a usable filename is
> > found. 
> For what purpose are the older cfg[number]_name files kept around?  I ask 
> because I would anticipate the default behavior for replacing configuration 
> files with their pending updates to be picking the newest update.  That said, 
> the previous versions would not serve a purpose, or is there something I 
> don't see?
Consider a case as follows:
1. package foo provides /etc/bar - the contents of the file differ depending on the USE flags.
2. an old version of foo is already installed, with USE='a'
3. user does "USE='a b' emerge foo" - /etc/._cfg0000_bar is created.
4. user then realizes he actually wanted USE=c as well, so does "USE='a b c' emerge foo" - this provides /etc/._cfg0001_bar.

You now have 3 files, non-identical:
/etc/bar
/etc/._cfg0000_bar
/etc/._cfg0001_bar

The user now runs etc-update or dispatch-conf or whatever tool they use
to manage their configurations.

The 'try again rule to find a usable filename' is specifically intended
for these cases where the configuration management is NOT used between
more than two changes of a configuration file (Ideally it should be, but
that's a different discussion).

-- 
Robin Hugh Johnson
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Michael Cummings]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ciaran McCreesh wrote:
> * If no existing file with the intended target name exists, or if the existing
>   file has identical content to the file to be installed, the file to be installed
>   is installed as normal.

Just one poor dev's opinion, but I prefer it when these files are
created as .sample's or examples. Invariably there is some tweaking that
needs to be done to the file, even when it's the initial version of the
file. This also "forces" the user to get acquainted with the config file
in question (samba and make.conf files come to mind). Of course,
tweaking something like etc-update (or equiv tool) to be able to compare
the sample/example file to the live one would be nice, but then I've
just talked myself in a circle renaming the ._cfg* files, haven't I?

- --

- -----o()o----------------------------------------------
Michael Cummings   |    #gentoo-dev, #gentoo-perl
Gentoo Perl Dev    |    on irc.freenode.net
Gentoo/SPARC
Gentoo/AMD64
GPG: 0543 6FA3 5F82 3A76 3BF7  8323 AB5C ED4E 9E7F 4E2E
- -----o()o----------------------------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.4 (GNU/Linux)

iD8DBQFFBf+kq1ztTp5/Ti4RAp0mAKCzpZGqnF6cl/hm+kklR4PxNj28AQCfT45Q
yfPdoOSEqMvKkhEkGFTJWmY=
=qoX4
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Peter Volkov (pva)]

[-- Attachment #1: Type: text/plain, Size: 992 bytes --]

On 2006-09-11 at 20:30 -0400, Michael Cummings wrote:
> Ciaran McCreesh wrote:
> > * If no existing file with the intended target name exists, or if the existing
> >   file has identical content to the file to be installed, the file to be installed
> >   is installed as normal.
> 
> Just one poor dev's opinion, but I prefer it when these files are
> created as .sample's or examples.

.sample or .example files are not related with configuration protection
and as it was already decided on such files should go
into /usr/share/doc.

> Invariably there is some tweaking that needs to be done to the file,
> even when it's the initial version of the file.

And this breaks General guidelines in gentoo developer handbook:
"Your package, when complete and unmasked, is supposed to "just work"
for the end-user. Tweaking the installed product to get it to work
should be optional; thus you need to install the package with reasonable
default settings."

Regards,
Peter.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Simon Stelling]

> Protected Locations
> ===================
> 
> Protected locations are determined by the ``CONFIG_PROTECT`` environment
> variable, which is defined in the profiles and which may be augmented or
> overridden by the current environment and user configuration files. This
> variable contains a space separated list of values which are matched against the
> beginning of a full file path and name of files to be installed.

"which are matched against the beginning of a full file path" would mean
that e.g. CONFIG_PROTECT="/etc/foo" would protect the following:

/etc/foobar/doh
/etc/foo
/etc/foobaz

.. or did I misunderstand something here? I don't know whether that is
the current behaviour of portage, but IMO it certainly shouldn't be. It
should rather be

/etc/foo (file)
or, if /etc/foo is a dir:
/etc/foo/*

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 developer
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1691 bytes --]

On Tue, 12 Sep 2006 10:19:40 +0200 Simon Stelling <blubb@gentoo.org>
wrote:
| > Protected Locations
| > ===================
| > 
| > Protected locations are determined by the ``CONFIG_PROTECT``
| > environment variable, which is defined in the profiles and which
| > may be augmented or overridden by the current environment and user
| > configuration files. This variable contains a space separated list
| > of values which are matched against the beginning of a full file
| > path and name of files to be installed.
| 
| "which are matched against the beginning of a full file path" would
| mean that e.g. CONFIG_PROTECT="/etc/foo" would protect the following:
| 
| /etc/foobar/doh
| /etc/foo
| /etc/foobaz
| 
| .. or did I misunderstand something here? I don't know whether that is
| the current behaviour of portage, but IMO it certainly shouldn't be.
| It should rather be
| 
| /etc/foo (file)
| or, if /etc/foo is a dir:
| /etc/foo/*

Mm. I had a play with this. I'd like someone else to do independent
tests, because I'm seeing something weird here. But it looks like
Portage's current behaviour is:

with CONFIG_PROTECT="/foo":
* if /foo is a file, it's not protected
* if /foo is a directory, its contents (including subdirectories) are
protected
* /foofoo (file) is not protected
* /foobar/baz is not protected

and weirdly, with CONFIG_PROTECT="/foo/"
* if /foo/ is a directory, its contents are protected during unmerge
but not during merge

All of this is rather weird, and doesn't match up to what I've been
told by Portage people that Portage is supposed to do...

-- 
Ciaran McCreesh
Mail            : ciaranm at ciaranm.org


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Zac Medico]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ciaran McCreesh wrote:
> On Tue, 12 Sep 2006 10:19:40 +0200 Simon Stelling <blubb@gentoo.org>
> wrote:
> | > Protected Locations
> | > ===================
> | > 
> | > Protected locations are determined by the ``CONFIG_PROTECT``
> | > environment variable, which is defined in the profiles and which
> | > may be augmented or overridden by the current environment and user
> | > configuration files. This variable contains a space separated list
> | > of values which are matched against the beginning of a full file
> | > path and name of files to be installed.
> | 
> | "which are matched against the beginning of a full file path" would
> | mean that e.g. CONFIG_PROTECT="/etc/foo" would protect the following:
> | 
> | /etc/foobar/doh
> | /etc/foo
> | /etc/foobaz
> | 
> | .. or did I misunderstand something here? I don't know whether that is
> | the current behaviour of portage, but IMO it certainly shouldn't be.
> | It should rather be
> | 
> | /etc/foo (file)
> | or, if /etc/foo is a dir:
> | /etc/foo/*
> 
> Mm. I had a play with this. I'd like someone else to do independent
> tests, because I'm seeing something weird here. But it looks like
> Portage's current behaviour is:
> 
> with CONFIG_PROTECT="/foo":
> * if /foo is a file, it's not protected
> * if /foo is a directory, its contents (including subdirectories) are
> protected
> * /foofoo (file) is not protected
> * /foobar/baz is not protected
> 
> and weirdly, with CONFIG_PROTECT="/foo/"
> * if /foo/ is a directory, its contents are protected during unmerge
> but not during merge
> 
> All of this is rather weird, and doesn't match up to what I've been
> told by Portage people that Portage is supposed to do...
> 

When I've looked at the relevant code, it's given me the impression
that it could use some improvement.  Frankly, I'm not surprised that
portage's CONFIG_PROTECT handling doesn't behave quite like one
would hope/expect in the cases mentioned above.  Anyway, I'd like to
fix it so that it behaves better in all of those cases.  Note that
bug 14321 already exists for that specific case that Simon has
mentioned.

Zac
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFFBvAU/ejvha5XGaMRAidrAJ9jQfHIHuDLomohU0JURE9f4fwPggCgvmhb
hnnzooKZCwmdDl4mG8wsqIA=
=J3/L
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Zac Medico]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ciaran McCreesh wrote:
> On Tue, 12 Sep 2006 10:19:40 +0200 Simon Stelling <blubb@gentoo.org>
> wrote:
> | > Protected Locations
> | > ===================
> | > 
> | > Protected locations are determined by the ``CONFIG_PROTECT``
> | > environment variable, which is defined in the profiles and which
> | > may be augmented or overridden by the current environment and user
> | > configuration files. This variable contains a space separated list
> | > of values which are matched against the beginning of a full file
> | > path and name of files to be installed.
> | 
> | "which are matched against the beginning of a full file path" would
> | mean that e.g. CONFIG_PROTECT="/etc/foo" would protect the following:
> | 
> | /etc/foobar/doh
> | /etc/foo
> | /etc/foobaz
> | 
> | .. or did I misunderstand something here? I don't know whether that is
> | the current behaviour of portage, but IMO it certainly shouldn't be.
> | It should rather be
> | 
> | /etc/foo (file)
> | or, if /etc/foo is a dir:
> | /etc/foo/*
> 
> Mm. I had a play with this. I'd like someone else to do independent
> tests, because I'm seeing something weird here. But it looks like
> Portage's current behaviour is:
> 
> with CONFIG_PROTECT="/foo":
> * if /foo is a file, it's not protected
> * if /foo is a directory, its contents (including subdirectories) are
> protected
> * /foofoo (file) is not protected
> * /foobar/baz is not protected
> 
> and weirdly, with CONFIG_PROTECT="/foo/"
> * if /foo/ is a directory, its contents are protected during unmerge
> but not during merge
> 
> All of this is rather weird, and doesn't match up to what I've been
> told by Portage people that Portage is supposed to do...
> 

I've attached to bug 14321 [1] a patch that I believe implements the
CONFIG_PROTECT behavior that most people would expect from portage.
The differences from previous behavior are as follows:

1) Allows files (not just directories) in CONFIG_PROTECT and
CONFIG_PROTECT_MASK.

2) Properly accounts for an optional trailing slash on directory paths.

3) Prevents /etc/foo from matching /etc/foobaz or /etc/foobaz/bar.

Testing of the patch (against portage-2.1.1) would be appreciated.

Zac

[1] http://bugs.gentoo.org/show_bug.cgi?id=14321#c15

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFFBzhF/ejvha5XGaMRApxqAJ0XcfuqkfNn8L68HLRRynSyXf9grgCcCgok
CNysJhEHA5mUvX84vmB8PU0=
=KPm0
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 357 bytes --]

On Tue, 12 Sep 2006 15:44:22 -0700 Zac Medico <zmedico@gentoo.org>
wrote:
| 3) Prevents /etc/foo from matching /etc/foobaz or /etc/foobaz/bar.

Is this really desired behaviour?

Once we decide that, I'll have a testsuite we can use. It's written for
Paludis, but easily portable.

-- 
Ciaran McCreesh
Mail            : ciaranm at ciaranm.org


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Zac Medico]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ciaran McCreesh wrote:
> On Tue, 12 Sep 2006 15:44:22 -0700 Zac Medico <zmedico@gentoo.org>
> wrote:
> | 3) Prevents /etc/foo from matching /etc/foobaz or /etc/foobaz/bar.
> 
> Is this really desired behaviour?

In my opinion, it is a desirable change.  I doubt that many people
(if anyone?) will miss that particular matching behavior.  If so,
please speak up now.

Zac

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFFB0la/ejvha5XGaMRAiTgAKDgoV3Vasw2Fk5r2mKy5FlltoCa2gCfXNj1
E/rA0gJXmwiZFZEnrlU7e+Q=
=+zAN
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

On Tuesday 12 September 2006 19:57, Zac Medico wrote:
> Ciaran McCreesh wrote:
> > On Tue, 12 Sep 2006 15:44:22 -0700 Zac Medico <zmedico@gentoo.org>
> >
> > wrote:
> > | 3) Prevents /etc/foo from matching /etc/foobaz or /etc/foobaz/bar.
> >
> > Is this really desired behaviour?
>
> In my opinion, it is a desirable change.  I doubt that many people
> (if anyone?) will miss that particular matching behavior.  If so,
> please speak up now.

i'd take the silence to mean no one likes the idea of /etc/foo 
matching /etc/foobar :)
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Ilya A. Volynets-Evenbakh]

Ciaran McCreesh wrote:
> On Tue, 12 Sep 2006 15:44:22 -0700 Zac Medico <zmedico@gentoo.org>
> wrote:
> | 3) Prevents /etc/foo from matching /etc/foobaz or /etc/foobaz/bar.
>
> Is this really desired behaviour?
>
> Once we decide that, I'll have a testsuite we can use. It's written for
> Paludis, but easily portable.
>
>   
Yes, it's desired behavior. I can't think of half-measure as being
useful - either
drop it completely, or implement full wildcard support.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Benno Schulenberg]

Ciaran McCreesh wrote:
> * If no existing file with the intended target name exists, or if 
>   the existing file has identical content to the file to be 
>   installed, the file to be installed is installed as normal.

I would much prefer new files to be treated as if replacing an 
existing zero length file.  When something is installed into /etc, 
etc-update should alert me to this, because logically speaking a 
new configuration file is a big configuration change.

Ideally the package manager would unconditionally respect the config 
protection area, and it should be up to tools like etc-update to 
(configurably) automerge new files and identical files, just like 
it can be configured to automerge trivial/comment changes.

Benno
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Daniel Gryniewicz]

[-- Attachment #1: Type: text/plain, Size: 1198 bytes --]

On Wed, 2006-09-13 at 19:47 +0200, Benno Schulenberg wrote:
> Ciaran McCreesh wrote:
> > * If no existing file with the intended target name exists, or if 
> >   the existing file has identical content to the file to be 
> >   installed, the file to be installed is installed as normal.
> 
> I would much prefer new files to be treated as if replacing an 
> existing zero length file.  When something is installed into /etc, 
> etc-update should alert me to this, because logically speaking a 
> new configuration file is a big configuration change.
> 
> Ideally the package manager would unconditionally respect the config 
> protection area, and it should be up to tools like etc-update to 
> (configurably) automerge new files and identical files, just like 
> it can be configured to automerge trivial/comment changes.
> 

I disagree.  If there is a sane default configuration for something
(which is most things), I want it installed, so it works out of the box.
I don't want to have to fiddle with config files to get sshd up and
running.

Obviously, if there's no sane default configuration (samba?), then the
installed configuration shouldn't do anything.

Daniel

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Benno Schulenberg]

Daniel Gryniewicz wrote:
> On Wed, 2006-09-13 at 19:47 +0200, Benno Schulenberg wrote:
> > Ideally the package manager would unconditionally respect the
> > config protection area, and it should be up to tools like
> > etc-update to (configurably) automerge new files and identical
> > files, just like it can be configured to automerge
> > trivial/comment changes.
>
> I disagree.  If there is a sane default configuration for
> something (which is most things), I want it installed, so it
> works out of the box. I don't want to have to fiddle with config
> files to get sshd up and running.

There's no need to fiddle: it will run out of the box right after 
running etc-update.  That is, when this tool is extended to deal 
with new files and configured to automerge them.

Benno
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Thomas de Grenier de Latour]

On Wed, 13 Sep 2006 19:47:12 +0200,
Benno Schulenberg <benno.schulenberg@gmail.com> wrote:

> I would much prefer new files to be treated as if replacing an 
> existing zero length file. 
...
> it should be up to tools like etc-update to (configurably) automerge
> new files

A quick look through my CONFIG_PROTECTed directories shows that, on a
total of ~1000 config files installed by ebuilds, only ~60 may have
affected my system when they were new and have been unconditionnaly
installed.  With such a false-positive rate, i would probably have soon
disabled the etc-update paranoid mode you propose, and i think most
users would have done the same.

I think that protection against harmfull new config files should be
selective to be useful.  It should only affect directories from which
files are blindly sourced by some services you are already running.
There, and only there¹, new config files are unexpected change of your 
existing configuration, and thus lead to unexpected behaviors.

 ¹ Well, ok, that's not exactly true. There is also the case of config
files being moved (a program expecting /etc/foo.conf in one version,
and /etc/foo/foo.conf in the later), things like that.  But imho, in
such cases, documentation (postinst messages, or GLEP 42) is enough,
whereas an anti-new-files-protection wouldn't really help.

The directories i'm thinking of are all this /etc/*.d/: "acpi.d",
"logrotate.d", "pam.d", etc.  There, adding a new file is really
just like appending a new chunk to an existing config file.

Implementation of a special anti-new-file-protection for this critical
directories could be done in at least two ways:
 - a global NEW_CONFIG_PROTECT variable (but i don't think it's would be
a good idea, too hard to maintain given the number of packages / devs
which would have a path to add to the list),
 - an ebuild-specific variable, which would be taken into account by the
contents merging function of the package manager (sure, this variable
should be accessible through aux_get() or alike, ie. not bash-level
only, but part of the ebuild metadatas).


But anyway, sorry for the off-topic Ciaran, i realize that this
discussion is far from being comments on the specs you've written.

--
TGL.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Benno Schulenberg]

Thomas de Grenier de Latour wrote:
> I think that protection against harmfull new config files should
> be selective to be useful.  It should only affect directories
> from which files are blindly sourced by some services you are
> already running. There, and only there¹, new config files are
> unexpected change of your existing configuration, and thus lead
> to unexpected behaviors.
>   [...]
> The directories i'm thinking of are all this /etc/*.d/: "acpi.d",
> "logrotate.d", "pam.d", etc.  There, adding a new file is really
> just like appending a new chunk to an existing config file.

Indeed.  But not only those, also the cron.*ly directories.  The 
specific example I was thinking of is slocate, that isntalls a 
script into /etc/cron.daily, which I don't want there.

> Implementation of a special anti-new-file-protection for this
> critical directories could be done in at least two ways:
>  - a global NEW_CONFIG_PROTECT variable [...]
>  - an ebuild-specific variable, [...]

Or via the config file of etc-update.  Let the package manager 
always create ._cfg* files in the protected areas, and let the 
updater tool figure out from its configuration which files can be 
automerged and which to show diffs for.

Benno
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Harald van Dijk]

On Mon, Sep 11, 2006 at 11:22:11PM +0100, Ciaran McCreesh wrote:
> Comments both on the nature and the specifics of the specification
> would be welcomed. In particular, I'd like to know if people think
> we're mandating the appropriate degree of specificity and whether we're
> providing sufficient generality to avoid overly restricting innovation.

I think this is overly restrictive, actually. It's a good idea to
specify which files and directories will be matched by CONFIG_PROTECT
and _MASK, since that's something ebuilds end up using, but it may be
better to leave the details on how they will be protected up to the
package manager (which can in turn make it configurable for the user).
For one example of what a package manager, if configured to do so,
should in my opinion be allowed to do: automatically remove unmodified
and abandoned configuration files on updates. (This is not the same as
setting CONFIG_PROTECT=-*.) For another, a package manager, if
configured to do so, should in my opinion be allowed to store the config
files on a (possibly local) cvs/svn server in addition to the real
filesystem, avoiding ._cfg* files altogether. Specifying how they will
be protected prevents things like this.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1593 bytes --]

On Thu, 14 Sep 2006 08:51:09 +0200 Harald van Dijk <truedfx@gentoo.org>
wrote:
| On Mon, Sep 11, 2006 at 11:22:11PM +0100, Ciaran McCreesh wrote:
| > Comments both on the nature and the specifics of the specification
| > would be welcomed. In particular, I'd like to know if people think
| > we're mandating the appropriate degree of specificity and whether
| > we're providing sufficient generality to avoid overly restricting
| > innovation.
| 
| I think this is overly restrictive, actually. It's a good idea to
| specify which files and directories will be matched by CONFIG_PROTECT
| and _MASK, since that's something ebuilds end up using, but it may be
| better to leave the details on how they will be protected up to the
| package manager (which can in turn make it configurable for the user).
| For one example of what a package manager, if configured to do so,
| should in my opinion be allowed to do: automatically remove unmodified
| and abandoned configuration files on updates. (This is not the same as
| setting CONFIG_PROTECT=-*.) For another, a package manager, if
| configured to do so, should in my opinion be allowed to store the
| config files on a (possibly local) cvs/svn server in addition to the
| real filesystem, avoiding ._cfg* files altogether. Specifying how
| they will be protected prevents things like this.

Hm, the specification doesn't preclude additional functionality. It
just describes how things should work when normal configuration
protection is in action.

-- 
Ciaran McCreesh
Mail            : ciaranm at ciaranm.org


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Harald van Dijk]

On Fri, Sep 15, 2006 at 07:39:35PM +0100, Ciaran McCreesh wrote:
> On Thu, 14 Sep 2006 08:51:09 +0200 Harald van Dijk <truedfx@gentoo.org>
> wrote:
> | On Mon, Sep 11, 2006 at 11:22:11PM +0100, Ciaran McCreesh wrote:
> | > Comments both on the nature and the specifics of the specification
> | > would be welcomed. In particular, I'd like to know if people think
> | > we're mandating the appropriate degree of specificity and whether
> | > we're providing sufficient generality to avoid overly restricting
> | > innovation.
> | 
> | I think this is overly restrictive, actually. It's a good idea to
> | specify which files and directories will be matched by CONFIG_PROTECT
> | and _MASK, since that's something ebuilds end up using, but it may be
> | better to leave the details on how they will be protected up to the
> | package manager (which can in turn make it configurable for the user).
> | For one example of what a package manager, if configured to do so,
> | should in my opinion be allowed to do: automatically remove unmodified
> | and abandoned configuration files on updates. (This is not the same as
> | setting CONFIG_PROTECT=-*.) For another, a package manager, if
> | configured to do so, should in my opinion be allowed to store the
> | config files on a (possibly local) cvs/svn server in addition to the
> | real filesystem, avoiding ._cfg* files altogether. Specifying how
> | they will be protected prevents things like this.
> 
> Hm, the specification doesn't preclude additional functionality. It
> just describes how things should work when normal configuration
> protection is in action.

Sure, the specification does not prohibit package managers from having
behaviour that can be configured to not match what is specified, but
that isn't the point. The point is that if configured as such, it isn't
a Gentoo package manager anymore.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]

On Mon, Sep 11, 2006 at 11:22:11PM +0100, Ciaran McCreesh wrote:
> Protected locations are determined by the ``CONFIG_PROTECT`` environment
> variable, which is defined in the profiles and which may be augmented or
> overridden by the current environment and user configuration files.

The user env override has no relevance to ebuilds relying on it, thus 
doesn't have any business being mandated imo (it's implementation 
choice, not requirement of ebuilds).

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 711 bytes --]

On Sat, 16 Sep 2006 00:17:21 -0700 Brian Harring <ferringb@gmail.com>
wrote:
| On Mon, Sep 11, 2006 at 11:22:11PM +0100, Ciaran McCreesh wrote:
| > Protected locations are determined by the ``CONFIG_PROTECT``
| > environment variable, which is defined in the profiles and which
| > may be augmented or overridden by the current environment and user
| > configuration files.
| 
| The user env override has no relevance to ebuilds relying on it, thus 
| doesn't have any business being mandated imo (it's implementation 
| choice, not requirement of ebuilds).

Hrm, so installing env.d files that set CONFIG_PROTECT isn't reliable?

-- 
Ciaran McCreesh
Mail            : ciaranm at ciaranm.org


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: Package Manager Specification: configuration protection

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1238 bytes --]

On Sat, Sep 16, 2006 at 11:02:06PM +0100, Ciaran McCreesh wrote:
> On Sat, 16 Sep 2006 00:17:21 -0700 Brian Harring <ferringb@gmail.com>
> wrote:
> | On Mon, Sep 11, 2006 at 11:22:11PM +0100, Ciaran McCreesh wrote:
> | > Protected locations are determined by the ``CONFIG_PROTECT``
> | > environment variable, which is defined in the profiles and which
> | > may be augmented or overridden by the current environment and user
> | > configuration files.
> | 
> | The user env override has no relevance to ebuilds relying on it, thus 
> | doesn't have any business being mandated imo (it's implementation 
> | choice, not requirement of ebuilds).
> 
> Hrm, so installing env.d files that set CONFIG_PROTECT isn't reliable?

Suspect you missed my point-

"Overriden by the current environment", aka the users environment 
at the time of executing $PKG_MANAGER).

To build configurationm, portage stacks a bunch of mappings- the last 
one is the environment the script was ran in.  In other words,

CONFIG_PROTECT='' emerge blah # does disable CONFIG_PROTECT

Portage allows CONFIG_PROTECT from the user env to override; that 
doesn't mean it's required for CONFIG_PROTECT support, which was the 
point.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]