[gentoo-portage-dev] About "system" and "world"

[gentoo-portage-dev] About "system" and "world"

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

So, what do people think about removing (some) of the special treatment
for the "system" and "world" targets?
Mainly I'm interested in removing the "selective" parameter that's
currently enabled for them (so for example without that parameter
`emerge world` would default to remerging packages, unless --update or
--noreplace are specified). That change has already been requested a
few times in the past by users, but OTOH it could probably upset people
who don't use --update with world/system.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] About "system" and "world"

[Ned Ludd]

On Sun, 2007-10-21 at 13:01 +0200, Marius Mauch wrote:
> So, what do people think about removing (some) of the special treatment
> for the "system" and "world" targets?
> Mainly I'm interested in removing the "selective" parameter that's
> currently enabled for them (so for example without that parameter
> `emerge world` would default to remerging packages, unless --update or
> --noreplace are specified). That change has already been requested a
> few times in the past by users, but OTOH it could probably upset people
> who don't use --update with world/system.

What would such a disruptive change really gain us? I personally feel 
our users need consistency from Gentoo. If they grew up doing 
'emerge world' and have come to expect that behavior and all of the 
sudden we change behavior on them.. Yeah I can see how ppl would get 
upset. Perhaps a less intrusive way would be to introduce another flag 
to get the specified behavior you are after.

-- 
Ned Ludd <solar@gentoo.org>

-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] About "system" and "world"

[Marius Mauch]

On Sun, 21 Oct 2007 05:23:45 -0700
Ned Ludd <solar@gentoo.org> wrote:

> On Sun, 2007-10-21 at 13:01 +0200, Marius Mauch wrote:
> > So, what do people think about removing (some) of the special
> > treatment for the "system" and "world" targets?
> > Mainly I'm interested in removing the "selective" parameter that's
> > currently enabled for them (so for example without that parameter
> > `emerge world` would default to remerging packages, unless --update
> > or --noreplace are specified). That change has already been
> > requested a few times in the past by users, but OTOH it could
> > probably upset people who don't use --update with world/system.
> 
> What would such a disruptive change really gain us?

The goal is to make package sets behave in a consistent way.

> I personally feel our users need consistency from Gentoo. If they 
> grew up doing 'emerge world' and have come to expect that behavior 
> and all of the sudden we change behavior on them.. Yeah I can see 
> how ppl would get upset.

As do I, which is why I haven't simply changed it.

> Perhaps a less intrusive way would be to introduce another
> flag to get the specified behavior you are after.

Well, the primary goal is to make all sets behave in a consistent way.
And some sets have the explicit purpose to rebuild stuff, so making
sets "selective" by default also has issues.
The proposed change would also make sets behave in the same way as
packages which is IMO another benefit.
But as I said, I can see that it could upset people.

One possible solution that I've thought about would be to remove the
hardcoded "selective" parameter and let the set configuration determine
if a set is selective or not (with missing = "false"), and then enable
it for world and system in our default config, e.g.

[world]
class = portage.sets.files.WorldSet
selective = True

and extend the --rebuild option with new arguments "always" and "never".
Don't really like the additional complexity though (and I haven't
checked how much work it would be).

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] About "system" and "world"

[Ned Ludd]

On Sun, 2007-10-21 at 15:12 +0200, Marius Mauch wrote:
> On Sun, 21 Oct 2007 05:23:45 -0700
> Ned Ludd <solar@gentoo.org> wrote:
> 
> > On Sun, 2007-10-21 at 13:01 +0200, Marius Mauch wrote:
> > > So, what do people think about removing (some) of the special
> > > treatment for the "system" and "world" targets?
> > > Mainly I'm interested in removing the "selective" parameter that's
> > > currently enabled for them (so for example without that parameter
> > > `emerge world` would default to remerging packages, unless --update
> > > or --noreplace are specified). That change has already been
> > > requested a few times in the past by users, but OTOH it could
> > > probably upset people who don't use --update with world/system.
> > 
> > What would such a disruptive change really gain us?
> 
> The goal is to make package sets behave in a consistent way.
> 
> > I personally feel our users need consistency from Gentoo. If they 
> > grew up doing 'emerge world' and have come to expect that behavior 
> > and all of the sudden we change behavior on them.. Yeah I can see 
> > how ppl would get upset.
> 
> As do I, which is why I haven't simply changed it.
> 
> > Perhaps a less intrusive way would be to introduce another
> > flag to get the specified behavior you are after.
> 
> Well, the primary goal is to make all sets behave in a consistent way.
> And some sets have the explicit purpose to rebuild stuff, so making
> sets "selective" by default also has issues.
> The proposed change would also make sets behave in the same way as
> packages which is IMO another benefit.
> But as I said, I can see that it could upset people.
> 
> One possible solution that I've thought about would be to remove the
> hardcoded "selective" parameter and let the set configuration determine
> if a set is selective or not (with missing = "false"), and then enable
> it for world and system in our default config, e.g.
> 
> [world]
> class = portage.sets.files.WorldSet
> selective = True
> 
> and extend the --rebuild option with new arguments "always" and "never".
> Don't really like the additional complexity though (and I haven't
> checked how much work it would be).


To me this really sounds like it's -e world but just with the world
file. Literally as if you were to do an
emerge -p $(</var/lib/portage/world)
Assuming that is what you intend for it to behave like then how about 
using -E for this behavior? Everybody would profit and nobody would 
become disgruntled when all of the sudden emerge started spiking 
the power bills. :)

-- 
Ned Ludd <solar@gentoo.org>

-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] About "system" and "world"

[Zac Medico]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marius Mauch wrote:
> One possible solution that I've thought about would be to remove the
> hardcoded "selective" parameter and let the set configuration determine
> if a set is selective or not (with missing = "false"), and then enable
> it for world and system in our default config, e.g.
> 
> [world]
> class = portage.sets.files.WorldSet
> selective = True
> 
> and extend the --rebuild option with new arguments "always" and "never".
> Don't really like the additional complexity though (and I haven't
> checked how much work it would be).

That seems like a reasonable solution to me. I've been working on
cleaning up the relevant code and things like that should get easier
to implement as I make more progress.

What about atoms that are greedy wrt installed SLOTs? Maybe there
should be a set config option for that too. There are currently some
special cases in depgraph.xcreate() and action_depclean() that
implement greedy atoms for the world set. I've been pondering the
idea of making world non-greedy for slots by default [1], since you
can add slot atoms to the world file to pull in any specific slot(s)
that you want.

Zac

[1] http://archives.gentoo.org/gentoo-user/msg_117468.xml
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)

iD8DBQFHG5kV/ejvha5XGaMRAq0XAJ954zlIao56uHvEtEeCJi++hKg6XQCfVAXf
VWI4c7aOp9ZQAjQpseXwhcA=
=WulO
-----END PGP SIGNATURE-----
-- 
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] Re: About "system" and "world"

[Ryan Hill]

Zac Medico wrote:
> implement greedy atoms for the world set. I've been pondering the
> idea of making world non-greedy for slots by default [1], since you
> can add slot atoms to the world file to pull in any specific slot(s)
> that you want.

That would rock.  Portage always insisting on pulling in the highest
SLOT whether needed or not is one of my biggest pet-peeves.


-- 
                  fonts / wxWindows / gcc-porting / treecleaners
  EFFD 380E 047A 4B51 D2BD  C64F 8AA8 8346 F9A4 0662 (0xF9A40662)

-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: About "system" and "world"

[Zac Medico]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ryan Hill wrote:
> Zac Medico wrote:
>> implement greedy atoms for the world set. I've been pondering the
>> idea of making world non-greedy for slots by default [1], since you
>> can add slot atoms to the world file to pull in any specific slot(s)
>> that you want.
> 
> That would rock.  Portage always insisting on pulling in the highest
> SLOT whether needed or not is one of my biggest pet-peeves.

Well, you can already use SLOT atoms in your world file if you don't
want the highest available. Packages that pull in >=foo are a
different story though. I suppose we can add something like a
- --upgrade=minimal option that prevents pulling in new slots if they
aren't required.

Zac
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)

iD8DBQFHHXiN/ejvha5XGaMRApBQAKDg5r8+l+bcLBEA+JRttstymUoYFgCg8j6M
m5j8mSrC0hfQP+TRGqnrC+0=
=hWGS
-----END PGP SIGNATURE-----
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: About "system" and "world"

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 1267 bytes --]

On Mon, 22 Oct 2007 21:29:02 -0700
Zac Medico <zmedico@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Ryan Hill wrote:
> > Zac Medico wrote:
> >> implement greedy atoms for the world set. I've been pondering the
> >> idea of making world non-greedy for slots by default [1], since you
> >> can add slot atoms to the world file to pull in any specific
> >> slot(s) that you want.
> > 
> > That would rock.  Portage always insisting on pulling in the highest
> > SLOT whether needed or not is one of my biggest pet-peeves.
> 
> Well, you can already use SLOT atoms in your world file if you don't
> want the highest available. Packages that pull in >=foo are a
> different story though. I suppose we can add something like a
> - --upgrade=minimal option that prevents pulling in new slots if they
> aren't required.

Don't restrict it to SLOTs though. "minimal" implies that only upgrades
required to satisfy the depgraph are performed. The described
behavior should be another value, e.g. "no-slot-change".

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] Re: About "system" and "world"

[Thomas de Grenier de Latour]

On 2007/10/23, Marius Mauch <genone@gentoo.org> wrote:

> On Mon, 22 Oct 2007 21:29:02 -0700
> Zac Medico <zmedico@gentoo.org> wrote:
> 
> > Well, you can already use SLOT atoms in your world file if you don't
> > want the highest available. Packages that pull in >=foo are a
> > different story though. I suppose we can add something like a
> > - --upgrade=minimal option that prevents pulling in new slots if
> > they aren't required.
> 
> Don't restrict it to SLOTs though. "minimal" implies that only
> upgrades required to satisfy the depgraph are performed. 

Couldn't this "minimal" behavior just be triggered by lack of the
--upgrade option ("emerge -D <set>")?

Actually, the current --upgrade behavior (with or without --deep)
is a bit weird imho, i would prefer something like this (with foo 
being either a set or an atom):

 * "emerge foo": do the minimum upgrades or new installs required to
get foo satisfied.  Only its direct deps are checked (and direct
deps of the unsatisfied ones, etc.), with the "minimal" heuristic
(upgrade them only if stricly required).

 * "emerge -u foo": do all required new installs and possible upgrades
of foo.  Only its direct deps are checked (and direct deps of the
unsatisfied ones, etc.), with the "minimal" heuristic (upgrade them only
if strictly needed).

 * "emerge -D foo": do the minimal upgrades or new installs required to
get foo satisfied.  Also, check all of its direct and indirect deps,
with the "minimal" heuristic (upgrade only if strictly needed).

 * "emerge -uD foo": do all required new installs and possible upgrades
of foo.  Also, all its direct and indirect deps are checked, and
upgraded where possible.


For those who wonder, and if i'm not mistaken, current behaviors of
this four combinations, expressed in those words, are:

 * "emerge foo": do all required new installs and possible upgrades
of foo.  Only its direct deps are checked (and direct deps of the
unsatisfied ones, etc.), with the "minimal" heuristic (upgrade them 
only if strictly needed).  
  => same as my "emerge -u foo"

 * "emerge -u foo": do all required new installs and possible upgrades
of foo and its direct deps.  Also, check all of their direct and
indirect deps, with the "minimal" heuristic (upgrade them only if
strictly needed).
  => i don't see usefulness of this special case.  Whether some code is
direct or indirect dep is often subject to some maintainer's choices,
and may vary with time (for instance, an internal lib of a big package
being splitted out as a separate dep of this package).

 * "emerge -D foo" and "emerge -uD foo" : do all required new installs
and possible upgrades of foo.  Also, all its direct and indirect deps
are checked, and upgraded where possible.
 => same as my "emerge -uD foo".  But waste of one of the four options
combinations.


-- 
TGL.
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: About "system" and "world"

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 3727 bytes --]

On Tue, 23 Oct 2007 21:11:48 +0200
Thomas de Grenier de Latour <degrenier@easyconnect.fr> wrote:

> On 2007/10/23, Marius Mauch <genone@gentoo.org> wrote:
> 
> > On Mon, 22 Oct 2007 21:29:02 -0700
> > Zac Medico <zmedico@gentoo.org> wrote:
> > 
> > > Well, you can already use SLOT atoms in your world file if you
> > > don't want the highest available. Packages that pull in >=foo are
> > > a different story though. I suppose we can add something like a
> > > - --upgrade=minimal option that prevents pulling in new slots if
> > > they aren't required.
> > 
> > Don't restrict it to SLOTs though. "minimal" implies that only
> > upgrades required to satisfy the depgraph are performed. 
> 
> Couldn't this "minimal" behavior just be triggered by lack of the
> --upgrade option ("emerge -D <set>")?

--upgrade != the current --update (at least that's what I assumed)

> Actually, the current --upgrade behavior (with or without --deep)
> is a bit weird imho, i would prefer something like this (with foo 
> being either a set or an atom):

Yes, --update has a history of being a mess of random behavior.
Personally I'd drop it completely, or alias it to --noreplace.

>  * "emerge foo": do the minimum upgrades or new installs required to
> get foo satisfied.  Only its direct deps are checked (and direct
> deps of the unsatisfied ones, etc.), with the "minimal" heuristic
> (upgrade them only if stricly required).
> 
>  * "emerge -u foo": do all required new installs and possible upgrades
> of foo.  Only its direct deps are checked (and direct deps of the
> unsatisfied ones, etc.), with the "minimal" heuristic (upgrade them
> only if strictly needed).
> 
>  * "emerge -D foo": do the minimal upgrades or new installs required
> to get foo satisfied.  Also, check all of its direct and indirect
> deps, with the "minimal" heuristic (upgrade only if strictly needed).
> 
>  * "emerge -uD foo": do all required new installs and possible
> upgrades of foo.  Also, all its direct and indirect deps are checked,
> and upgraded where possible.
 
Maybe --upgrade as name was an unfortunate choice, --resolver might be
better. What I'd like ideally (as a user) is to replace --update,
--deep, --noreplace and --emptytree with the following options:

--resolver={minimal,noslotchange,leastchange,default}
minimal: select the lowest possible version
noslotchange: like default, but try to avoid slotchanges
leastchange: select the version with the smallest version delta
compared to the installed version, otherwise like default
default: current behavior

--rebuild={never,always,changeduse,newuse,changeddeps,selected}
never: never rebuild packages
always: always rebuild packages (like current --emtpytree)
changeduse: rebuild packages if $USE has changed
newuse: rebuild packages if $USE or $IUSE has changed
changeddeps: rebuild packages if any of its direct dependencies were
updated
selected: rebuild the selected arguments (root nodes)

--deplevel=<n>
check n level of dependencies for updates (with -1 == infinite)
of course all dependencies have to be satisfied, this controls only
what deps should be checked for optional updates

current defaults would be --resolver=default --rebuild=selected
--deplevel=0

--update would be --rebuild=never --deplevel=1
--deep would be --deplevel=-1
--emptytree would be --rebuild=always --deplevel=-1

But that's just me dreaming of a better world, where we don't have
to deal with legacies ;)

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] Re: About "system" and "world"

[Thomas de Grenier de Latour]

On 2007/10/23, Marius Mauch <genone@gentoo.org> wrote:

> --upgrade != the current --update (at least that's what I assumed)

Oops, right, i have overlooked Zac's post, seen '--usomething', and
immediatly thought of '-u' with some optionnal additional parameter.

> What I'd like ideally (as a user) is to replace --update,
> --deep, --noreplace and --emptytree with the following options:
> 
> --resolver={minimal,noslotchange,leastchange,default}
> ...

I would call "noslotchange" (resp. "leastchange") "inslot" (resp.
"closest"), for conciseness, but otherwise, yes, sounds like a
complete and useful set of choices.

> --rebuild={never,always,changeduse,newuse,changeddeps,selected}
> ...

Good idea, i had never realized i would love that too before seeing
it written down :)

> --deplevel=<n>
> ...

I'm not sure i see usefulness of levels which are strictly beetween zero
and infinity (but to emulate the old -u), but why not...

> But that's just me dreaming of a better world, where we don't have
> to deal with legacies ;)

Bah, you're introducing a new options set, and have (i think) ways to
emulate all of the old ones with them, so what is stopping you?  You
would just have to forbid mixes of old style and new style on the same
command line. (Only problem i can see in having the two sets coexisting
for some time is that you may run out of free characters for short
options.)

Well, with a way to define default values of your three ne options in  
a config file ($EMERGE_DEFAULT_OPTS ?), i think i could live happy in
your better world.

-- 
TGL.
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: About "system" and "world"

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

On Wed, 24 Oct 2007 00:03:47 +0200
Thomas de Grenier de Latour <degrenier@easyconnect.fr> wrote:

> Bah, you're introducing a new options set, and have (i think) ways to
> emulate all of the old ones with them, so what is stopping you?  You
> would just have to forbid mixes of old style and new style on the same
> command line.

Mainly the work required to implement it ;) And I'd rather finish
dynoparse first so it can be used here.
Oh, and ideally the mentioned values wouldn't be hardcoded lists, but
come from a pluggable resolver system, but that probably counts as
overengineering.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] About "system" and "world"

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 1013 bytes --]

On Sun, 21 Oct 2007 15:12:31 +0200
Marius Mauch <genone@gentoo.org> wrote:

> Well, the primary goal is to make all sets behave in a consistent way.
> And some sets have the explicit purpose to rebuild stuff, so making
> sets "selective" by default also has issues.
> The proposed change would also make sets behave in the same way as
> packages which is IMO another benefit.

This actually has more consequences: "selective" is a global setting,
you can't enable it just for some arguments. Therefore if packages and
sets are treated differently it requires that they can't be mixed on
the commandline (and if we'd make the setting configurable for each set
then only one set can be used at any time). And right now I can't think
of another reason why that restriction would be necessary.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]