[gentoo-dev] Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[gentoo-dev] Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Kevin]

Pasted from bugzilla.  Please pardon the ugly newline formatting.


I'm a longtime (>10 yrs) Linux admin and I've been using Gentoo for
perhaps 2
years and I'm super impressed with Gentoo, having gotten very annoyed
with the
rpm-based nightmare upgrade situation presented by most of the other
distros,
but one thing I'd really like to see in Gentoo is a way of safely keeping my
Gentoo boxes up to date in an automated way.  I know that may sound
paradoxical
and mutually contradictory.  I realize that production systems should not be
upgraded before trying out the upgrade on a testbed system, but I've
found that
routine cron jobs of emerge world are unsafe because some packages need a
human's attention for upgrading (like apache or postfix when config files
should be left untouched or updated or merged with new config files or some
other issue that needs a human's attention) whereas doing nothing for a long
time (while the portage tree evolves) makes for a box that has been
veritably
left behind, sometimes making it difficult or impossible to upgrade.

I'd like to have the capability of being able to list some packages that
should
never be upgraded automatically (I realize I can do this to some degree
already
with portage), some others that are very unlikely to break from an automated
upgrade and thus should always be upgraded automatically, and some packages
(which may fit in either or both of these categories) that must be
upgraded in
a certain order in order to avoid breaking something and thus, should
probably
be upgraded automatically or (if flagged for preventing automatic
upgrades by
the admin) should be brought to the attention of the admin (say with an
email
to root or something) as needing attention to avoid breakage.

I am asking for this feature after having spent an entire weekend upgrading
various packages by hand, one or a few at a time, after carefully
considering
whether or not it would be safe to upgrade this or that package, and after
having (lazily) not upgraded anything on this production box in a long
time.
The experience has left me rather exhausted (with a sore ass from
sitting down
for so long) and wishing for something better.  One noteworthy experience in
particular is that I found that many packages suffered sandbox violations on
attempted upgrades, and I troubleshot this problem for a long time before it
occurred to me that I might want to upgrade the sandbox package and then try
upgrading these packages.  That solved the sandbox violation problem.
It seems
to me that this was a case where an automated system could have insisted on
upgrading the sandbox package first, before the others.  Perhaps there
should
have been a dependency, so that when I tried to upgrade the ncurses
library, it
automatically pulled in the newer sandbox package as a prerequisite (for
that
is what it turned out to be).

After writing this much, it occurs to me that perhaps the capabilities
that I
describe here may already be in Gentoo/portage in some way that I've yet to
fully discover and/or utilize, but despite having installed many Gentoo
systems
and read the Handbook (and many other Gentoo docs) many times, I've yet
to see
a good write-up on how to do what I describe here.  And perhaps the fact
that
the sandbox package was not a dependency for the ncurses package (and
several
others that also broke during emerge with sandbox violations) was the real
"bug" so to speak, rather than the idea that Gentoo is missing this major
feature that I'm asking for.  I'm really not sure which might be true, but I
just thought I'd ask.

One thing that I'm pretty sure is currently not possible with portage,
however, and that I'd definitely like to see as a part of this idea is a
way of setting thresholds on version numbers of packages in portage such
that the automated upgrade system will only upgrade a package
automatically if the difference in version numbers between the installed
package and the newest available package in portage is greater than some
admin-tunable amount.  For example, I might not want to upgrade emacs or
xemacs just because a new -r number becomes available.  Maybe I don't
want to have such a big package upgraded automatically unless there is a
new major or minor version number.

Thanks again to all the developers who have made Gentoo.  It's a really
terrific distro.


------- Comment #1 From Radek Podgorny 2006-04-24 08:25 PST [reply] -------

Maybe, the packages themselves could be assigned something like a
safe-upgrade-flag...


------- Comment #2 From Jakub Moc 2006-04-26 08:46 PST [reply] -------

Please, take such ideas to portage/devel mailing lists... Bugzilla is
not the
best place to discuss abstract ideas. Thanks.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2070 bytes --]

On Wed, 2006-04-26 at 12:29 -0400, Kevin wrote:
> One thing that I'm pretty sure is currently not possible with portage,
> however, and that I'd definitely like to see as a part of this idea is a
> way of setting thresholds on version numbers of packages in portage such
> that the automated upgrade system will only upgrade a package
> automatically if the difference in version numbers between the installed
> package and the newest available package in portage is greater than some
> admin-tunable amount.  For example, I might not want to upgrade emacs or
> xemacs just because a new -r number becomes available.  Maybe I don't
> want to have such a big package upgraded automatically unless there is a
> new major or minor version number.
> 
> Thanks again to all the developers who have made Gentoo.  It's a really
> terrific distro.

Jakub meant the portage-devel mailing list, not this one.

Anyway, most of this can be done already using /etc/portage files and
some well-written cron scripts.  You can lock down versions of specific
packages quite easily using your own package.mask and package.unmask
files, along with package.keywords.  However, one thing you can *never*
do is assume that *any* package that has *any* kind of configuration
files or is a library will *never* change in an incompatible way.

Basically, what you want is the assurances of a binary distribution that
things will "just work" when upgraded, yet you still want the power of
Gentoo.  Honestly, I don't see portage ever being able to really support
anything like this so long as the tree continues to change.  It simply
doesn't seem to be compatible with how Gentoo development is done.  Then
again, I could be completely off my rocker and the portage team might
already have all of this implemented in some super-secret internal-only
version.  As I said, this would probably be best conversed with them on
the portage-devel list.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Kevin]

Chris Gianelloni wrote:
> On Wed, 2006-04-26 at 12:29 -0400, Kevin wrote:
>> One thing that I'm pretty sure is currently not possible with portage,
>> however, and that I'd definitely like to see as a part of this idea is a
>> way of setting thresholds on version numbers of packages in portage such
>> that the automated upgrade system will only upgrade a package
>> automatically if the difference in version numbers between the installed
>> package and the newest available package in portage is greater than some
>> admin-tunable amount.  For example, I might not want to upgrade emacs or
>> xemacs just because a new -r number becomes available.  Maybe I don't
>> want to have such a big package upgraded automatically unless there is a
>> new major or minor version number.
>>
>> Thanks again to all the developers who have made Gentoo.  It's a really
>> terrific distro.
> 
> Jakub meant the portage-devel mailing list, not this one.

woops.  when i saw portage/devel i figured one or the other... guess not...

> 
> Anyway, most of this can be done already using /etc/portage files and
> some well-written cron scripts.  You can lock down versions of specific
> packages quite easily using your own package.mask and package.unmask
> files, along with package.keywords.

Yes, and as I wrote, I'm aware of your points here, and I do use these
capabilities fairly extensively now, but it is not the sort of
fine-tunable system that I have in mind where I could inform the
automated-update-system (for lack of a better name) of my wishes in
terms of which packages are safe (in my judgment, as the sysadmin of
that particular box) to upgrade in an unattended manner and which are
not (at least, not unless I'm much less well-informed on Gentoo than I
believe myself to be, which I'm not denying might be true).

And unless I'm way off-base, the version-difference-threshold notion
described above is not implemented in portage now.  Someone please
correct me if I'm wrong.

>  However, one thing you can *never*
> do is assume that *any* package that has *any* kind of configuration
> files or is a library will *never* change in an incompatible way.

Well your comment is certainly true in the most extreme interpretation,
but the same thing can be accurately said about whether or not one
should assume that the sun is going to rise tomorrow or that the
universe won't disappear in a quantum fluctuation while you're sleeping,
but IMO, such extreme statements have little value in day-to-day
application.  Everyone must make some assumptions about nearly
everything or it becomes nearly impossible to function.  I make all
kinds of assumptions in administering computers and they almost always
make my life much, MUCH easier than it would be without the assumptions.
 Sometimes they bite me, but only rarely.  The key to success here is
having the judgment to know what is relatively safe to make assumptions
about and what is not.  Judgment is something that only a human can
provide... not a computer.  This is why I want greater and more granular
control over upgrading packages in Gentoo.  Aside from the points you
make above (and I may be missing some other features currently present
in Gentoo), my choices now are, in the grossest terms: upgrade every
package by hand, one at a time, while sitting in front of the computer
(which is very close to what I spent last weekend doing) or do an emerge
world and hope for the best.  IMO, that's not much control and does not
allow for the application of judgment except in the former option (which
is very, very time consuming).

> 
> Basically, what you want is the assurances of a binary distribution that
> things will "just work" when upgraded, 

No.  Not true at all.  And for that matter, binary distributions (in my
10+ years of experience with them) simply do NOT just work: not for
Slackware, not for Yellowdog, not for SuSE, not for Redhat, nor
Mandrake, nor any of a dozen others I've tried.  I found that quite the
opposite was true which was one of the reasons I switched to Gentoo
(which I've found, usually DOES just work after upgrades; not always,
but usually---and this is a credit to the Gentoo developers).

What I really want is to make the process of maintaining Gentoo boxes
over the long term easier (IOW: less time-consuming) than is now true,
by adding some functionality that AFAICT does not now exist which would
allow me to automate some things, turn off automation of other things,
and as the sysadmin, have control over what those things should be.  In
my mind at least, the central theme in Gentoo of choice dovetails nicely
with what I'm trying to describe here: control and choice that is highly
fine-tunable by the owner of the box in regards to package upgrades.

I'm not a member of the portage-devel mailing list so I'm going to drop
this now.  If someone here is a member of both, then please feel free to
cross-post this thread to whatever forum is most appropriate for it.
After spending 30-45 minutes trying to help improve Gentoo by posting a
new (AFAICT) idea in bugzilla and again here, I feel like I've done
enough.  IMHO, this is an idea that would add great value to Gentoo and
I can't help but think that many sysadmins who must maintain many boxes
would agree, but I have no particular attachment to the idea that would
make me want to go around every mailing list under the sun trumpeting my
idea to anyone who will listen.  I'm just posting an idea that seemed
like a good one to me.  The devs may take it or leave it as they see fit.

Regards,
Kevin
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Jean-Francois Gagnon Laporte]

On 4/26/06, Kevin <lists@gnosysllc.com> wrote:
>
> What I really want is to make the process of maintaining Gentoo boxes
> over the long term easier (IOW: less time-consuming) than is now true,
> by adding some functionality that AFAICT does not now exist which would
> allow me to automate some things, turn off automation of other things,
> and as the sysadmin, have control over what those things should be.  In
> my mind at least, the central theme in Gentoo of choice dovetails nicely
> with what I'm trying to describe here: control and choice that is highly
> fine-tunable by the owner of the box in regards to package upgrades.
>
>
> Regards,
> Kevin
> --

Have a look at GLCU (http://www.panhorst.com/glcu/). Might not be the
perfect solution for your needs but it might help. Tools like this
one, /etc/portage and a private overlay for testing and/or pinning
would be pretty usefull for you right now. Might want to check GLEP 19
IIRC for the enterprise tree idea.

Regards,

Jean-Francois

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 4217 bytes --]

On Wed, 2006-04-26 at 14:24 -0400, Kevin wrote:
> And unless I'm way off-base, the version-difference-threshold notion
> described above is not implemented in portage now.  Someone please
> correct me if I'm wrong.

You're off-base.

See, you can, for example, mask all revisions within the same version of
a package in your /etc/portage/package.mask file quite easily.  So you
*could* have xemacs, using your own example, only ask to upgrade once
the stable version in the tree went over a certain threshold.

> Well your comment is certainly true in the most extreme interpretation,
> but the same thing can be accurately said about whether or not one
> should assume that the sun is going to rise tomorrow or that the
> universe won't disappear in a quantum fluctuation while you're sleeping,
> but IMO, such extreme statements have little value in day-to-day
> application.  Everyone must make some assumptions about nearly
> everything or it becomes nearly impossible to function.  I make all
> kinds of assumptions in administering computers and they almost always
> make my life much, MUCH easier than it would be without the assumptions.

I'm sorry, but do your friends call you Duncan?  I'll leave it at that.

>  Sometimes they bite me, but only rarely.  The key to success here is
> having the judgment to know what is relatively safe to make assumptions
> about and what is not.  Judgment is something that only a human can
> provide... not a computer.  This is why I want greater and more granular
> control over upgrading packages in Gentoo.  Aside from the points you
> make above (and I may be missing some other features currently present
> in Gentoo), my choices now are, in the grossest terms: upgrade every
> package by hand, one at a time, while sitting in front of the computer
> (which is very close to what I spent last weekend doing) or do an emerge
> world and hope for the best.  IMO, that's not much control and does not
> allow for the application of judgment except in the former option (which
> is very, very time consuming).

You missed the ability to lock down to specific package versions, which
is already a 100% possibility with current portage.  You can lock down
the versions to *anything that you want* via package.mask and
package.unmask, then simply have your system run an "emerge --update
--deep world" to automatically upgrade any and all packages not listed
in your mask files.

> What I really want is to make the process of maintaining Gentoo boxes
> over the long term easier (IOW: less time-consuming) than is now true,
> by adding some functionality that AFAICT does not now exist which would
> allow me to automate some things, turn off automation of other things,
> and as the sysadmin, have control over what those things should be.  In
> my mind at least, the central theme in Gentoo of choice dovetails nicely
> with what I'm trying to describe here: control and choice that is highly
> fine-tunable by the owner of the box in regards to package upgrades.

Yup.  It's called /etc/portage and we've had it for a while.  You simply
seem to be missing its flexibility.

> I'm not a member of the portage-devel mailing list so I'm going to drop
> this now.  If someone here is a member of both, then please feel free to
> cross-post this thread to whatever forum is most appropriate for it.
> After spending 30-45 minutes trying to help improve Gentoo by posting a
> new (AFAICT) idea in bugzilla and again here, I feel like I've done
> enough.  IMHO, this is an idea that would add great value to Gentoo and
> I can't help but think that many sysadmins who must maintain many boxes
> would agree, but I have no particular attachment to the idea that would
> make me want to go around every mailing list under the sun trumpeting my
> idea to anyone who will listen.  I'm just posting an idea that seemed
> like a good one to me.  The devs may take it or leave it as they see fit.

Yep.  It was such a good idea that the portage team implemented it quite
some time ago.  *grin*

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Kevin]

Jean-Francois Gagnon Laporte wrote:
> On 4/26/06, Kevin <lists@gnosysllc.com> wrote:
>> What I really want is to make the process of maintaining Gentoo boxes
>> over the long term easier (IOW: less time-consuming) than is now true,
>> by adding some functionality that AFAICT does not now exist which would
>> allow me to automate some things, turn off automation of other things,
>> and as the sysadmin, have control over what those things should be.  In
>> my mind at least, the central theme in Gentoo of choice dovetails nicely
>> with what I'm trying to describe here: control and choice that is highly
>> fine-tunable by the owner of the box in regards to package upgrades.
> 
> Have a look at GLCU (http://www.panhorst.com/glcu/). Might not be the
> perfect solution for your needs but it might help. Tools like this
> one, /etc/portage and a private overlay for testing and/or pinning
> would be pretty usefull for you right now. Might want to check GLEP 19
> IIRC for the enterprise tree idea.

Thanks very kindly for your reply and pointer, Jean-Francois!

-Kevin
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Kevin]

Chris Gianelloni wrote:
>        Honestly, I don't see portage ever being able to really
> support anything like this so long as the tree continues to change.
> It simply doesn't seem to be compatible with how Gentoo development is
> done.

and Chris Gianelloni wrote:

> 
> Yup.  It's called /etc/portage and we've had it for a while.  You simply
> seem to be missing its flexibility.
<snip>
> Yep.  It was such a good idea that the portage team implemented it quite
> some time ago.  *grin*
> 

Which is it, Chris?

Impossible that portage will ever be able to really support anything
like this...

or...

already done and I (along with Jean-Francois and Michael Schilling) am
just too stupid to see it?

Make up your mind...

For all the credit that I give to the Gentoo developers, you are one
from whom I would withdraw it.  You are merely a polarity responder and
as such, not worth engaging any further.  You would do Gentoo a service
by keeping your mouth shut in response to posts like mine (and many
others I'm sure).  If I were not so self-assured, you would simply drive
me away from Gentoo with such ugly remarks, and I've no doubt that you
have driven away many others who are less self-assured.

You're in my kill file now, so don't waste your time and the bandwidth
by responding.  All mailing lists have a topic and attempting to
ridicule other people is not on-topic in any of them...  certainly not
here.  It just facilitates making you look like a jerk which you need no
assistance with.

-Kevin
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Mike Frysinger]

zing !

and with this post it's probably best to let this subthread die before we get 
any more offtrack
-mike
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] General flaming guidelines ;-)

[Patrick Lauer]

[-- Attachment #1: Type: text/plain, Size: 2145 bytes --]

On Wed, 2006-04-26 at 15:55 -0400, Kevin wrote:
> Which is it, Chris?
You've taken that out of context ...

> Make up your mind...
I think he has, but wasn't able to communicate his ideas to you in an adequte way

> For all the credit that I give to the Gentoo developers, you are one
> from whom I would withdraw it.  You are merely a polarity responder and
> as such, not worth engaging any further.  You would do Gentoo a service
> by keeping your mouth shut in response to posts like mine (and many
> others I'm sure).  If I were not so self-assured, you would simply drive
> me away from Gentoo with such ugly remarks, and I've no doubt that you
> have driven away many others who are less self-assured.
Dude ... calm down. Seriously. Insulting or angering us won't help anything and will only make us ignore any good ideas you may have.
Your condescending tone makes me want to ignore anything you say ...

> You're in my kill file now, so don't waste your time and the bandwidth
> by responding.  All mailing lists have a topic and attempting to
> ridicule other people is not on-topic in any of them...  certainly not
> here.  It just facilitates making you look like a jerk which you need no
> assistance with.
Deliberately misunderstanding people and then yelling at them is also
not something that's on-topic for a mailinglist. 

In the last two weeks or so I've seen many users that ignore advice,
feel insulted at every turn and are not cooperative in any way. This is
frustrating and makes those of us who interact with users irritated. I
really wish we could have a level-headed discussion instead of
insulting, insinuating and trying to annoy people.

If you feel that a developer is uncooperative, that you don't get the
attention you deserve ... if you have any idea how to make things
better, send a mail to user-relations@gentoo.org 
Please don't take your frustrations out on developers, let us try to fix
the problem.
I hope that we can help you resolve any conflicts you may or may not
have.

wkr,
Patrick,
UserRel Monkey 
-- 
Stand still, and let the rest of the universe move

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Peter]

On Wed, 26 Apr 2006 15:30:35 -0400, Kevin wrote:

> Jean-Francois Gagnon Laporte wrote:
>> On 4/26/06, Kevin <lists@gnosysllc.com> wrote:
>>> What I really want is to make the process of maintaining Gentoo boxes
>>> over the long term easier (IOW: less time-consuming) than is now true,
>>> by adding some functionality that AFAICT does not now exist which would
>>> allow me to automate some things, turn off automation of other things,
>>> and as the sysadmin, have control over what those things should be.  In
>>> my mind at least, the central theme in Gentoo of choice dovetails nicely
>>> with what I'm trying to describe here: control and choice that is highly
>>> fine-tunable by the owner of the box in regards to package upgrades.
>> 
>> Have a look at GLCU (http://www.panhorst.com/glcu/). Might not be the
>> perfect solution for your needs but it might help. Tools like this
>> one, /etc/portage and a private overlay for testing and/or pinning
>> would be pretty usefull for you right now. Might want to check GLEP 19
>> IIRC for the enterprise tree idea.
> 
> Thanks very kindly for your reply and pointer, Jean-Francois!
> 
> -Kevin

PMFJI. It seems you want the best of both worlds, and as such are asking a
lot! I fully understand your desire, but IMHO, you can come very close,
but never achieve your stated goal. FWIW, here's what I do.

I have a daily cron job which simply does:

emerge -puNDvt --nospinner world >/home/${MAILTO}/emerge.log

Then, if there is something I don't want to emerge, or a trivial -r?
revision to an ebuild I don't want, I mask it. For example, recently, gcc
went from 3.4.5 to 3.4.5-r1. To me, the change was unnecessary. So I
masked it. There are ways, as previously noted to mask to the major or
minor version, or exclude revisions as well. The problem with that
approach is that a particular revision _COULD_ be important or useful to
you. However, when I do a package.mask entry, I normally use = and mask a
specific version. Using >= is dicey since you would miss an upgrade.

However, from the way I use gentoo, having an automated system would be
quite counterintuitive. Semi-automated? Sure thing. In any event, still
beats the $#^%@ out of RPMs!

Good luck.

-- 
Peter


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Kevin]

Thanks for your informative reply, Peter.  I think I'll try your method
for awhile.  I'm sure it's less time consuming than my current method,
if perhaps still not ideal, and although I do realize this idea may be
an unattainable utopia, by Jean-Francois pointing me to glcu, I'm glad
to see that I'm not the only person who would like to see an improvement
in this area, and that someone's already been working on it for awhile
now.  This motivates me to take a look at Michael's code and see if I
can help make it still better.

And although in my perception the thread speaks for itself, I'd like to
apologize to anyone who perceived any of my comments as condescending,
for that was not my intent.  Perception is an interesting thing in that
it depends very heavily on the one doing the perceiving and the
author/speaker cannot know everything about all of the possible readers
(thus all the possible perceptions/interpretations).  I did my best to
be clear and to the point and not intentionally condescending, using
examples to elaborate, and I did not deliberately misunderstand anything.

Above all else, I wish to make it clear that IMO, after fiddling with
many different Linux distros for more than 10 years, the Gentoo
developers have got the closest that I've seen to reaching that utopia.
 Very nice work, folks, and thank you very much for making such a
terrific distro.

-Kevin
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Duncan]

Chris Gianelloni posted <1146079658.32740.15.camel@cgianelloni.nuvox.net>,
excerpted below,  on Wed, 26 Apr 2006 15:27:38 -0400:

> I'm sorry, but do your friends call you Duncan?  I'll leave it at that.

Who, me?  <looks around>  No, safe to say, /not/ me.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Richard Fish]

On 4/26/06, Kevin <lists@gnosysllc.com> wrote:
> I'd like to have the capability of being able to list some packages that should
> never be upgraded automatically (I realize I can do this to some degree already
> with portage), some others that are very unlikely to break from an automated
> upgrade and thus should always be upgraded automatically, and some packages

So maybe this could be satisified by allowing user-defined categories
of packages beyond system and world?  Something like world, system,
fragile, non-fragile?

I think you could probably implement something like this yourself with
a bit of trickery with the /var/lib/portage/world list.  You could
copy world to non-fragile, remove anything that you consider fragile
from it, and then do an "automatic" update with:

cp /var/lib/portage/world /var/lib/portage/world.bak
cp /var/lib/portage/non-fragile /var/lib/portage/world
emerge -DNuv world
cp /var/lib/portage/world.bak /var/lib/portage/world

A similar script for the fragile packages would let you update those
as a group, obviously using the -p and/or --ask options as you like.

Going back to -user now...

-Richard

-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Duncan]

Richard Fish posted
<7573e9640604262210v4ec963f0l904bce949be3824f@mail.gmail.com>, excerpted
below,  on Wed, 26 Apr 2006 22:10:27 -0700:

> So maybe this could be satisified by allowing user-defined categories
> of packages beyond system and world?  Something like world, system,
> fragile, non-fragile?

Actually, that's called set support, and it's actively planned for a
future portage.  I haven't been tracking it to the point where I know what
the implementation status is, but it's very likely in portage-core (aka
savior aka portage-ng, portage folks correct me if I'm simply confused) to
some extent now.  Again, portage-devel would be the preferred place for
discussions.  (As used to be common courtesy with groups/lists, reading up
on the last few weeks to 3 months worth of existing messages before
posting something that will then be understood to be a FAQ, is nice, but
the devs don't usually bite too hard if you don't. =8^)

> I think you could probably implement something like this yourself with
> a bit of trickery with the /var/lib/portage/world list.  You could
> copy world to non-fragile, remove anything that you consider fragile
> from it, and then do an "automatic" update with:
> 
> cp /var/lib/portage/world /var/lib/portage/world.bak
> cp /var/lib/portage/non-fragile /var/lib/portage/world
> emerge -DNuv world
> cp /var/lib/portage/world.bak /var/lib/portage/world
> 
> A similar script for the fragile packages would let you update those
> as a group, obviously using the -p and/or --ask options as you like.

Actually, the idea is slightly more complicated at one step, and slightly
less at another, but very workable with existing portage.

You can't simply begin with the world file, as that by definition won't
necessarily include stuff in system.  Rather, begin with the world file,
then add the stuff in system.  (The best way is probably to walk your
profile tree following the parent nodes, and include what you find in
packages.)

Or, start generate the initial list with an 

emerge --pretend --emptytree > newpkglist

Note that the latter, however, will include "trunk and branch" packages as
well as the "leaf" packages normally found in world.  This would therefore
require the use of --oneshot with any emerge based on the list, so as to
prevent stuffing world with unnecessary dependencies.  Those dependencies
can also change over time, one of the reasons you /don't/ want them in
world (and why an occasional emerge --depclean --pretend is recommended as
routine system hygiene on a Gentoo system, generating a list to either be
added to world or unmerged as necessary, read the docs for more
information if you aren't already doing this as part of your normal
routine).  Due to this change over time, if you use this generation
method, you'll also wish to regenerate the list periodically.  OTOH, it'll
give you a more accurate list to start with, if you consider any of the
dependencies especially fragile or robust, since they'd not be directly
handled using the first generation method, only treated as dependencies.

In any case, once you get your list and weed out the stuff you /don't/
want on it, rather than doing that copy trickery, try this:

emerge -NuDv $(cat /path/to/listfile)

Again, the usual pretend/ask situation applies, and --oneshot should be
added since you are updating specific packages from the command line. 
Also consider the effect of the -D and -N flags, as depending on your
exact needs, --newuse and --deep may or may not be suitable.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Richard Fish]

On 4/27/06, Duncan <1i5t5.duncan@cox.net> wrote:
> In any case, once you get your list and weed out the stuff you /don't/
> want on it, rather than doing that copy trickery, try this:

Yeah, much smarter than my cp tricks.  Although using emerge to
generate the package list will have a problem in that it contains the
versions, which is not what you want for this.  But we seem to have
going completely off-topic for -dev now...

-Richard

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Kevin]

Thank you both for your detailed replies.  I've got quite a list of
additional notions now for trying to implement this idea myself, and I'm
very grateful for the thoughtful suggestions.  I also have a better
understanding of current portage capabilities, so I appreciate the
additional commentary on that.

If I explore this idea with any further discussion, I'll be sure to
follow the suggestions here about another list and reading past messages
on that list.

-Kevin

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Wishlist: an automated package upgrade system with fine-tunable sysadmin control

[Radoslaw Stachowiak]

On 4/27/06, Kevin <lists@gnosysllc.com> wrote:

> If I explore this idea with any further discussion, I'll be sure to
> follow the suggestions here about another list and reading past messages
> on that list.

Feel free to look at http://dev.gentoo.org/~radek/portki/

Short feature lists:
* designed to run in fully automated way
* uses master/slave portage replication mode with separate _own_
master portage repo
* additional own repos for home made software
* allows different upgrade cycles per machine (weekly/monthly/yearly, glsa only)
* allows automatic cfg upgrade _if_ file was not touched by human.
* does all necessary cleaning
* extensive logging
* does revdep-rebuild and other necessary portage tricks
* allows central configuration (useflags, make.conf etc.)
* it used binpkg in early version but I discarded this idea because of:
.. problems generated even on compatibile hosts
.. necessity to have all the same cfg (which is often not the case)
.. small gain (on todays servers compilation is fast)
.. much harder individual changes to the machines
* its just an integration work (mostly bash and python) around
standard gentoo tools, although its in highly automated state.
* easy to extend

Biggest drawbacks:
* some bugs :)
* lacking documentation

--
radoslaw.

-- 
gentoo-dev@gentoo.org mailing list