[gentoo-server] Stable portage tree

[gentoo-server] Stable portage tree

[Jan Meier]

Hello,

how is the status of the stable portage tree? Is it already available? 

I am really interested in it because I am tired of frequently updates on my 
server just because there is a new version. Doing only security update would 
be nice.

Regards

Jan
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Marten Persson]

On Wednesday 16 August 2006 09.06, Jan Meier wrote:
> Hello,
>
> how is the status of the stable portage tree? Is it already available?
>
> I am really interested in it because I am tired of frequently updates on my
> server just because there is a new version. Doing only security update
> would be nice.
>
> Regards
>
> Jan
Whu do you need the latest versions? My servers run upates once or twice 
yearly and some security patching in between. 

Just a thought.

Marten
-- 
Höjebromölla
Mårten Persson

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Jan Meier]

Am Mittwoch 16 August 2006 11:01 schrieb Marten Persson:
> On Wednesday 16 August 2006 09.06, Jan Meier wrote:
> > Hello,
> >
> > how is the status of the stable portage tree? Is it already available?
> >
> > I am really interested in it because I am tired of frequently updates on
> > my server just because there is a new version. Doing only security update
> > would be nice.
> >
> > Regards
> >
> > Jan
>
> Whu do you need the latest versions? My servers run upates once or twice
> yearly and some security patching in between.

No, I do not need the latest version. But I do not want to do "some security 
patching", I want to have every security risk patched (updated), without 
updating all the dependencies. That's the point.

For example emerge -u imagemagick shows a really long list for updating, I do 
not think that all of them are really needed. 

Regards

Jan


> Just a thought.
>
> Marten
> --
> Höjebromölla
> Mårten Persson

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Craig Webster]

On 16 Aug 2006, at 10:19, Jan Meier wrote:
> No, I do not need the latest version. But I do not want to do "some  
> security
> patching", I want to have every security risk patched (updated),  
> without
> updating all the dependencies. That's the point.
>
> For example emerge -u imagemagick shows a really long list for  
> updating, I do
> not think that all of them are really needed.

Have you tried using glsa-check?

Cheers,
Craig
--
No long-term contracts, no complicated signup forms, no hidden costs.
Xeriom 2.0: Web hosting made easy. Coming soon! http://xeriom.net/


-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Jan Meier]

Am Mittwoch 16 August 2006 11:36 schrieb Craig Webster:
> On 16 Aug 2006, at 10:19, Jan Meier wrote:
> > No, I do not need the latest version. But I do not want to do "some
> > security
> > patching", I want to have every security risk patched (updated),
> > without
> > updating all the dependencies. That's the point.
> >
> > For example emerge -u imagemagick shows a really long list for
> > updating, I do
> > not think that all of them are really needed.
>
> Have you tried using glsa-check?

I am using glsa-check for reporting vulnerable software, currently not for 
updating. 
I will give "emerge imagemagick" a shot, maybe that has less dependencies :).  
With your answeres in mind I came to the opinion that there is not a real 
need for a "stable portage tree". 

Regards

Jan

> Cheers,
> Craig
> --
> No long-term contracts, no complicated signup forms, no hidden costs.
> Xeriom 2.0: Web hosting made easy. Coming soon! http://xeriom.net/

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Ian P. Christian]

[-- Attachment #1: Type: text/plain, Size: 1246 bytes --]

On 08/16/06 Jan Meier wrote:
> I am using glsa-check for reporting vulnerable software, currently 
> not for updating. I will give "emerge imagemagick" a shot, maybe that 
> has less dependencies :) . With your answeres in mind I came to the 
> opinion that there is not a real need for a "stable portage tree". 

I personally think there is a a large need for a stable tree.

I run 10s of servers, and I'm sure there's people on this list who run
many more.

Updating every 6/12 months is fine in principle, but it means going
though 10's of machines updating config files and resolving conflics.
This is a painful task, it's fine for 1 machine, it's fine for 5... but
you have any real number of servers to maintain and it ends up taking
hours or days to upgrade your servers.

A stable tree that has an update cycle of something like 6 months and
perhaps a security overlay (implement as an overlay perhaps to reduce
the sync time and therefore resources) would be idea - then upgrading
between 'releases' could be well documented and coordinated.
Unfortunatly, this is a huge project - and without a small/medium team
of dedicated gentoo devs, it's not going to happen.

-- 
Ian P. Christian ~ http://pookey.co.uk


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable portage tree

[Paul Kölle]

Ian P. Christian wrote:
> On 08/16/06 Jan Meier wrote:
>> I am using glsa-check for reporting vulnerable software, currently 
>> not for updating. I will give "emerge imagemagick" a shot, maybe that 
>> has less dependencies :) . With your answeres in mind I came to the 
>> opinion that there is not a real need for a "stable portage tree". 
> 
> I personally think there is a a large need for a stable tree.
[ snipp ]
The basic problem here is: Upstream may not publish "security fixes" but
just a new (fixed) version. If you want a "stable" tree, you have to
watch upstream cvs/svn/mailing lists and backport fixes. That is a lot
of work.

cheers
 Paul
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Ian P. Christian]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

On 08/16/06 Paul Kölle wrote:
> The basic problem here is: Upstream may not publish "security fixes" 
> but just a new (fixed) version. If you want a "stable" tree, you have 
> to watch upstream cvs/svn/mailing lists and backport fixes. That is a 
> lot of work.

that infrastructure is already in place in gentoo. Package maintainers
do it... they need to just make it clear when they update an ebuild
weather it's a general upgrade, or a security upgrade.


-- 
Ian P. Christian ~ http://pookey.co.uk


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable portage tree

[Paul Kölle]

Ian P. Christian wrote:
> On 08/16/06 Paul Kölle wrote:
>> The basic problem here is: Upstream may not publish "security fixes" 
>> but just a new (fixed) version. If you want a "stable" tree, you have 
>> to watch upstream cvs/svn/mailing lists and backport fixes. That is a 
>> lot of work.
> 
> that infrastructure is already in place in gentoo. Package maintainers
> do it... they need to just make it clear when they update an ebuild
> weather it's a general upgrade, or a security upgrade.

glsa-check will tell you if it's a security upgrade, but it will do
version bumps including ${PV} nevertheless. That is, your dependency
tree will change and possibly lead to unwanted upgrades (read: upgrade
with possible config changes, new features, new bugs).
AFAIK gentoo devs don't do backports, i.e. if samba has a vulnerability
in say 3.0.23a which is fixed in 3.0.23b, you won't get a  "security
fixes only" 3.0.23a-r1 but just 3.0.23b with new features *and* fixed bugs.

cheers
 Paul
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Jan Meier]

Am Mittwoch 16 August 2006 12:18 schrieb Ian P. Christian:
> On 08/16/06 Paul Kölle wrote:
> > The basic problem here is: Upstream may not publish "security fixes"
> > but just a new (fixed) version. If you want a "stable" tree, you have
> > to watch upstream cvs/svn/mailing lists and backport fixes. That is a
> > lot of work.
>
> that infrastructure is already in place in gentoo. Package maintainers
> do it... they need to just make it clear when they update an ebuild
> weather it's a general upgrade, or a security upgrade.

I think every update because of security reasons has a security announcement.

I would be willing to start such a stable tree, I am thinking of taking a 
current portage tree, delete all ~arch ebuilds and create an overlay. Every 
time a security announcement is fired up I will add the newer ebuild to the 
overlay, checking for any really needed depencies.

The main portage tree will be updatedwith every new release, and the older 
trees will be supported until three new releases. Supported architecture 
would be currently only x86.

The overlay and the portage snapshot will I make public available.

What do you think about this?
The main problem is that it does not match the philosophy of gentoo. If other 
architectures should also be available it would be a lot of work.

Regards 

Jan

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Paul Kölle]

Jan Meier wrote:
> I would be willing to start such a stable tree, I am thinking of taking a 
> current portage tree, delete all ~arch ebuilds and create an overlay. Every 
> time a security announcement is fired up I will add the newer ebuild to the 
> overlay, checking for any really needed depencies.

~arch doesn't hurt, so the main difference to glsa-check+standard tree
would be old ebuilds not being deleted right? AFAIK that can be done by
removing the --delete and --delete-after flag from PORTAGE_RSYNC_OPTS in
/etc/make.conf (dunno if thats "supported" though).

cheers
 Paul
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Jan Meier]

Am Mittwoch 16 August 2006 15:12 schrieb Paul Kölle:
> Jan Meier wrote:
> > I would be willing to start such a stable tree, I am thinking of taking a
> > current portage tree, delete all ~arch ebuilds and create an overlay.
> > Every time a security announcement is fired up I will add the newer
> > ebuild to the overlay, checking for any really needed depencies.
>
> ~arch doesn't hurt, so the main difference to glsa-check+standard tree
> would be old ebuilds not being deleted right? 

No, the advantage would be that new ebuilds would not come into the portage 
tree. Only security relevant ebuilds, formerly which fix security holes, 
would come into the tree (kernel, php, mysql, apache, etc. should not be 
stopped from entering the portage tree).
This has the advantage that there would be less packages to update when the 
system has to be updated. And if there are security relevant updates there 
would not be as much dependency updates as with the normal tree.

Take a look here:
http://www.gentoo.org/proj/en/glep/glep-0019.html

Regards

Jan

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Paul Kölle]

Jan Meier wrote:
> Am Mittwoch 16 August 2006 15:12 schrieb Paul Kölle:
>> Jan Meier wrote:
>>> I would be willing to start such a stable tree, I am thinking of taking a
>>> current portage tree, delete all ~arch ebuilds and create an overlay.
>>> Every time a security announcement is fired up I will add the newer
>>> ebuild to the overlay, checking for any really needed depencies.
>> ~arch doesn't hurt, so the main difference to glsa-check+standard tree
>> would be old ebuilds not being deleted right? 
> 
> No, the advantage would be that new ebuilds would not come into the portage 
> tree. Only security relevant ebuilds, formerly which fix security holes, 
> would come into the tree (kernel, php, mysql, apache, etc. should not be 
> stopped from entering the portage tree).
Sorry, I don't get it. Why are you concerned about packages in the tree
you don't use? Is it about space savings?

> This has the advantage that there would be less packages to update when the 
> system has to be updated. And if there are security relevant updates there 
> would not be as much dependency updates as with the normal tree.
The depgraph of a bumped package does not depend on being bumped due to
a GLSA or not. If you only use glsa-check, you will get GLSA triggered
upgrades only and glsa-check will emerge the lowest safe version
possible. Keeping old versions around is sufficient to prevent unneeded
upgrades. If you want something like "emerge -u --stable world", well
then you would need a dedicated tree for --stable but thats way more
work than just deleting ~arch ebuilds you wouldn't use anyway.

> 
> Take a look here:
> http://www.gentoo.org/proj/en/glep/glep-0019.html
This glep talkes about a "stable tree" which conforms to some "higher"
QA standars than <arch> but I haven't seen much work here. Portage does
not support the "stable:<arch>" syntax and there is no sign gentoo devs
can handle those "higher QA" currently (see my comments on backporting
and missing seperate security patches upstream).

cheers
 Paul
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Jan Meier]

Am Mittwoch 16 August 2006 16:11 schrieb Paul Kölle:
> Jan Meier wrote:
> > Am Mittwoch 16 August 2006 15:12 schrieb Paul Kölle:
> >> Jan Meier wrote:
> >>> I would be willing to start such a stable tree, I am thinking of taking
> >>> a current portage tree, delete all ~arch ebuilds and create an overlay.
> >>> Every time a security announcement is fired up I will add the newer
> >>> ebuild to the overlay, checking for any really needed depencies.
> >>
> >> ~arch doesn't hurt, so the main difference to glsa-check+standard tree
> >> would be old ebuilds not being deleted right?
> >
> > No, the advantage would be that new ebuilds would not come into the
> > portage tree. Only security relevant ebuilds, formerly which fix security
> > holes, would come into the tree (kernel, php, mysql, apache, etc. should
> > not be stopped from entering the portage tree).
>
> Sorry, I don't get it. Why are you concerned about packages in the tree
> you don't use? Is it about space savings?

Eh, no. In my opinion it is clear what I want to say, so I have nothing to 
add.

> > This has the advantage that there would be less packages to update when
> > the system has to be updated. And if there are security relevant updates
> > there would not be as much dependency updates as with the normal tree.
>
> The depgraph of a bumped package does not depend on being bumped due to
> a GLSA or not. If you only use glsa-check, you will get GLSA triggered
> upgrades only and glsa-check will emerge the lowest safe version
> possible. Keeping old versions around is sufficient to prevent unneeded
> upgrades. If you want something like "emerge -u --stable world", well
> then you would need a dedicated tree for --stable but thats way more
> work than just deleting ~arch ebuilds you wouldn't use anyway.

The ~arch ebuilds are not the point, the stable ebuilds which potentially be 
upgraded are the point. If you say that glsa-check does only update the 
package which is security relevant and tries not to update the dependencies 
then this is what I want.

Regards

Jan


-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 999 bytes --]

On Wed, 16 Aug 2006 16:40:01 +0200
Jan Meier <jan.meier@zmnh.uni-hamburg.de> wrote:

> The ~arch ebuilds are not the point, the stable ebuilds which
> potentially be upgraded are the point. If you say that glsa-check
> does only update the package which is security relevant and tries not
> to update the dependencies then this is what I want.

It will only update dependencies when they are strictly required by the
new version, same like emerge if you don't use -u (which should
only be used for system and world updates anyway). Basically 
    glsa-check -f some-glsa
will call
    emerge --oneshot $EMERGE_OPTS =package-version
where 'version' is the lowest "safe" version that doesn't result in a
downgrade (of course if the system isn't affected it won't do anything).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable portage tree

[Sune Kloppenborg Jeppesen]

[-- Attachment #1: Type: text/plain, Size: 374 bytes --]

On Wednesday 16 August 2006 13:26, Jan Meier wrote:
> I think every update because of security reasons has a security
> announcement.
Not every security issue results in a GLSA [1].

[1] http://www.gentoo.org/security/en/vulnerability-policy.xml

-- 
Sune Kloppenborg Jeppesen (Jaervosz)
Operational Manager
Gentoo Linux Security Team
http://security.gentoo.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable portage tree

[Jan Meier]

Am Mittwoch 23 August 2006 07:30 schrieb Sune Kloppenborg Jeppesen:
> On Wednesday 16 August 2006 13:26, Jan Meier wrote:
> > I think every update because of security reasons has a security
> > announcement.
>
> Not every security issue results in a GLSA [1].
> [1] http://www.gentoo.org/security/en/vulnerability-policy.xml

Ahh, good to know. 

Regards

Jan
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Alex Efros]

Hi!

On Wed, Aug 16, 2006 at 11:00:21AM +0100, Ian P. Christian wrote:
> Updating every 6/12 months is fine in principle, but it means going
> though 10's of machines updating config files and resolving conflics.
> This is a painful task, it's fine for 1 machine, it's fine for 5... but
> you have any real number of servers to maintain and it ends up taking
> hours or days to upgrade your servers.

Yeah, your right. But there simple solution for this: update your servers
every 3-4 days, and you will be surprised how ease and quick this task become.
You'll need from a couple of seconds to 2-3 minutes in average for such update!
Usually a few not important for you applications will be updated, which
can't broke anything on your server, and which require few seconds to
update their config files. Sometimes one of applications critical for your
server become updated, and this require more attention, but it's much
better to update ONE such important application instead of updating ALL of
such important applications every 6-12 month. And this way you always can
ease fallback to previous version of this application if something goes
wrong on your server, add broken (for you) version to
/etc/portage/package.mask, report bug and wait for next update.

I've tried all these ways of updating my servers in last 2 years:
update every few days, update only security issues, update every 6-12 months
and found first way much more ease, effective and manageable than others.
With two other ways I also wanna 'stable portage tree', with first way I
don't need it - ARCH=x86 IS A 'stable portage tree' for me now. :)

-- 
			WBR, Alex.
-- 
gentoo-server@gentoo.org mailing list

RE: [gentoo-server] Stable portage tree

[Jesse, Rich]

Constant and needless updating servers is the exact opposite of
"stable".  Server stability equates to money in almost all business,
IMHO.  Why on earth would I risk my stability on a daily basis by
emerging world?  Remember that the ONLY reason to upgrade a server is if
there is discernable benefit.  The benefit may be a security fix, bug
fix, supportability, enhancement, or it just looks cooler -- that's for
the user/benefactor(s) to decide.

By default, Portage doesn't lend itself to this.  I don't need/want the
latest Postgres just because it's available, especially when the upgrade
would require data and/or app migration.  Upgrades warrant testing.  I
can't justify spend hundreds of man-hours testing all available apps on
a given system just because some program went from v4.3 to 4.3-1.

I also can't justify upgrading just because Gentoo no longer wants to
keep last year's ebuild around.  Thankfully, a sysadmin can make use of
OVERLAY and rsync (*without* "--delete"!) to create their own portage
tree, complete with all the old rebuilds.  Anyone that's tried to
upgrade an old OpenSSH knows what happens on the ensuing revdep-rebuild
-- ebuilds are gone, and you're stuck in the mud.

RedHat is stable.  It's also a PITA to maintain for some business apps.
Building Oracle on RedHat requires arcane incantations and animal
sacrifice.  But doing the same on Gentoo is the same as any flavor of
Unix.  So, I use RedHat in production, but Gentoo on my R&D desktop.
But that doesn't mean I don't need stability.  Any major libs get
changed and I need to relink Oracle.  Then I need to wonder what changed
and how to test it.  It's just not worth the hassle for almost all
updates for me.

I'm way short on time and way too terse here.  This is the kinda stuff
that needs to be debated over copius amounts of really freakin good
beer.

My $.02,
Rich


-----Original Message-----
From: Alex Efros [mailto:powerman@powerman.asdfGroup.com] 
Sent: Wednesday, August 16, 2006 6:30 AM
To: gentoo-server@lists.gentoo.org
Subject: Re: [gentoo-server] Stable portage tree

Hi!

On Wed, Aug 16, 2006 at 11:00:21AM +0100, Ian P. Christian wrote:
> Updating every 6/12 months is fine in principle, but it means going
> though 10's of machines updating config files and resolving conflics.
> This is a painful task, it's fine for 1 machine, it's fine for 5...
but
> you have any real number of servers to maintain and it ends up taking
> hours or days to upgrade your servers.

Yeah, your right. But there simple solution for this: update your
servers
every 3-4 days, and you will be surprised how ease and quick this task
become.
You'll need from a couple of seconds to 2-3 minutes in average for such
update!
Usually a few not important for you applications will be updated, which
can't broke anything on your server, and which require few seconds to
update their config files. Sometimes one of applications critical for
your
server become updated, and this require more attention, but it's much
better to update ONE such important application instead of updating ALL
of
such important applications every 6-12 month. And this way you always
can
ease fallback to previous version of this application if something goes
wrong on your server, add broken (for you) version to
/etc/portage/package.mask, report bug and wait for next update.

I've tried all these ways of updating my servers in last 2 years:
update every few days, update only security issues, update every 6-12
months
and found first way much more ease, effective and manageable than
others.
With two other ways I also wanna 'stable portage tree', with first way I
don't need it - ARCH=x86 IS A 'stable portage tree' for me now. :)

-- 
			WBR, Alex.
-- 
gentoo-server@gentoo.org mailing list

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Alex Efros]

Hi!

On Wed, Aug 16, 2006 at 09:16:55AM -0500, Jesse, Rich wrote:
> Constant and needless updating servers is the exact opposite of "stable".

Yeah.

But last years show ARCH=x86 is stable enough and such updates
very rare broke anything, so "constant" in this case doesn't result in
so many troubles as it sounds.

About "needless" - as I said before, in last years I've tried all ways
to update servers - exactly because I also wanna install only security
fixes for everything plus sometimes update some critical for my tasks
packages because of important bug fixes there... but this doesn't work 
in long term, :( while "constant updating" solve these issues without
introducing too many new problems.

> By default, Portage doesn't lend itself to this.  I don't need/want the
> latest Postgres just because it's available, especially when the upgrade
> would require data and/or app migration.  Upgrades warrant testing.  I
> can't justify spend hundreds of man-hours testing all available apps on
> a given system just because some program went from v4.3 to 4.3-1.

Hmm... Again, x86 is stable enough to avoid such retesting on each update.
I agree it's nice idea to retest everything, but it's just impossible -
you should define some intelligent amount of retesting which you able to
do quickly after update. Something like smoke testing in few clicks to be
sure your app is running and working with database is enough for most cases.
If some deeper problems arise in this app just because of database update
from 4.3 to 4.3.1 then it's probably because of bug in your app and it's
better to fix it NOW.

Probably this way isn't acceptable for you - I'm mostly administrate
servers dedicated for few complex apps, and it's ease to quickly check
them all after update.

Also, I don't think your example is good and realistic. So critical
components as database isn't update often, newer version of databases
isn't usually marked as dependency for some other app, so you usually
isn't forced to update it ASAP - you can delay database update until
you'll read changelog and become sure your apps are ready for it.

> I also can't justify upgrading just because Gentoo no longer wants to
> keep last year's ebuild around.  Thankfully, a sysadmin can make use of
> OVERLAY and rsync (*without* "--delete"!) to create their own portage
> tree, complete with all the old rebuilds.  Anyone that's tried to
> upgrade an old OpenSSH knows what happens on the ensuing revdep-rebuild
> -- ebuilds are gone, and you're stuck in the mud.

Yeah, I know. But removing --delete doesn't guaranty ability to install
old ebuild - just because ebuilds sometimes changed without versions
bumping, and reinstalling same version few months later can result in
compilation using different patches and/or configure options, etc.

Such "old" ebuild even can fail to unpack, see this example:
1) [January] foo-1.0.ebuild added, it use files/foo.patch
2) [Febrary] foo-1.0.ebuild deleted,
	     foo-2.0.ebuild added, it also use files/foo.patch, but this
	     is completely different patch while it has same name as
	     previous patch :(

And another problem: removing old ebuild from portage mean it isn't
supported anymore, so you doesn't get GLSA and bugfixes for it. This is
why naive initiative of Jan Meier (in second subthread of this thread)
will not work:

>> I think every update because of security reasons has a security announcement.
>> 
>> I would be willing to start such a stable tree, I am thinking of taking a
>> current portage tree, delete all ~arch ebuilds and create an overlay. Every
>> time a security announcement is fired up I will add the newer ebuild to the
>> overlay, checking for any really needed depencies.

> But that doesn't mean I don't need stability.  Any major libs get
> changed and I need to relink Oracle.  Then I need to wonder what changed

Yeah, but... there always some reason why things like glibc updates, and
you free to update it or delay update because you don't have time now
to relink Oracle.

There is a big difference between 'install only selected updates' and
'install all updates except selected'. I prefer second because first
don't work in long term (I got troubles installing security updates after
about 6-8 months going this way). To support first way and get 'stable
portage tree' we need big enough team of Gentoo devs dedicated for this
task. For now it doesn't looks like they willing to do this. 

Maybe 'Debian stable' is right choice for ppl who vote for 'stable
portage tree' - it has only very old, really stable packages and only
critical updates (I doesn't use Debian myself, so maybe I'm wrong about it).

> I'm way short on time and way too terse here.  This is the kinda stuff
> that needs to be debated over copius amounts of really freakin good
> beer.

Agreed! :)

-- 
			WBR, Alex.
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Ian P. Christian]

[-- Attachment #1: Type: text/plain, Size: 2396 bytes --]

Perhaps this is simply just a case of accepting there's 2 schools of
though on how to keep a system upto date.  If this is the case, Gentoo
certainly doesn't' lend itself well to the school I attend, and clearly
I'm not the only person who's there.


Alex Efros wrote:
> very rare broke anything, so "constant" in this case doesn't result in
> so many troubles as it sounds.
> 
> in long term, :( while "constant updating" solve these issues without
> introducing too many new problems.

Twice you've suggested there are problems, and it's ok because there
haven't been many. This really isn't the case.  I can't afford to
upgrade 10's of machines every week and test them all (mostly they do
different things obviously).

> Hmm... Again, x86 is stable enough to avoid such retesting on each update.
> I agree it's nice idea to retest everything, but it's just impossible -

No, it's not. On a 6/12 month cycle (or like ubuntu for example, I
*think* it's 18) you get plenty of time to setup your stuff on some test
systems and test them out properly.  Perhaps giving them a week or two's
worth of stress testing.

> If some deeper problems arise in this app just because of database update
> from 4.3 to 4.3.1 then it's probably because of bug in your app and it's
> better to fix it NOW.

I'm sorry, but that is just crazy talk ;)
You clearly don't deal with PHP, where a point release can break a LOT
of things, some things you might not notice by loading 2 or 3 pages from
a website.

> Probably this way isn't acceptable for you - I'm mostly administrate
> servers dedicated for few complex apps, and it's ease to quickly check
> them all after update.

Can I ask how many? Perhaps this is just that you've not hit the point
where it's just a PITA yet.
I used to have no problem running 5 or 6 machines, but now it's just a
nightmare.

> Maybe 'Debian stable' is right choice for ppl who vote for 'stable
> portage tree' - it has only very old, really stable packages and only
> critical updates (I doesn't use Debian myself, so maybe I'm wrong about it).

Or, some might suggest the answer for those that want a 'stable portage
tree' is to provide... wait for it... it's a radical suggestion... a
stable portage tree? :)

Yours, occasionally sarcastically and no disrespect meant -

Ian

-- 
Ian P. Christian ~ http://pookey.co.uk


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable portage tree

[Alex Efros]

[-- Attachment #1: Type: text/plain, Size: 3061 bytes --]

Hi!

On Wed, Aug 16, 2006 at 05:07:46PM +0100, Ian P. Christian wrote:
> Twice you've suggested there are problems, and it's ok because there
> haven't been many. This really isn't the case.  I can't afford to

In my exp and after reading ml I think constant updates in x86 result in 
1-2 issues per year. I think it's ok. I think it's better to get these
issues isolated, after updating 2-3 packages, and with ability to fallback
to previous package versions, than get these issues after massive update
of everything every 6-12 months and without ability to fallback.

Also I'm usually make `emerge --sync` and then wait 2-3 days reading ml
before running `emerge -uDNa world` - only in hope to avoid these
'1-2 issues per year', because if something so bad happens ppl in ml
usually notify about it very quickly.

> systems and test them out properly.  Perhaps giving them a week or two's
> worth of stress testing.

Yeah, I'm doing this 1-2 week stress testing by installing updates on
developers servers first, then on production servers. But this really
needed then some core package updated - linux kernel, perl, mysql, apache -
everybody has own list of critical packages and it isn't too big usually.

> I'm sorry, but that is just crazy talk ;)
> You clearly don't deal with PHP, where a point release can break a LOT
> of things, some things you might not notice by loading 2 or 3 pages from
> a website.

Yeah, you right about me. I don't deal with PHP and I never administrate
more than 5-6 servers. :) But I think it happens sometime, so this
discussion is very interesting for me - I wanna learn other's experience
and be ready for situations where my own experience will not work anymore.

It still isn't clear for me why update strategy for 100 servers differ
from 5-6 servers. I don't believe in 100 servers doing really DIFFERENT
tasks with really different configurations (at least - in all these
servers managed by single admin :)). If most of these server has similar
configurations then it's ease to setup few test servers updated
constantly and have production servers updated with some delay after test
servers.


P.S. About PHP. I don't deal with PHP because of only one reason:
I convince my boss what PHP is too unsecure (Ohh, I feel millions of PHP
fanatics will kill me now :)) and we moved all our PHP apps into
dedicated server, which we specially buy for this task, and I'm not really
think about security and updates of this server - I'm sure it can be hacked
just because of holes in PHP scripts which I can't audit and fix.
This may sounds terribly, but... overall security equal to security of
weakness place, and I don't think my attitude to updating this server
lowering it overall security. Myself, selecting between hacking one of
apache/ssh/qmail services on non-updated-in-12-months server with Hardened
Gentoo and hacking a lot of different (both custom and opensource) PHP apps
on this server will choose PHP without thinking too much. :)

-- 
			WBR, Alex.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable portage tree

[Alex Efros]

Hi!

On Wed, Aug 16, 2006 at 05:46:18PM +0200, Karl Hiramoto wrote:
> You have to understand that people in production environments can not do 
> this.  You can not risk a server being off line every few days..  If you 
> have 10 severs, doing this you would 1-2 hours a week doing updates.  
> With 100 servers, you may need a full time employee just to do updates.

I'm understanding this, and I'm working in production environment. :)
If you've 10+, or even 100 servers, then most of them usually have same
configuration (3-4 different configurations), and you can dedicate 1-2
servers for testing updates before installing them of all servers.

> I think perhaps a good suggestion would be for example:
> Gentoo enterprise release 2006.0  with it's own rsync mirror, then only 
> security update ebuilds, or major bugs get added to this rsync mirror.  
> This release could be timed with a official gentoo live cd release.
> 
> When the admins want to do a major upgrade, they point their rsync 
> mirror to 2007.0   for example.

Yeah, but, as I said before, this require many Gentoo devs dedicated for
this task... and these devs must not be newbies, they must be security
experts and strong QA. For now I don't see enthusiasm from Gentoo devs to
work on this task.

All other solutions like 'update once in 6-12 months' for my experience is
much worse than 'update constantly everything except selected packages'.

-- 
			WBR, Alex.
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Sune Kloppenborg Jeppesen]

[-- Attachment #1: Type: text/plain, Size: 688 bytes --]

On Wednesday 16 August 2006 18:04, Alex Efros wrote:
> Hi!
>
> On Wed, Aug 16, 2006 at 05:46:18PM +0200, Karl Hiramoto wrote:
> Yeah, but, as I said before, this require many Gentoo devs dedicated for
> this task... and these devs must not be newbies, they must be security
> experts and strong QA. For now I don't see enthusiasm from Gentoo devs to
> work on this task.
Currently we don't have the manpower needed for such a task. Some of us worked 
on GLEP 19 about a year ago but it has been dormant since then as we 
encountered quite a few problems.

-- 
Sune Kloppenborg Jeppesen (Jaervosz)
Operational Manager
Gentoo Linux Security Team
http://security.gentoo.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable portage tree

[Sune Kloppenborg Jeppesen]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]

On Wednesday 16 August 2006 17:46, Karl Hiramoto wrote:
> Alex Efros wrote:
> > Yeah, your right. But there simple solution for this: update your servers
> > every 3-4 days, and you will be surprised how ease and quick this task
> > become. You'll need from a couple of seconds to 2-3 minutes in average
> > for such update!
>
> You have to understand that people in production environments can not do
> this.  You can not risk a server being off line every few days..  If you
> have 10 severs, doing this you would 1-2 hours a week doing updates.
> With 100 servers, you may need a full time employee just to do updates.
With 100 servers some should be more or less identical giving you at least a 
few oppertunities to save time.

Previously I used to work for a hosting provider and in my memory we had less 
than one problem per server per year and we didn't even build packages 
centrally.

-- 
Sune Kloppenborg Jeppesen (Jaervosz)
Operational Manager
Gentoo Linux Security Team
http://security.gentoo.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

baselayout was Re: [gentoo-server] Stable portage tree

[Robert Welz]

Jan Meier wrote:
> Hello,
> 
> how is the status of the stable portage tree? Is it already available? 
> 
> I am really interested in it because I am tired of frequently updates on my 
> server just because there is a new version. Doing only security update would 
> be nice.
> 
> Regards
> 
> Jan

I have noticed three updates to baselayout in three days. Is there a 
real reason for that high frequency of updates? I have the problem of 
etc-update on 14 servers and really could spend my time on something 
more productive i.e. learning ldap, fixing sguile and debugging xen for 
nfs. Now I fix all those init.d files all the day.

Just my 2 cents,
Regards,
Robert
-- 
gentoo-server@gentoo.org mailing list

Re: baselayout was Re: [gentoo-server] Stable portage tree

[Mark Rudholm]

Robert Welz wrote:
> Jan Meier wrote:
>> Hello,
>>
>> how is the status of the stable portage tree? Is it already available?
>> I am really interested in it because I am tired of frequently updates
>> on my server just because there is a new version. Doing only security
>> update would be nice.
>>
>> Regards
>>
>> Jan
> 
> I have noticed three updates to baselayout in three days. Is there a
> real reason for that high frequency of updates? I have the problem of
> etc-update on 14 servers and really could spend my time on something
> more productive i.e. learning ldap, fixing sguile and debugging xen for
> nfs. Now I fix all those init.d files all the day.

The general complaint I'm hearing about Gentoo is the lack of
configuration stability.  Updates that aren't backward-compatible
are a pain.  I had to reboot a system that hadn't been booted in
about a year and the modules didn't load because of the changes to
modules.autoload.  I've had to clean up Apache conf files 'cause they
moved.  I've had to deal with moving to the new "modular" xorg (and
try to hunt down all the X tools I used to have).  Not to mention
the baselayout changes...

I used to laugh at http://www.funroll-loops.org/ but lately it
really does seem that the distro is being managed by those on
the young side.

Not that any other Linux distro is any better.  I'm contemplating
going back to BSD, which is my company's standard anyway.

-Mark
-- 
gentoo-server@gentoo.org mailing list

Re: baselayout was Re: [gentoo-server] Stable portage tree

[Jonas Fietz]

Hi

> are a pain.  I had to reboot a system that hadn't been booted in
> about a year and the modules didn't load because of the changes to
> modules.autoload.  I've had to clean up Apache conf files 'cause they
> moved.  I've had to deal with moving to the new "modular" xorg (and
> try to hunt down all the X tools I used to have).  Not to mention
> the baselayout changes...
Well, Xorg was an upstream decision, the config-files for apache where 
simply wrong before, so that had to be fixed, and about the changes in 
modules.autoload, i am not so sure.
But to the people needing a stable portage tree: It is really a totally 
different ideology which is somewhat diametral to what gentoo does. 
Gentoo does _not_ have real releases, which some people, me included, 
think is a good thing. Also, i think if the security-fixes are just 
backported, I personally believe that unless there are many people 
helping with the effort there will be more bugs introduced by this, as 
most of the time the codes might not know the code base as well.
Also, you are complaining about the long list of updates when doing a -u 
somewhat. Those are _real_ dependencies, even if they were just imagined 
by some hallucinating gentoo dev ;). So normally there would not be a 
way around installing them.
But on an infrastructure as big as some are talking about here, there 
usually are few types of servers, so that it can be tested anyway. And 
maybe, those types of companies should be more willing to spend a few 
bucks to the gentoo project, maybe about the new "adopt a gentoo-dev"-page.

Ok, ranted enough ;)

Jonas Fietz


DISCLAIMER: I AM NOT A GENTOO DEV
-- 
gentoo-server@gentoo.org mailing list

Re: baselayout was Re: [gentoo-server] Stable portage tree

[Dice R. Random]

On 8/16/06, rdmurray@bitdance.com <rdmurray@bitdance.com> wrote:
> Just simplifying the etc-update process by having an option to silently
> install files that haven't been locally modified would help a _lot_.
> This was my big complaint about FreeBSD, too....
>
> Unfortunately I'm not doing enough server maintenance work myself these
> days to be able to justify taking the time to cook up some code for this.
>
> --David
> --
> gentoo-server@gentoo.org mailing list
>
>

You want dispatch-conf: http://gentoo-wiki.com/TIP_dispatch-conf
-- 
gentoo-server@gentoo.org mailing list

RE: baselayout was Re: [gentoo-server] Stable portage tree

[Jesse, Rich]

I'll have to check into that.  I still wish folks would adopt sdiff
(instead of diff) when dealing with output intended for human
consumption, which is why I wrote a quickie "ecfg" script to find the
etc-update-able config files show me changes via sdiff instead.

Thanks for the pointer!

Rich

-----Original Message-----
From: Dice R. Random [mailto:dicerandom@gmail.com] 
Sent: Wednesday, August 16, 2006 4:04 PM
To: gentoo-server@lists.gentoo.org
Subject: Re: baselayout was Re: [gentoo-server] Stable portage tree

[snip]
You want dispatch-conf: http://gentoo-wiki.com/TIP_dispatch-conf

-- 
gentoo-server@gentoo.org mailing list

Re: baselayout was Re: [gentoo-server] Stable portage tree

[Brian Kroth]

Jesse, Rich wrote:
> I'll have to check into that.  I still wish folks would adopt sdiff
> (instead of diff) when dealing with output intended for human
> consumption, which is why I wrote a quickie "ecfg" script to find the
> etc-update-able config files show me changes via sdiff instead.

You can do this in /etc/etc-update.conf or /etc/dispatch-conf.conf by 
changing the 'diff="..."' line.  I personally like colordiff.

Brian
-- 
gentoo-server@gentoo.org mailing list

RE: baselayout was Re: [gentoo-server] Stable portage tree

[Jesse, Rich]

Yet another excellent idea!  Thanks, Brian!

unalias ecfg

:)

Rich 

-----Original Message-----
From: Brian Kroth [mailto:bpkroth@wisc.edu] 
Sent: Thursday, August 17, 2006 7:45 AM
To: gentoo-server@lists.gentoo.org
Subject: Re: baselayout was Re: [gentoo-server] Stable portage tree

Jesse, Rich wrote:
> I'll have to check into that.  I still wish folks would adopt sdiff
> (instead of diff) when dealing with output intended for human
> consumption, which is why I wrote a quickie "ecfg" script to find the
> etc-update-able config files show me changes via sdiff instead.

You can do this in /etc/etc-update.conf or /etc/dispatch-conf.conf by 
changing the 'diff="..."' line.  I personally like colordiff.

Brian
-- 
gentoo-server@gentoo.org mailing list

-- 
gentoo-server@gentoo.org mailing list

Re: baselayout was Re: [gentoo-server] Stable portage tree

[Robert Welz]

Dice R. Random wrote:
> On 8/16/06, rdmurray@bitdance.com <rdmurray@bitdance.com> wrote:
>> Just simplifying the etc-update process by having an option to silently
>> install files that haven't been locally modified would help a _lot_.
>> This was my big complaint about FreeBSD, too....
>>
>> Unfortunately I'm not doing enough server maintenance work myself these
>> days to be able to justify taking the time to cook up some code for this.
>>
>> --David
>> -- 
>> gentoo-server@gentoo.org mailing list
>>
>>
> 
> You want dispatch-conf: http://gentoo-wiki.com/TIP_dispatch-conf

That looks great, thank you!

Greetings,
Robert
-- 
gentoo-server@gentoo.org mailing list

Re: baselayout was Re: [gentoo-server] Stable portage tree

[rdmurray]

On Wed, 16 Aug 2006 at 14:03, Dice R. Random wrote:
> You want dispatch-conf: http://gentoo-wiki.com/TIP_dispatch-conf

Thanks!  I see disptach-conf is now mentioned in emerge --help config,
but of course I haven't read that text in a couple years :)

--David
-- 
gentoo-server@gentoo.org mailing list

Re: baselayout was Re: [gentoo-server] Stable portage tree

[Kerin Millar]

On 16/08/06, rdmurray@bitdance.com <rdmurray@bitdance.com> wrote:
> On Wed, 16 Aug 2006 at 21:21, Robert Welz wrote:
> > i.e. learning ldap, fixing sguile and debugging xen for nfs. Now I fix all
> > those init.d files all the day.
>
> Just simplifying the etc-update process by having an option to silently
> install files that haven't been locally modified would help a _lot_.
> This was my big complaint about FreeBSD, too....

I agree with this - it would make things easier. Note that you can
alleviate this problem to a certain extent by the use of
CONFIG_PROTECT_MASK. For example, if you never alter your stock init.d
scripts or, say, udev configuration files then you could put
CONFIG_PROTECT_MASK="/etc/init.d /etc/udev" in /etc/make.conf. Any
files beneath these directories will simply be clobbered by subsequent
installation of packages that place files there (remember,
CONFIG_PROTECT="/etc" is defined by default).

Cheers,

--Kerin
-- 
gentoo-server@gentoo.org mailing list

Re: baselayout was Re: [gentoo-server] Stable portage tree

[kashani]

Robert Welz wrote:

> I have noticed three updates to baselayout in three days. Is there a 
> real reason for that high frequency of updates? I have the problem of 
> etc-update on 14 servers and really could spend my time on something 
> more productive i.e. learning ldap, fixing sguile and debugging xen for 
> nfs. Now I fix all those init.d files all the day.

I'd guess those would be "oops" bugs that they are fixing which is why I 
don't update a package unless it's a glsa or it's been in portage more 
than a week.

kashani
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable portage tree

[Christian Spoo]

[-- Attachment #1.1: Type: text/plain, Size: 172 bytes --]

Just had a look at the changes in baselayout. There were only some
grammatical fixes in some of the init-scripts. Something you even
needn't to reboot for.

Christian

[-- Attachment #1.2: mail.vcf --]
[-- Type: text/x-vcard, Size: 281 bytes --]

begin:vcard
fn:Christian Spoo
n:Spoo;Christian
adr;quoted-printable:;;Am Kaiser 26;W=C3=BCrselen;NRW;52146;Deutschland
email;internet:mail@christian-spoo.info
tel;fax:02405/475071
tel;home:02405/493466
tel;cell:0176/61055475
x-mozilla-html:FALSE
version:2.1
end:vcard


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 888 bytes --]

RE: [gentoo-server] Stable Portage tree

[Jesse, Rich]

Hi Sean,

I think your statement *can* be true, but it doesn't have to be nor is
it by default.  By definition change is not stable and IMHO Gentoo
changes frequently (e.g. portage/ebuilds).  It's really up to the
administrator(s) to accomplish this stability.  Gentoo gives you the
flexibility to keep your installed packages up-to-date or stable, it's
just not stable by default.  BTW, I'm interpreting "stable" here to mean
server and application uptime.

Why do I say "not stable" by default?  I believe that blindly emerging
world every night is asking for stability trouble.  One example I can
think of is portage itself.  At some version recently, the "strict"
FEATURE became the default, which broke several ebuilds that had
previously worked.  I've also had issues with openssh and ssh
commandline switch changes.  Others that come to mind are with the
multimedia stuff like transcode and the codec libs, but those aren't
typically server packages.

But if one doesn't emerge world on a "regular" basis, Gentoo becomes
broken because ebuilds have a limited lifetime and will be deleted from
emerge syncs even though that ebuild is still installed.  By "broken" I
mean that updates -- especially major security updates -- of base
packages can no longer be successfully integrated into a running system
because the admin must run revdep-rebuild after updating these packages
(openssh comes to mind).  But since revdep-rebuild requires the original
ebuilds of affected packages and many packages' ebuilds can age out (be
deleted from portage), revdep-rebuild fails.  I've run into this and was
left with basically a broken system that had to be either rebuilt from
scratch or have an emerge world run.  I chose the former because I had
little faith that the massive emerge world would have been successful.

So what's an Admin supposed to do?  As someone mentioned, I use
PORTDIR_OVERLAY (nee PORTAGE_OVERLAY) in /etc/make.conf.  In my case, my
PORTDIR_OVERLAY is shared via NFS to my two Gentoo desktops'
PORTDIR_OVERLAYs, so all have access to my latest portage
"automatically" without resorting to multiple emerge syncs.  Here's the
script I run every night on my Gentoo server:

rsync -rlt /usr/portage /my/portdir_overlay
# don't use "esync" for max compatibility.
emerge sync
eupdatedb -q

It's simple, but does the job for me.  I've been doing this since my
systems' inceptions, so I'm guaranteed to have the "old" ebuilds on hand
for the necessary revdep-rebuilds.  Note the caveat of the
PORTDIR_OVERLAY lagging one day behind, but that's acceptable for me.

One change I would like to see in Gentoo to improve stability would be
to make the delete of ebuilds in an emerge sync optional, defaulting to
the current "yes".  I'm not saying that an emerge sync should verify
that no ebuilds that are currently installed get deleted (I don't
believe this to be an easy option to implement), just an all-or-nothing
option.  I understand that over time the portage tree grows (my current
overlay is ~100MB larger) and that this would not be good for Gentoo's
rsync servers worldwide, but I'm willing to give up 1GB of disk on my
server for some package/system stability.  I know this was discussed on
the Gentoo-Alpha list (where I got the PORTDIR_OVERLAY idea), but I've
since retired my wonderful Alpha.

Just my $.02 on the whole shmear,
Rich

Rich Jesse                        System/Database Administrator
rich.jesse@quadtechworld.com      QuadTech, Sussex, WI USA




-----Original Message-----
From: Sean Cook [mailto:scook@kinex.net] 
Sent: Friday, September 23, 2005 11:48 PM
To: gentoo-server@lists.gentoo.org
Subject: Re: [gentoo-server] Stable Portage tree

[snip]

Gentoo is stable and is not bleeding edge unless you are using keywords
and unmasking from the stable distro.

[snip]

-- 
gentoo-server@gentoo.org mailing list

[gentoo-server] Stable Portage tree

[Phillip Berry]

Hello,

Just wondering if there has been any progress on the stable portage tree? 

Also, syncing the normal tree removes old versions of ebuilds, obviously this 
is inappropriate for a production environment where for various reasons it is 
sometimes neccessary to stay at an arbitrary version of an application.  The 
loss of the ebuild specific to the legacy version of the application is a 
pain, will the stable tree retain older versions of ebuilds instead of 
removing them?

Also, will security updates ever be backported?

As is i said, i'm just wondering...

Cheers
Phil

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Sune Kloppenborg Jeppesen]

Hi Phil,

On Thursday 22 September 2005 12:00, Phillip Berry wrote:
> Just wondering if there has been any progress on the stable portage tree?
If you're thinking about GLEP 19 nothing much has been accomplished for quite 
a few months now. I think all involved parties have too much to do already. 

> Also, syncing the normal tree removes old versions of ebuilds, obviously
> this is inappropriate for a production environment where for various
> reasons it is sometimes neccessary to stay at an arbitrary version of an
> application.  The loss of the ebuild specific to the legacy version of the
> application is a pain, will the stable tree retain older versions of
> ebuilds instead of removing them?
You could keep them in your own portage (overlay) tree and only sync with the 
official as necessary.

> Also, will security updates ever be backported?
If manpower permits, but with the current manpower situation I think it is 
unlikely.

>
> As is i said, i'm just wondering...

-- 
Sune Kloppenborg Jeppesen (Jaervosz)
Gentoo Linux Security Team
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Phillip Berry]

Hi Sune,

Thankyou for the answer, after a quick search it is indeed the "GLEP 19"  that 
satisfied my query.   I also found the answer to my other question: 

"All ebuilds should remain in the tree for a minimum of one year. This allows 
users to upgrade as infrequently as once per year without risking the stable 
portage tree leaving them behind without an upgrade path."

It's truly unfortunate that that particular effort has lost momentum.

Could i trouble you to point me to some more specific discourse regarding the 
stable portage tree? I wish i could offer my help but I'm afraid i don't have 
enough of an understanding of the depths of Portage to be of any great use...

Cheers
Phil


On Thursday 22 September 2005 23:41, Sune Kloppenborg Jeppesen wrote:
> Hi Phil,
>
> On Thursday 22 September 2005 12:00, Phillip Berry wrote:
> > Just wondering if there has been any progress on the stable portage tree?
>
> If you're thinking about GLEP 19 nothing much has been accomplished for
> quite a few months now. I think all involved parties have too much to do
> already.
>
> > Also, syncing the normal tree removes old versions of ebuilds, obviously
> > this is inappropriate for a production environment where for various
> > reasons it is sometimes neccessary to stay at an arbitrary version of an
> > application.  The loss of the ebuild specific to the legacy version of
> > the application is a pain, will the stable tree retain older versions of
> > ebuilds instead of removing them?
>
> You could keep them in your own portage (overlay) tree and only sync with
> the official as necessary.
>
> > Also, will security updates ever be backported?
>
> If manpower permits, but with the current manpower situation I think it is
> unlikely.
>
> > As is i said, i'm just wondering...
>
> --
> Sune Kloppenborg Jeppesen (Jaervosz)
> Gentoo Linux Security Team
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 3349 bytes --]

Phillip Berry wrote:
> Hi Sune,
> 
> Thankyou for the answer, after a quick search it is indeed the "GLEP 19"  that 
> satisfied my query.   I also found the answer to my other question: 
> 
> "All ebuilds should remain in the tree for a minimum of one year. This allows 
> users to upgrade as infrequently as once per year without risking the stable 
> portage tree leaving them behind without an upgrade path."
> 
> It's truly unfortunate that that particular effort has lost momentum.
> 
> Could i trouble you to point me to some more specific discourse regarding the 
> stable portage tree? I wish i could offer my help but I'm afraid i don't have 
> enough of an understanding of the depths of Portage to be of any great use...

Well, the problem right now is what kind of a route do we want to take?
For example, if Gentoo wanted to try and maintain an enterprise ready
solution to the stable tree issue, I don't think we could do it. On the
other hand, if we wanted to establish a few tools/solutions that provide
some enterprise ready functionality, I think we may be able to do that.

Some ideas I had was just starting small. Bring back the server page on
w.g.o and start adding documents on how to manage Gentoo servers in an
enterprise setting. It'd probably be tied to the docs team in some
fashion. Right now, there are a couple of docs on how to setup specific
applications, but none really about server administration. I know for a
fact that I do things in a specific way to maintain the infra servers
without breakage. I'd like to find time and make a doc about that.

Next, come out with a plan or goals we would like to achieve.
Stabilizing the tree isn't an easy task. Sure, we could use the
snapshots used for 2005.1, etc, but maintainence and QA are the biggest
problem. Dealing with security updates for example is one issue. Do we
tackle the problem by doing backport patches, or do we just version
bump, or do we offer both? Parts of that are more for a third party
entity to try and resolve because of the resources we'd need. Seeing a
mini fork of Gentoo for the enterprise is one path I see happening down
the road. Reason being, specific things would need to be changed to make
Gentoo *really* enterprise ready, such things that would disrupt the
current Gentoo's development. I would not want to see such things
happening and hindering what we already are good at. Its almost like the
Ubuntu project, but not funded by a billionaire :).

Anyways, ways to get this rolling again? Start combining/creating
documentation to help server admins out there to manage Gentoo better.
Come up with a set of goals/projects for us to attain and prioritize
them. Start getting folks to work on them.

In the past, most of us either got busy with other projects or real life
issues, or the folks we had just kind of disappeared. I'm planning on
getting this rolling again, but things always come up. Also, folks
always come up with 10 different ways to solve the same problem. We'll
never come up with something that makes everyone happy.

Anyways, I'd love to hear your feedback and opinions!

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable Portage tree

[Sune Kloppenborg Jeppesen]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

On Thursday 22 September 2005 18:27, Lance Albertson wrote:
> Well, the problem right now is what kind of a route do we want to take?
> For example, if Gentoo wanted to try and maintain an enterprise ready
> solution to the stable tree issue, I don't think we could do it. On the
> other hand, if we wanted to establish a few tools/solutions that provide
> some enterprise ready functionality, I think we may be able to do that.
Unfortunately I think you're right. While I would like to contribute to the 
maintainance of a stable Portage tree, it is definately beyond what a handful 
of devs can accomplish in the long run.

New docs on the other hand should be a better priority to start out with. 

> Anyways, I'd love to hear your feedback and opinions!
And I'd love to help with the docs:-)

-- 
Sune Kloppenborg Jeppesen
Gentoo Linux Security Team

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable Portage tree

[Eduardo Tongson]

> > Well, the problem right now is what kind of a route do we want to take?
> > For example, if Gentoo wanted to try and maintain an enterprise ready
> > solution to the stable tree issue, I don't think we could do it. On the
> > other hand, if we wanted to establish a few tools/solutions that provide
> > some enterprise ready functionality, I think we may be able to do that.
> Unfortunately I think you're right. While I would like to contribute to the
> maintainance of a stable Portage tree, it is definately beyond what a handful
> of devs can accomplish in the long run.
>
> New docs on the other hand should be a better priority to start out with.
>
> > Anyways, I'd love to hear your feedback and opinions!
> And I'd love to help with the docs:-)

Good to see it's gaining momentum again. Volunteers can probably help
in some way if devs let them know what specifically needs helping.

Good luck :)

--ed

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 1692 bytes --]

Eduardo Tongson wrote:
>>>Well, the problem right now is what kind of a route do we want to take?
>>>For example, if Gentoo wanted to try and maintain an enterprise ready
>>>solution to the stable tree issue, I don't think we could do it. On the
>>>other hand, if we wanted to establish a few tools/solutions that provide
>>>some enterprise ready functionality, I think we may be able to do that.
>>
>>Unfortunately I think you're right. While I would like to contribute to the
>>maintainance of a stable Portage tree, it is definately beyond what a handful
>>of devs can accomplish in the long run.
>>
>>New docs on the other hand should be a better priority to start out with.
>>
>>
>>>Anyways, I'd love to hear your feedback and opinions!
>>
>>And I'd love to help with the docs:-)
> 
> 
> Good to see it's gaining momentum again. Volunteers can probably help
> in some way if devs let them know what specifically needs helping.

I looked around a bit and saw we have a sysadmin page [1] talking about
various sysadmin type things. At this point I'm wanting to ask you as
fellow gentoo server admins, what do *you* want to see on the main
Gentoo server site? What is missing now that we need to fill in the
gaps? Any specific documentation we should cover? What goals/projects
should we look at developing? Do any of you already have some tools
developed that maybe we could host/help with?

[1] http://www.gentoo.org/doc/en/index.xml?catid=sysadmin

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable Portage tree

[Ian P. Christian]

[-- Attachment #1: Type: text/plain, Size: 1196 bytes --]

On Thursday 22 September 2005 18:57, Lance Albertson wrote:
> I looked around a bit and saw we have a sysadmin page [1] talking about
> various sysadmin type things. At this point I'm wanting to ask you as
> fellow gentoo server admins, what do *you* want to see on the main
> Gentoo server site? What is missing now that we need to fill in the
> gaps? Any specific documentation we should cover? What goals/projects
> should we look at developing? Do any of you already have some tools
> developed that maybe we could host/help with?

I would love to see more work done in this area.  I would love to see more 
documentation on running more then one machine.  I run lots of gentoo 
servers, and maintaining them is so much work.  Docuemntation on setting up a 
central compile server, a package distribution system, how to pull reports 
off what needs updating on what servers from a central place, this kind of 
thing - that would be amazingly useful.  I think more and more people are 
starting to consider gentoo for servers, which is fantastic, and effort in 
this area will only increase the number of users.

Kind Regards,

-- 
Ian P. Christian ~ http://pookey.co.uk

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable Portage tree

[Ramon van Alteren]

+1 from me
That's definitly stuff I would like to read about.

Ian P. Christian wrote:

>I would love to see more work done in this area.  I would love to see more 
>documentation on running more then one machine.  I run lots of gentoo 
>servers, and maintaining them is so much work.  Docuemntation on setting up a 
>central compile server, a package distribution system, how to pull reports 
>off what needs updating on what servers from a central place, this kind of 
>thing - that would be amazingly useful.  I think more and more people are 
>starting to consider gentoo for servers, which is fantastic, and effort in 
>this area will only increase the number of users.
>  
>
Ramon

-- 
Change what you're saying,
Don't change what you said

The Eels

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Linux GNUbie]

On Thu, 2005-09-22 at 19:18 +0100, Ian P. Christian wrote:
> 
> I would love to see more work done in this area.  I would love to see more 
> documentation on running more then one machine.  I run lots of gentoo 
> servers, and maintaining them is so much work.  Docuemntation on setting up a 
> central compile server, a package distribution system, how to pull reports 
> off what needs updating on what servers from a central place, this kind of 
> thing - that would be amazingly useful.  I think more and more people are 
> starting to consider gentoo for servers, which is fantastic, and effort in 
> this area will only increase the number of users.

Speaking of this issue, actually I'm planning to setup a box with a 24
SATA HDDs RAID5 array on a dual Opteron box intended to run Subversion
to handle huge video files before this year end.  I'm still afraid of
using Gentoo because I'm not that yet convinced on it's stability
especially when dealing file servers.  I don't want to gamble and take a
risk of my terabytes files but I also know that taking a risk is part of
our life.  Honestly, I'm choosing between Gentoo and CentOS 4.1.

Good luck.

---
Linux GNUbie <gnubieATgmailDOTcom>

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Eduardo Tongson]

> Speaking of this issue, actually I'm planning to setup a box with a 24
> SATA HDDs RAID5 array on a dual Opteron box intended to run Subversion
> to handle huge video files before this year end.  I'm still afraid of
> using Gentoo because I'm not that yet convinced on it's stability
> especially when dealing file servers.  I don't want to gamble and take a
> risk of my terabytes files but I also know that taking a risk is part of
> our life.  Honestly, I'm choosing between Gentoo and CentOS 4.1.
>
> Good luck.
>

What makes CentOS 4.1 "more" stable than gentoo care to elaborate?

--ed

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Linux GNUbie]

On Fri, 2005-09-23 at 17:01 +0000, Eduardo Tongson wrote:
> 
> What makes CentOS 4.1 "more" stable than gentoo care to elaborate?

The beauty of running a binary based GNU/Linux distribution not
particularly on CentOS alone but in general (includes Debian, Red Hat,
SuSE, Mandriva, etc.) is before the updates are released to the public
it has been tested and compiled for use in enterprise production use.
When I say updates here it doesn't mean of a new version number of the
packages.  Instead, the security and bug fixes for the packages
installed in the system.

This is also the issue raised by Mr. Phillip Berry who started this
thread.  I for one wants to have a Gentoo system in an enterprise
production use.  This is not about bleeding edge, optimization,
performance and control.

Try to think of managing 100 servers all running Gentoo on 5 to 10
different offices/companies with different services and customed
applications in production use.  Do you think you can still manage all
of them?

---
Linux GNUbie <gnubieATgmailDOTcom>

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Sean Cook]

> On Fri, 2005-09-23 at 17:01 +0000, Eduardo Tongson wrote:
>>
>> What makes CentOS 4.1 "more" stable than gentoo care to elaborate?
>
> The beauty of running a binary based GNU/Linux distribution not
> particularly on CentOS alone but in general (includes Debian, Red Hat,
> SuSE, Mandriva, etc.) is before the updates are released to the public
> it has been tested and compiled for use in enterprise production use.
> When I say updates here it doesn't mean of a new version number of the
> packages.  Instead, the security and bug fixes for the packages
> installed in the system.

so obviously we need to exclude operating systems like FreeBSD and OpenBSD
for any type of production use because they do not have binary packages?

>
> This is also the issue raised by Mr. Phillip Berry who started this
> thread.  I for one wants to have a Gentoo system in an enterprise
> production use.  This is not about bleeding edge, optimization,
> performance and control.

Gentoo is stable and is not bleeding edge unless you are using keywords
and unmasking from the stable distro.

>
> Try to think of managing 100 servers all running Gentoo on 5 to 10
> different offices/companies with different services and customed
> applications in production use.  Do you think you can still manage all
> of them?

this assumes that one could manage 100 server of any distro in this
manor... in fact gentoo lends itself to this environment better that most
linux distrobutions because of portage overlays that allow you to tag
specific machines for beta and production based on packages.  It also
allow you to build multiple packages and distribute them from a single
source after testing.

By the time you get to 20 servers anyway if you are using cvs (or other)
to maintain config files and certain aspects of the os you are asking for
trouble.

That being said may we put this thread to rest?  Gentoo is perfectly
capable of running in a production environment, I have personally have 15
servers all running gentoo 2005.1.  I also have several debian servers and
freebsd servers.... it is all simply a matter of comfort.  you should
never put any system into a production environment unless your comfortable
admining that system... period.  So lets stop the pissing contest of my
distro can beat up your distro and get to the real issues.

>
> ---
> Linux GNUbie <gnubieATgmailDOTcom>
>
> --
> gentoo-server@gentoo.org mailing list
>
>


-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Linux GNUbie]

On Sat, 2005-09-24 at 00:48 -0400, Sean Cook wrote:
> 
> so obviously we need to exclude operating systems like FreeBSD and OpenBSD
> for any type of production use because they do not have binary packages?

They're not GNU/Linux, they're *BSDs.  I was only referring to GNU/Linux
distributions.

> Gentoo is stable and is not bleeding edge unless you are using keywords
> and unmasking from the stable distro.

Granted that Gentoo is stable for production use because I believe
you're not the only systems administrator that use it in production
environment.  I am just wondering if the services you're running on your
Gentoo systems are customed applications which might be requiring
commercial databases like Oracle and IBM DB2, and also requires a
particular version of tools and libraries of the said applications.

> this assumes that one could manage 100 server of any distro in this
> manor... in fact gentoo lends itself to this environment better that most
> linux distrobutions because of portage overlays that allow you to tag
> specific machines for beta and production based on packages.  It also
> allow you to build multiple packages and distribute them from a single
> source after testing.

I don't have any problems with the rest of the packages to update in
Gentoo.  I personally like the package management of Gentoo and I'm very
much excited and hopefully be confident to use it in production.  Again,
I am just worried if my customed applications will not work one day
simply because the tools and libraries that my applications are very
much dependent were updated to the latest versions which it shouldn't
happen.  I only need the security and bug fixes updates for the packages
that my customed applications are dependent.

> freebsd servers.... it is all simply a matter of comfort.  you should
> never put any system into a production environment unless your comfortable
> admining that system... period.  So lets stop the pissing contest of my
> distro can beat up your distro and get to the real issues.

You're right in here.  Hopefully I'll be confident and comfortable
enough to use Gentoo in my production environment.

Thanks.

---
Linux GNUbie <gnubieATgmailDOTcom>

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Phillip Berry]

On Saturday 24 September 2005 14:48, Sean Cook wrote:
> > On Fri, 2005-09-23 at 17:01 +0000, Eduardo Tongson wrote:
> >> What makes CentOS 4.1 "more" stable than gentoo care to elaborate?
> >
> > The beauty of running a binary based GNU/Linux distribution not
> > particularly on CentOS alone but in general (includes Debian, Red Hat,
> > SuSE, Mandriva, etc.) is before the updates are released to the public
> > it has been tested and compiled for use in enterprise production use.
> > When I say updates here it doesn't mean of a new version number of the
> > packages.  Instead, the security and bug fixes for the packages
> > installed in the system.
>
> so obviously we need to exclude operating systems like FreeBSD and OpenBSD
> for any type of production use because they do not have binary packages?
>
> > This is also the issue raised by Mr. Phillip Berry who started this
> > thread.  I for one wants to have a Gentoo system in an enterprise
> > production use.  This is not about bleeding edge, optimization,
> > performance and control.
>
> Gentoo is stable and is not bleeding edge unless you are using keywords
> and unmasking from the stable distro.
>
> > Try to think of managing 100 servers all running Gentoo on 5 to 10
> > different offices/companies with different services and customed
> > applications in production use.  Do you think you can still manage all
> > of them?
>
> this assumes that one could manage 100 server of any distro in this
> manor... in fact gentoo lends itself to this environment better that most
> linux distrobutions because of portage overlays that allow you to tag
> specific machines for beta and production based on packages.  It also
> allow you to build multiple packages and distribute them from a single
> source after testing.
>
> By the time you get to 20 servers anyway if you are using cvs (or other)
> to maintain config files and certain aspects of the os you are asking for
> trouble.
>
> That being said may we put this thread to rest?  Gentoo is perfectly
> capable of running in a production environment, I have personally have 15
> servers all running gentoo 2005.1.  I also have several debian servers and
> freebsd servers.... it is all simply a matter of comfort.  you should
> never put any system into a production environment unless your comfortable
> admining that system... period.  So lets stop the pissing contest of my
> distro can beat up your distro and get to the real issues.
>
> > ---
> > Linux GNUbie <gnubieATgmailDOTcom>
> >
> > --
> > gentoo-server@gentoo.org mailing list

Hello,

With respect, i would rather not put this thread to rest just yet. An 
interesting discourse has been created, unfortunately it was temporarily 
dragged into a distribution war, that is over now.  The original thread is 
100% on topic, myself, and various other gentlemen have expressed our 
interest in some sort of stable portage tree.

I would like to take this moment to explore the core problem that i have 
experienced, because i am certain that my original position has been lost 
within the recent exchange.  

My wish has little to do with 'enterprise support' or Gentoos readiness for a 
production environment, it is simply an request for improvement on how those 
of us who do run Gentoo on servers manage them.

After some thought, it occurs to me that the solution that requires minimal 
work from the already overworked volunteers may be as simple as some 
reasonable tools to manage existing portage functionality, consider the 
following, based upon Svens advice to manually backport ebuilds ;

1. Automated generation of a portage overlay based on the currently installed 
packages, these would then be safe from sync. The tool would need to be able 
to drag ebuilds out of some sort of archive for existing machines.
2. An easy way to introduce and manage and remove ebuilds in the overlay

Wouldn't that be a reasonably stable tree?  It means there is no major 
architectural changes to Gentoo, but then allows administrators the 
flexibility to determine what is stable, what version to sit on and when to 
upgrade.

Thankyou for reading this far, i hope my suggestions and ideas are both 
reasonable and useful.

Kind Regards
Phil
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]


> 1. Automated generation of a portage overlay based on the currently installed 
> packages, these would then be safe from sync. The tool would need to be able 
> to drag ebuilds out of some sort of archive for existing machines.
> 2. An easy way to introduce and manage and remove ebuilds in the overlay

How is this different than making a tree based off of the snapshots
created by releng for each release? Now, if you want to add newer stuff
later on, it would be nice to have a tool to grab all the nessary
ebuilds required for that newer ebuild. Perhaps thats what you're
talking about.

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable Portage tree

[Phillip Berry]

On Monday 26 September 2005 00:46, Lance Albertson wrote:
> > 1. Automated generation of a portage overlay based on the currently
> > installed packages, these would then be safe from sync. The tool would
> > need to be able to drag ebuilds out of some sort of archive for existing
> > machines. 2. An easy way to introduce and manage and remove ebuilds in
> > the overlay
>
> How is this different than making a tree based off of the snapshots
> created by releng for each release? Now, if you want to add newer stuff
> later on, it would be nice to have a tool to grab all the nessary
> ebuilds required for that newer ebuild. Perhaps thats what you're
> talking about.

Hello,

I believe the snapshot only contains the newest versions of ebuilds?

Perhaps i should be clearer:

When initially run on an existing server it would identify all currently 
installed applications and copy the applications associated ebuild (version 
specific) to the portage overlay.  This would create a portage tree specific 
to that machine. I imagine this is pretty easily done with a shell script.

It would then be nice to have either a tool or for emerge to be somewhat aware 
of the fact that there is a separate tree and be able to add and remove 
ebuilds and manage the new tree at will. So `emerge -u postfix` would place 
the new courier ebuild into the new tree as well as upgrading, etc,etc.

It's nothing grand, just automation of what could be with effort done by hand. 

It would serve three purposes:

1. Protects the ebuilds of applications the admin can't/doesn't want to 
upgrade from dissappearing.
2. Gives the admin more control over what gets upgraded, i realise 
that /etc/portage/ does this but requires explicit action to protect a 
version, this is passive.
3. As an fringe benefit is easily backed up and allows another machine to be 
easily brought up with exactly the same applications and versions.

I feel that i'm not articulating my thoughts very well and i apologise, 
perhaps i'll develop something for myself and see if it works before i 
trouble you all anymore.

Cheers
Phil




-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Ben Munat]

Just to throw my two-devil's-advocate-cents in here, I maintain a gentoo server and -- 
though I certainly appreciate getting well-tested, problem-free updates -- I think the 
biggest problem gentoo faces is how far behind the curve it is. It's just plain 
embarassing that there are still no (unmasked) versions of PHP5, MySQL4.1 or 5, or Java 
5/Tomcat 5.5 in portage... all of which have been freely available for over a year.

I understand that it's an all volunteer effort and I really do appreciate all the work of 
the devs. However, as counter-point to this discussion of a stable tree, I am wondering 
aloud if the developers' time might be better spent catching gentoo up with the rest of 
the world (esp. FreeBSD).

Like I say, just my .02...

Ben
-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Sven Vermeulen]

[-- Attachment #1: Type: text/plain, Size: 1470 bytes --]

On Sun, Sep 25, 2005 at 09:17:07AM -0700, Ben Munat wrote:
> Just to throw my two-devil's-advocate-cents in here, I maintain a gentoo 
> server and -- though I certainly appreciate getting well-tested, 
> problem-free updates -- I think the biggest problem gentoo faces is how far 
> behind the curve it is. It's just plain embarassing that there are still no 
> (unmasked) versions of PHP5, MySQL4.1 or 5, or Java 5/Tomcat 5.5 in 
> portage... all of which have been freely available for over a year.

This is the other side of "enterprise Gentoo". Some enterprises need the
fast pace that open source delivers. 

Anyway, there are reasons why MySQL 4.1/5 are not available in the /stable/
tree yet - you can however find the ebuilds in Portage. They are either
marked as ~arch (ebuild needs testing) or listed in package.mask (software
needs testing). Some of these versions (like 5) are said by MySQL AB
themselves that they shouldn't be used for production use yet. Gentoo
generally follows their advise. After all, who are we to question the
developers of MySQL :)

Same for PHP5. 

Yes, J2EE support with Gentoo is a bit behind but I'm sure this'll change in
the near future.

Wkr,
      Sven Vermeulen

-- 
  Gentoo Foundation Trustee          |  http://foundation.gentoo.org
  Gentoo Documentation Project Lead  |  http://www.gentoo.org/proj/en/gdp
  Gentoo Council Member  

  The Gentoo Project   <<< http://www.gentoo.org >>>

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable Portage tree

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 1064 bytes --]

Ben Munat wrote:
> Just to throw my two-devil's-advocate-cents in here, I maintain a gentoo
> server and -- though I certainly appreciate getting well-tested,
> problem-free updates -- I think the biggest problem gentoo faces is how
> far behind the curve it is. It's just plain embarassing that there are
> still no (unmasked) versions of PHP5, MySQL4.1 or 5, or Java 5/Tomcat
> 5.5 in portage... all of which have been freely available for over a year.
> 

I now that for example Debian and Fedore do not support sun-jdk at all
because they can offer only free as in freedom software. The free java
is still months or years away from supporting 1.4. Tomcat 5.5 of course
is something that we should get to the main tree. It has been sitting in
the java experimental tree for quite a while. One of the reasons Tomcat
5.5 has not progressed is that our Tomcat maintainer disappeared a while
ago. I think Debian only recently added Tomcat 4 to main or something
like that.

>
> Like I say, just my .02...
> 
> Ben

Regards,
Petteri Räty


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: [gentoo-server] Stable Portage tree

[Eduardo Tongson]

> I believe the snapshot only contains the newest versions of ebuilds?

No. see
<http://gentoo.osuosl.org/releases/snapshots/2005.1/>

>
> Perhaps i should be clearer:
> ...
> ...
>
> 1. Protects the ebuilds of applications the admin can't/doesn't want to
> upgrade from dissappearing.
> 2. Gives the admin more control over what gets upgraded, i realise
> that /etc/portage/ does this but requires explicit action to protect a
> version, this is passive.
> 3. As an fringe benefit is easily backed up and allows another machine to be
> easily brought up with exactly the same applications and versions.
>

As others have said PORTDIR_OVERLAY can accomplish some of what
you require especially (1) and (3), even RSYNC_EXCLUDEFROM can be
used. Though portage.mask can handle (2), a tool to manage all of them
and even extend it to work with a releng snapshot is a sure treat.

>
> I feel that i'm not articulating my thoughts very well and i apologise,
> perhaps i'll develop something for myself and see if it works before i
> trouble you all anymore.

No problem. mailing lists are meant for these.
Good luck with the tool and keep us posted :-)

--ed

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Sven Vermeulen]

[-- Attachment #1: Type: text/plain, Size: 2593 bytes --]

On Sat, Sep 24, 2005 at 11:29:56AM +0800, Linux GNUbie wrote:
> The beauty of running a binary based GNU/Linux distribution not
> particularly on CentOS alone but in general (includes Debian, Red Hat,
> SuSE, Mandriva, etc.) is before the updates are released to the public
> it has been tested and compiled for use in enterprise production use.

You have something quite similar with Gentoo. After all, in what way does
testing an ebuild or testing a binary package differ? The ebuild uses the
same source for all testers. The only difference is that ebuilds can behave
differently based on your USE flags (and a few other variables in your
make.conf - but USE is the most notable one).

Some people ask me if their mega-size-USE-flag affects performance in a
negative way. It doesn't, assuming that you still understand why you set
certain USE flags in that variable. Don't forget, the behavior of most
ebuilds doesn't change with each USE flag change - only to those the ebuild
listens to.

For instance, if you check the USE flags for mysql, you'll notice that it
only is affected by:
  berkdb, big-tables, debug, doc, minimal, perl, readline, ssl, static, tcpd

How many MySQL users do you think have tested the result of the MySQL ebuild
using your USE flags? That'll be quite a lot - and we are not only talking
about Gentoo users, but also general MySQL users who manually build the
sources (many MySQL production users do this this way) using the configure
--with-blabla/--without-blabla tags that are mapped onto the USE flags in
Gentoo.

Anyway, what I wanted to state was that you can easily test packages
yourself. Build the package on your buildserver, deploy that package on your
development machine, stage it through the necessary pillars (like testing,
staging, simulation) to eventually deploy the new package on your production
system.

This is a pattern used on many environments, even by those using pure binary
packages. The difference with Gentoo is that they are immediately pinned to
that binary package while Gentoo allows you to improve the package without
much effort, slim down the installation to what you need (effectively
decreasing the possibility of a security flaw or software bug affecting you)
and even optimize the build to your system environment.

Wkr,
      Sven Vermeulen

-- 
  Gentoo Foundation Trustee          |  http://foundation.gentoo.org
  Gentoo Documentation Project Lead  |  http://www.gentoo.org/proj/en/gdp
  Gentoo Council Member  

  The Gentoo Project   <<< http://www.gentoo.org >>>

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable Portage tree

[Patrick Lauer]

[-- Attachment #1: Type: text/plain, Size: 905 bytes --]

On Sat, 2005-09-24 at 00:16 +0800, Linux GNUbie wrote:
> Speaking of this issue, actually I'm planning to setup a box with a 24
> SATA HDDs RAID5 array on a dual Opteron box intended to run Subversion
> to handle huge video files before this year end.  I'm still afraid of
> using Gentoo because I'm not that yet convinced on it's stability
> especially when dealing file servers. 
Well ... that's always hard to predict :-)
But that's why you have a clone of that system for update testing and a
full backup, right? :-)
>  I don't want to gamble and take a
> risk of my terabytes files but I also know that taking a risk is part of
> our life.  Honestly, I'm choosing between Gentoo and CentOS 4.1.
Uhm ... what would make CentOS more qualified for "heavy lifting" than, say, Ubuntu or RedHat?

wkr,
Patrick
Gentoo Evangelist :-)
-- 
Stand still, and let the rest of the universe move

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-server] Stable Portage tree

[A. Khattri]

On Sat, 24 Sep 2005, Linux GNUbie wrote:

> Honestly, I'm choosing between Gentoo and CentOS 4.1.

CentOS would *never* make my shortlist for server OS ;-)

I have been running Gentoo servers in production for quite sometime
without any stability problems of any kind - no crashes.

I can only pass on my experience - if you want guarantees and support, you
better get out your wallet and go for RHEL or Suse.


-- 

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Phillip Berry]

On Friday 23 September 2005 03:03, Sune Kloppenborg Jeppesen wrote:
> On Thursday 22 September 2005 18:27, Lance Albertson wrote:
> > Well, the problem right now is what kind of a route do we want to take?
> > For example, if Gentoo wanted to try and maintain an enterprise ready
> > solution to the stable tree issue, I don't think we could do it. On the
> > other hand, if we wanted to establish a few tools/solutions that provide
> > some enterprise ready functionality, I think we may be able to do that.
>
> Unfortunately I think you're right. While I would like to contribute to the
> maintainance of a stable Portage tree, it is definately beyond what a
> handful of devs can accomplish in the long run.
>
> New docs on the other hand should be a better priority to start out with.
>
> > Anyways, I'd love to hear your feedback and opinions!
>
> And I'd love to help with the docs:-)

Hello,

Could some point me to, or provide a more specific breakdown of some of the 
roles and tasks that would need to be tended to?

Forgive me if i appear ignorant to the problems and issues with maintaining an 
enterprise ready tree but in my mind running a stable tree would involve :

1. Identifiying a version of a package that is stable
2. Marking that package as stable
3. Pushing that package to the rsync servers
4. ?
5. ?
6. ?

What are the items that are missing from my list?

Cheers
Phil

-- 
gentoo-server@gentoo.org mailing list

Re: [gentoo-server] Stable Portage tree

[Patrick Lauer]

[-- Attachment #1: Type: text/plain, Size: 1312 bytes --]

On Fri, 2005-09-23 at 11:32 +1000, Phillip Berry wrote:
> Hello,
> 
> Could some point me to, or provide a more specific breakdown of some of the 
> roles and tasks that would need to be tended to?
> 
> Forgive me if i appear ignorant to the problems and issues with maintaining an 
> enterprise ready tree but in my mind running a stable tree would involve :
> 
> 1. Identifiying a version of a package that is stable
keep upgrades to a reasonable minimum ... "never touch a running system"
> 2. Marking that package as stable
that includes lots of testing. Then some more testing. ...
Also each major release (2005.0/2005.1/2006.0/...) would most likely be
its own branch and need testing ...
Did I mention QA and testing? ;-)
> 3. Pushing that package to the rsync servers
Why rsync? updates for an "enterprise" tree should be infrequent enough
for tarballs to be easier
(less overhead, easier to see what needs to be fetced, ...)
> 4. ?
Backport security fixes to older versions?
> 5. ?
> 6. ?
> 
> What are the items that are missing from my list?
That would only provide a stable ebuild base.
Extra items such as reliable support etc. aren't even mentioned here but
would most likely be needed / very useful.

Patrick
-- 
Stand still, and let the rest of the universe move

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-server] Stable Portage tree

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 3496 bytes --]

Phillip Berry wrote:
> On Friday 23 September 2005 03:03, Sune Kloppenborg Jeppesen wrote:
> 
>>On Thursday 22 September 2005 18:27, Lance Albertson wrote:
>>
>>>Well, the problem right now is what kind of a route do we want to take?
>>>For example, if Gentoo wanted to try and maintain an enterprise ready
>>>solution to the stable tree issue, I don't think we could do it. On the
>>>other hand, if we wanted to establish a few tools/solutions that provide
>>>some enterprise ready functionality, I think we may be able to do that.
>>
>>Unfortunately I think you're right. While I would like to contribute to the
>>maintainance of a stable Portage tree, it is definately beyond what a
>>handful of devs can accomplish in the long run.
>>
>>New docs on the other hand should be a better priority to start out with.
>>
>>
>>>Anyways, I'd love to hear your feedback and opinions!
>>
>>And I'd love to help with the docs:-)
> 
> 
> Hello,
> 
> Could some point me to, or provide a more specific breakdown of some of the 
> roles and tasks that would need to be tended to?
> 
> Forgive me if i appear ignorant to the problems and issues with maintaining an 
> enterprise ready tree but in my mind running a stable tree would involve :
> 
> 1. Identifiying a version of a package that is stable
> 2. Marking that package as stable
> 3. Pushing that package to the rsync servers
> 4. ?
> 5. ?
> 6. ?

The idea I had (at least initially), was using the snapshot releng
builds for their release as our base tree to use for a release. After
they finalize the tree, I'd see a group of folks going through and doing
another round of QA and fixing any more bugs that may crop up. After its
been established as a 'good' tree, I'd see us releasing that as
something like 2005.1E or something like that. That part of a 'stable'
tree is relatively easy to do aside from any issues cropping up from the
QA section.

The real fun begins during the maintenance phase where GLSAs, and
critical software (data crippling type of things) updates need to be
updated in it. This is where we'd need to decide whether to go the back
port route, or just force folks to upgrade if the GLSA affects it.
Essentially, our group would be managing their own tree. It sounds
fairly simple, but to ensure a level of quality, we'd need to test the
new ebuild/upgrade in some type of QA environment (which we currently
don't have enough good man power for).

Next, say when the next release comes out (2006.0E), we'd have to come
up with a clean upgrade path to ensure no breakages. This is the part
that will require a lot of time and effort. Ideally, it'd be nice if we
came up with a helper script to 'fix' things as the upgrade happens.

Last, we'd need to decide how long to keep a particular tree updated. If
we went on two releases per year, after 2 years we'd have 4 trees to
keep up to date and come up with upgrade plans for each of those.

Needless to say, doing it the 'right' way isn't very easy to do. Thats
one of the things our server team would need to establish is what kind
of level do we want to maintain. Thats kind of where the idea of a third
party would come into play and possibly help fund a few folks to do this
kind of work as a job :).

Thoughts on that rough plan?

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable Portage tree

[Patrick Lauer]

[-- Attachment #1: Type: text/plain, Size: 3018 bytes --]

On Fri, 2005-09-23 at 11:20 -0500, Lance Albertson wrote:
> The idea I had (at least initially), was using the snapshot releng
> builds for their release as our base tree to use for a release. After
> they finalize the tree, I'd see a group of folks going through and doing
> another round of QA and fixing any more bugs that may crop up. 
That sounds reasonable. I suggest that new versions of ebuilds should be
~ARCHed so that dedicated updates are easier (no overlays etc.)
> After its
> been established as a 'good' tree, I'd see us releasing that as
> something like 2005.1E or something like that. That part of a 'stable'
> tree is relatively easy to do aside from any issues cropping up from the
> QA section.
Agreed
> The real fun begins during the maintenance phase where GLSAs, and
> critical software (data crippling type of things) updates need to be
> updated in it. This is where we'd need to decide whether to go the back
> port route, or just force folks to upgrade if the GLSA affects it.
I'd say upgrade - backports would consume an extreme amount of manpower
while testing still needs to be done by the end-user anyway. 
> Essentially, our group would be managing their own tree. It sounds
> fairly simple, but to ensure a level of quality, we'd need to test the
> new ebuild/upgrade in some type of QA environment (which we currently
> don't have enough good man power for).
That's a bit of a chicken-and-egg problem ;-)
> Next, say when the next release comes out (2006.0E), we'd have to come
> up with a clean upgrade path to ensure no breakages. This is the part
> that will require a lot of time and effort. Ideally, it'd be nice if we
> came up with a helper script to 'fix' things as the upgrade happens.
So do we offer upgrades for 2005.1E or do we "force" people to upgrade at least bi-yearly?

> Last, we'd need to decide how long to keep a particular tree updated. If
> we went on two releases per year, after 2 years we'd have 4 trees to
> keep up to date and come up with upgrade plans for each of those.
I think 2 years is reasonable, any users that wish to keep a tree longer 
should be prepared to support that themselves (and if that only means 
hiring three gentoo devs fulltime ;-) )

> Needless to say, doing it the 'right' way isn't very easy to do. Thats
> one of the things our server team would need to establish is what kind
> of level do we want to maintain. Thats kind of where the idea of a third
> party would come into play and possibly help fund a few folks to do this
> kind of work as a job :).
Agreed. In the beginning a "static" tree would be a nice start though
> Thoughts on that rough plan?
(1) Create a 2005.1-static tree
(2) start adding GLSA updates as they come up
(3) add new ebuilds ~arch so that updates are user-controlled
(4) slowly announce our plans and pull in more helpers
(5) create an extended, more precise roadmap ;-)

wkr,
Patrick 

-- 
Stand still, and let the rest of the universe move

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-server] Stable Portage tree

[Sven Vermeulen]

[-- Attachment #1: Type: text/plain, Size: 1444 bytes --]

On Thu, Sep 22, 2005 at 11:27:15AM -0500, Lance Albertson wrote:
> Some ideas I had was just starting small. Bring back the server page on
> w.g.o and start adding documents on how to manage Gentoo servers in an
> enterprise setting. It'd probably be tied to the docs team in some
> fashion. Right now, there are a couple of docs on how to setup specific
> applications, but none really about server administration. I know for a
> fact that I do things in a specific way to maintain the infra servers
> without breakage. I'd like to find time and make a doc about that.

The GDP would really like to extend the current system administration
documentation with more expert-like guides/redbooks/technical docs. It isn't
that easy to find volunteers though and because these documents are probably
very specific (apart from the Gentoo Security Guide which is also definitely
a must-read for administrators) most GDP members can't validate/accept bug
reports.

In other words, we do need someone to fall back to when we can't resolve a
bugreport ourselves. Think of it as if the GDP is first/second line and the
maintainer of the document is third line support.

Wkr,
      Sven Vermeulen


-- 
  Gentoo Foundation Trustee          |  http://foundation.gentoo.org
  Gentoo Documentation Project Lead  |  http://www.gentoo.org/proj/en/gdp
  Gentoo Council Member  

  The Gentoo Project   <<< http://www.gentoo.org >>>

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-server] Stable Portage tree

[Sven Vermeulen]

[-- Attachment #1: Type: text/plain, Size: 1694 bytes --]

On Thu, Sep 22, 2005 at 08:00:11PM +1000, Phillip Berry wrote:
> Just wondering if there has been any progress on the stable portage tree? 
> 
> Also, syncing the normal tree removes old versions of ebuilds, obviously this 
> is inappropriate for a production environment where for various reasons it is 
> sometimes neccessary to stay at an arbitrary version of an application.  The 
> loss of the ebuild specific to the legacy version of the application is a 
> pain, will the stable tree retain older versions of ebuilds instead of 
> removing them?

I think it isn't totally difficult to develop and maintain a stable tree for
an environment. You install Gentoo, let the install go through a few pillars
depending on your environment. Then, monitor Portage upgrades and backport
those to your stable environment.

I have had good luck with this approach for dedicated servers. After all,
when you know what software is available on the server (only a small portion
of all the software available through Portage) upgrades are a lot less
frequent.

Any upgrades that are pending (for instance JRE updates if you are running
J2EE servers) can easily be sorted out. It's still a lot of manual work
though, but I think it isn't easy to concentrate this on the distribution.

After all, one environment always differs from another. Where minor upgrades
are acceptable by a few, others might not like it. 

Wkr,
      Sven Vermeulen

-- 
  Gentoo Foundation Trustee          |  http://foundation.gentoo.org
  Gentoo Documentation Project Lead  |  http://www.gentoo.org/proj/en/gdp
  Gentoo Council Member  

  The Gentoo Project   <<< http://www.gentoo.org >>>

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]