[gentoo-dev] Maintaining production systems - and losing ebuilds

[gentoo-dev] Maintaining production systems - and losing ebuilds

[Ron OHara]

Hi,

I want to raise an issue resulting from my experience so far in using 
Gentoo as the basis of production systems. Some may ask why? - but 
basically 'portage' seems to offer the very best framework for ongoing 
maintenance/admin of systems, though it's not perfect in that role.

In essence, the continuous, easy upgrade capability of portage is great 
for a development system and should be an excellent mechanism for 
critical security (and other) upgrades in a production environment (and 
it is).
The problems arise because of the continuous easy upgrades!! - the main 
benefit is also the main problem.

I have just hit a real life hassle with a security upgrade. The history 
of it goes like this:

[background]
The example system in trouble is an old P233, and used to be on the end 
of a dialup link (it's now ADSL).
Gentoo has been installed for about 10 months and the last time it was 
brought completely up to date was about 6 months ago (emerge rsync && 
emerge -u world)
[/background]


[creating a problem]

As you have guessed, I've just had some system problems - partly of my 
own creation, but partly because of how Gentoo operates.

My real problem came from doing 'emerge rsync', and then just 
(selectively) doing 'emerge -u openssl'

This installed 'openssl-0.9.7' and removed 'openssl-0.9.6' - 
unfortunately lots of stuff on the system was compiled and linked 
against 'openssl-0.9.6' and they promptly broke. IE. Serious outage on a 
production system.

There is a script designed to fix this called 'revdep-rebuild' which 
scans all the installed binaries for broken dependencies and then 
recompiles them which should make them link against the nice new 
'openssl-0.9.7'

except!!! - revdep-rebuild carefully tries to recompile the exact 
versions of software you have installed (good idea) - but the Gentoo 
central repository has since deleted some of the build scripts for these 
older versions and when I did the 'emerge rsync', the scripts were also 
removed from my system. So I ended up where I am now - I have to go 
through and do 'emerge -u world' and then 'revdep-rebuild' to get it all 
working... not nice when there are nearly 200 packages to 
download/recompile on an old P233

[/creating a problem]




As you can see, I was intending to leave the installed set of packages 
(and versions) alone. For this machine (and any production system), I 
dont want to install each and every little patch as it comes along. The 
machine is 'stable' - so I only want to apply upgrades on a very 
selective, controlled, manual basis - but still use portage for the 
package management.
This is a very common tactic for 'production' machines, where you want 
the minimum number of changes to reduce your risks of outage.

The trap is that 'emerge rsync' removes old .ebuilds that your installed 
machine may need if revdep-rebuild is to be able to recovery things 
after a critical library is rebuilt.
In the way portage works, the only time it is safe for 'emerge rsync' to 
delete ebuilds, is immediately after successfully doing 'emerge -u world'.


Is there a way to suppress the 'delete' part of  rsync? Maybe a setting 
in /etc/make.conf ?

That way, even though Gentoo may have removed the relevant (old) ebuild 
I want, the target machine would have it's local portage version for 
future recompiles.... I can afford the disk space!!!




Regards
Ron OHara
PS: This is not a 'casual' problem for me - I've convinced a client to 
use Gentoo for the basis of their deployments and the plan is supposed 
to be for around 900 sites!! - catering for production software support 
for the next decade is very relevant to things in this scenario.








--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Lisa Seelye]

[-- Attachment #1: Type: text/plain, Size: 622 bytes --]

On Sun, 2003-11-09 at 22:00, Ron OHara wrote:
> Hi,
> 
> I want to raise an issue resulting from my experience so far in using 
> Gentoo as the basis of production systems. Some may ask why? - but 
> basically 'portage' seems to offer the very best framework for ongoing 
> maintenance/admin of systems, though it's not perfect in that role.

There are a couple things you may want to look into.

First, have you considered setting up your own rsync repository?
Second, how about using PORTAGE_OVERLAY to save ebuilds.


-- 
Regards,
-Lisa
<Vix ulla tam iniqua pax, quin bello vel aequissimo sit potior>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Luke-Jr]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 10 November 2003 03:00 am, Ron OHara wrote:
> except!!! - revdep-rebuild carefully tries to recompile the exact
> versions of software you have installed (good idea) - but the Gentoo
> central repository has since deleted some of the build scripts for these
> older versions and when I did the 'emerge rsync', the scripts were also
> removed from my system.
Sounds like a bug with revdep-rebuild. I'd think it should be modified to use 
the ebuild in /var/db/pkg/*/*/ so it gets the same one that was installed...
- -- 
Luke-Jr
Developer, Gentoo Linux
http://www.gentoo.org/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/rwZYZl/BHdU+lYMRAgsEAJ9IvfEM7HpgsBBE1zZeQjlYyAXeDQCfS7Aj
LFltACIflkYHPN2kXXieV5w=
=FzqD
-----END PGP SIGNATURE-----


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Jason Stubbs]

On Monday 10 November 2003 12:00, Ron OHara wrote:
> This installed 'openssl-0.9.7' and removed 'openssl-0.9.6' -
> unfortunately lots of stuff on the system was compiled and linked
> against 'openssl-0.9.6' and they promptly broke. IE. Serious outage on a
> production system.

There is work going on to fix this. Others will be able to answer this better 
than me.

> There is a script designed to fix this called 'revdep-rebuild' which
> scans all the installed binaries for broken dependencies and then
> recompiles them which should make them link against the nice new
> 'openssl-0.9.7'
>
> except!!! - revdep-rebuild carefully tries to recompile the exact
> versions of software you have installed (good idea) - but the Gentoo
> central repository has since deleted some of the build scripts for these
> older versions and when I did the 'emerge rsync', the scripts were also
> removed from my system. So I ended up where I am now - I have to go
> through and do 'emerge -u world' and then 'revdep-rebuild' to get it all
> working... not nice when there are nearly 200 packages to
> download/recompile on an old P233

I would call this a bug in revdep-rebuild. If it wants to use the package 
that's installed rather than the latest one when rebuilding it should use the 
ebuild that IS in /var/db/pkg/ rather than an ebuild that MAY be in /usr/
portage. Just my $0.02.

Regards,
Jason

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Ron OHara]

Lisa Seelye wrote:

>On Sun, 2003-11-09 at 22:00, Ron OHara wrote:
>  
>
>>Hi,
>>
>>I want to raise an issue resulting from my experience so far in using 
>>Gentoo as the basis of production systems. Some may ask why? - but 
>>basically 'portage' seems to offer the very best framework for ongoing 
>>maintenance/admin of systems, though it's not perfect in that role.
>>    
>>
>
>There are a couple things you may want to look into.
>
>First, have you considered setting up your own rsync repository?
>Second, how about using PORTAGE_OVERLAY to save ebuilds.
>
>
>  
>
An rsync repository is another part of the production deployment issues, 
(especially for bandwidth issues) but ideally the overall process should 
not force me to duplicate the managment effort that already goes into 
maintaining the Gentoo portage 'repository'.  That work is already being 
done so it seems silly to have to manually administer a downstream 
repository just to preserve 'old' ebuilds - and even then, the true 
repository of which ebuilds are needed for a specific system is held on 
that system .. not on another server.

To a degree, the same thing applies to the PORTAGE_OVERLAY setting - 
that tree may be a suitable place to preserve older ebuilds that are 
being removed from the central portage, but I dont want to maintain it 
manually on hundreds of systems.



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Matthew Kennedy]

Ron OHara <rono@sentuny.com.au> writes:

> Hi,
>
> I want to raise an issue resulting from my experience so far in using

[...]

arch or ~arch Gentoo system?

> This installed 'openssl-0.9.7' and removed 'openssl-0.9.6' - 
> unfortunately lots of stuff on the system was compiled and linked

[...]

My understanding was that a new 0.9.7 ebuild did not build the 0.9.6
library.  I've seen one other user run into exactly this problem.

Matt
-- 
Matthew Kennedy
Gentoo Linux Developer

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Antonio Dolcetta]

On Mon, 10 Nov 2003 14:00:36 +1100
Ron OHara <rono@sentuny.com.au> wrote:

> Is there a way to suppress the 'delete' part of  rsync? Maybe a setting 
> in /etc/make.conf ?
> 
> That way, even though Gentoo may have removed the relevant (old) ebuild 
> I want, the target machine would have it's local portage version for 
> future recompiles.... I can afford the disk space!!!

you could simply edit /usr/bin/emerge like this (lines will probably wrap)


--- /usr/bin/emerge     2003-10-24 17:33:53.000000000 +0200
+++ emerge      2003-11-10 10:12:25.806702152 +0100
@@ -1743,7 +1743,7 @@
                                mytimeout=int(portage.settings["RSYNC_TIMEOUT"])
                        except:
                                pass
-               mycommand="/usr/bin/rsync -rlptDvz --progress --stats --delete --delete-after --timeout="+str(mytimeout)+" --exclude='distfiles/*' --exclude='local/*' --exclude='packages/*' "
+               mycommand="/usr/bin/rsync -rlptDvz --progress --stats --timeout="+str(mytimeout)+" --exclude='distfiles/*' --exclude='local/*' --exclude='packages/*' "
                if portage.settings.has_key("RSYNC_EXCLUDEFROM"):
                        if os.path.exists(portage.settings["RSYNC_EXCLUDEFROM"]):
                                mycommand=mycommand+" --exclude-from "+portage.settings["RSYNC_EXCLUDEFROM"]

ciao

	Antonio

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Eldad Zack]

> This installed 'openssl-0.9.7' and removed 'openssl-0.9.6' - 
> unfortunately lots of stuff on the system was compiled and linked 
> against 'openssl-0.9.6' and they promptly broke. IE. Serious outage on a 
> production system.

Not just that. if wget depends on openssl-0.9.6 and now the library is 
gone, you can't even download wget sources to perform a recompile.

I'd suggest a special precaution with production enviornments - always 
build binary packages.
That way you can roll back very fast if things don't work out.
You can use quickpkg to make one from the existing filesystem if you're 
about to upgrade important packages if you didn't build a binary package 
in the first place.

Also, you might want to change /etc/make.conf AUTOCLEAN="yes" to no.



Eldad


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Jens Hoffrichter]

On Mon, Nov 10, 2003 at 11:36:02AM +0200, Eldad Zack wrote:

> > This installed 'openssl-0.9.7' and removed 'openssl-0.9.6' - 
> > unfortunately lots of stuff on the system was compiled and linked 
> > against 'openssl-0.9.6' and they promptly broke. IE. Serious outage on a 
> > production system.
> 
> Not just that. if wget depends on openssl-0.9.6 and now the library is 
> gone, you can't even download wget sources to perform a recompile.
> 
> I'd suggest a special precaution with production enviornments - always 
> build binary packages.
> That way you can roll back very fast if things don't work out.
> You can use quickpkg to make one from the existing filesystem if you're 
> about to upgrade important packages if you didn't build a binary package 
> in the first place.
Could you describe this a bit more detailed, please? How to make binary
packages? Or at least tell where to look for documentation? :)

Thanks in advance!

CU all,
Jens
-- 
GPG: 1024D/CF884D50  F2E8 F7FC F823 6464 4E9D  EFAB 6EE9 8B9C CF88 4D50
------------------------------------------------------------------------
 Jens Hoffrichter / joho@hausboot.org / Joho@IRC / Fon: +46-650-5376989
  Deck of Cards: $1.29. Card Table: $14.99. "101 Solitaire Variations"
  book: $6.59. Finding a cheaper replacement for the one thing Windows
                       is ideal for: priceless.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Luke-Jr]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 10 November 2003 11:26 am, Jens Hoffrichter wrote:
> Could you describe this a bit more detailed, please? How to make binary
> packages? Or at least tell where to look for documentation? :)
Might try 'emerge --help' or 'nano /etc/make.conf'...
I just stick buildpkg in my FEATURES var and use 'emerge -K' when remerging 
stuff...
- -- 
Luke-Jr
Developer, Gentoo Linux
http://www.gentoo.org/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/r3ndZl/BHdU+lYMRAj1ZAJ4uxeWgmaTV8PhfJ2/dd1aR9qWzTQCdFaLC
mWQ8aaMvGZqoi9/1FHC2Z44=
=8nfd
-----END PGP SIGNATURE-----


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1697 bytes --]

On Sun, 2003-11-09 at 22:46, Ron OHara wrote:
> Lisa Seelye wrote:
> 
> >On Sun, 2003-11-09 at 22:00, Ron OHara wrote:
> >  
> >
> >>Hi,
> >>
> >>I want to raise an issue resulting from my experience so far in using 
> >>Gentoo as the basis of production systems. Some may ask why? - but 
> >>basically 'portage' seems to offer the very best framework for ongoing 
> >>maintenance/admin of systems, though it's not perfect in that role.
> >>    
> >>
> >
> >There are a couple things you may want to look into.
> >
> >First, have you considered setting up your own rsync repository?
> >Second, how about using PORTAGE_OVERLAY to save ebuilds.
> >
> >
> >  
> >
> An rsync repository is another part of the production deployment issues, 
> (especially for bandwidth issues) but ideally the overall process should 
> not force me to duplicate the managment effort that already goes into 
> maintaining the Gentoo portage 'repository'.  That work is already being 
> done so it seems silly to have to manually administer a downstream 
> repository just to preserve 'old' ebuilds - and even then, the true 
> repository of which ebuilds are needed for a specific system is held on 
> that system .. not on another server.
> 
> To a degree, the same thing applies to the PORTAGE_OVERLAY setting - 
> that tree may be a suitable place to preserve older ebuilds that are 
> being removed from the central portage, but I dont want to maintain it 
> manually on hundreds of systems.

Two words...

NFS mounts

=]
> 
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Eldad Zack]

On Mon, 10 Nov 2003, Jens Hoffrichter wrote:

> On Mon, Nov 10, 2003 at 11:36:02AM +0200, Eldad Zack wrote:
> 
> > > This installed 'openssl-0.9.7' and removed 'openssl-0.9.6' - 
> > > unfortunately lots of stuff on the system was compiled and linked 
> > > against 'openssl-0.9.6' and they promptly broke. IE. Serious outage on a 
> > > production system.
> > 
> > Not just that. if wget depends on openssl-0.9.6 and now the library is 
> > gone, you can't even download wget sources to perform a recompile.
> > 
> > I'd suggest a special precaution with production enviornments - always 
> > build binary packages.
> > That way you can roll back very fast if things don't work out.
> > You can use quickpkg to make one from the existing filesystem if you're 
> > about to upgrade important packages if you didn't build a binary package 
> > in the first place.
> Could you describe this a bit more detailed, please? How to make binary
> packages? Or at least tell where to look for documentation? :)

look at the emerge manpage. when you merge new builds you can emerge -b - 
that will make portage build a binary (tbz2) as well as emerge into the 
system.
If you want to make a binary out of the filesystem you've got now - It's 
as simple as "quickpkg package-name" and there's a manpage as well 
for quickpkg (it's part of portage).
Just keep in mind one thing about quickpkg - (from the manpage): "The 
downside is that the package will contain the files that exist on your 
filesystem even if they have modified since they were first installed."



Eldad



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Ron OHara]

It's an ~arch system ..  and that currently means openssh-0.9.7 --- 
which removed openssh-0.9.6 and lead to the other complications.

Ron

Matthew Kennedy wrote:

>Ron OHara <rono@sentuny.com.au> writes:
>
>  
>
>>Hi,
>>
>>I want to raise an issue resulting from my experience so far in using
>>    
>>
>
>[...]
>
>arch or ~arch Gentoo system?
>
>  
>
>>This installed 'openssl-0.9.7' and removed 'openssl-0.9.6' - 
>>unfortunately lots of stuff on the system was compiled and linked
>>    
>>
>
>[...]
>
>My understanding was that a new 0.9.7 ebuild did not build the 0.9.6
>library.  I've seen one other user run into exactly this problem.
>
>Matt
>  
>



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Benjamin Coles]

[-- Attachment #1: Type: text/plain, Size: 1764 bytes --]

	Lately I've been seeing more and more problems revolving around these
issues, being a gentooist makes you a better than average Linux user and
should be able to find out the answer to your problems on forums or
bugzilla. Having said this does not mean I don't believe we shouldn't
fix it.
	Currently I'm trying to revive Gentoo-Corporate and Gentoo Stable which
I consider the same thing in the end. One stable release and then just
release security updates from then on until the next major release. This
will solve a lot of problems and expand gentoo into the corporate world
and with the upcoming red hat discontinuing it's efforts in the open
source community, this would be a perfect time for gentoo to push itself
out. There are things holding back this project but I am also determined
to get it rolling again.

-------------
Benjamin Coles
Gentoo Infrastructure


On Mon, 2003-11-10 at 20:31, Ron OHara wrote:
> It's an ~arch system ..  and that currently means openssh-0.9.7 --- 
> which removed openssh-0.9.6 and lead to the other complications.
> 
> Ron
> 
> Matthew Kennedy wrote:
> 
> >Ron OHara <rono@sentuny.com.au> writes:
> >
> >  
> >
> >>Hi,
> >>
> >>I want to raise an issue resulting from my experience so far in using
> >>    
> >>
> >
> >[...]
> >
> >arch or ~arch Gentoo system?
> >
> >  
> >
> >>This installed 'openssl-0.9.7' and removed 'openssl-0.9.6' - 
> >>unfortunately lots of stuff on the system was compiled and linked
> >>    
> >>
> >
> >[...]
> >
> >My understanding was that a new 0.9.7 ebuild did not build the 0.9.6
> >library.  I've seen one other user run into exactly this problem.
> >
> >Matt
> >  
> >
> 
> 
> 
> --
> gentoo-dev@gentoo.org mailing list
> 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Benjamin Coles]

It's an effort to get a stable tree out there that doesn't change much. We
had a few people from IBM that said they would love to change to gentoo
but the portage changes too quickly and they can't risk emerging something
and having it break afterwards. There are many more factors in
implementating this but it's still in the planning stages.

-Benjamin

> On Mon, Nov 10, 2003 at 09:04:43PM -0800, Benjamin Coles wrote:
>
>> 	Currently I'm trying to revive Gentoo-Corporate and Gentoo Stable
>> which
>
>
> what is this gentoo corporate you speak of ?




--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Lisa Seelye]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

On Tue, 2003-11-11 at 12:36, Benjamin Coles wrote:
> It's an effort to get a stable tree out there that doesn't change much. We
> had a few people from IBM that said they would love to change to gentoo
> but the portage changes too quickly and they can't risk emerging something
> and having it break afterwards. There are many more factors in
> implementating this but it's still in the planning stages.


Sounds like just a slower changing rsync tree, based on the stable.g.o
motif.


-- 
Regards,
-Lisa
<Vix ulla tam iniqua pax, quin bello vel aequissimo sit potior>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 856 bytes --]

On Monday 10 November 2003 18:48, Eldad Zack wrote:
> look at the emerge manpage. when you merge new builds you can emerge -b -
> that will make portage build a binary (tbz2) as well as emerge into the
> system.
> If you want to make a binary out of the filesystem you've got now - It's
> as simple as "quickpkg package-name" and there's a manpage as well
> for quickpkg (it's part of portage).
> Just keep in mind one thing about quickpkg - (from the manpage): "The
> downside is that the package will contain the files that exist on your
> filesystem even if they have modified since they were first installed."

Be aware though that sometimes quickpkg does not work as well as merge-time 
packages. The latter are to be preferred.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Spider]

[-- Attachment #1: Type: text/plain, Size: 1694 bytes --]

begin  quote
On Tue, 11 Nov 2003 13:25:18 -0500
Lisa Seelye <lisa@gentoo.org> wrote:

> On Tue, 2003-11-11 at 12:36, Benjamin Coles wrote:
> > It's an effort to get a stable tree out there that doesn't change
> > much. We
> > had a few people from IBM that said they would love to change to
> > gentoo
> > but the portage changes too quickly and they can't risk emerging
> > something
> > and having it break afterwards. There are many more factors in
> > implementating this but it's still in the planning stages.
> 
> 
> Sounds like just a slower changing rsync tree, based on the stable.g.o
> motif.
> 


No, in fact not. 
For Corporate use in this case, we should release a snapshot, reduced
tree, stable tree, or something like that.
Then -NEVER- -EVER- Change that.

All updates to that tree should be separated into a second release tree
which contains errata, and be avaiable for continuous update for about a
year or more.


Yes, this is in fact the thing that made Debian completely unusable at
work (they apply fixes to their main tree)  , and also rules out Gentoo.

For corporate use, we know we can fork it. Thats not  the issue. the
issue is that we -will- fork things, and we need the baseline to -never
ever- change. Because errata can then be merged into mainline at the
corporates own time and developers will.

This is actually a big hamstring with our current distribution model,
that works great for homeusers and smaller servers, but badly prevents
anyone from fully taking advantage of the flexibility of Gentoo.

//Spider

-- 
begin  .signature
This is a .signature virus! Please copy me into your .signature!
See Microsoft KB Article Q265230 for more information.
end

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Don Seiler]

[-- Attachment #1: Type: text/plain, Size: 1651 bytes --]

On Tue, Nov 11, 2003 at 09:53:27PM +0100, Spider wrote:
> begin  quote
> Yes, this is in fact the thing that made Debian completely unusable at
> work (they apply fixes to their main tree)  , and also rules out Gentoo.

Ditto.  Devs can have gentoo on their workstations (and we do), but for
servers we have to use RedHat AS.  Although this mainly too is because
we are an Oracle shop and Oracle only supports RHAS and Suse.  But
ebrostig pointed out that Oracle will "never" support Gentoo because it
changes too much:

http://forums.gentoo.org/viewtopic.php?t=25080

"Oracle has never and will never certify the databases with Gentoo. The
only 2 distributiuons currently certified with Oracle 9i is RedHat
Advanced Server 2.1 and SuSE SLES 7.1

...

The reason is very simple.

In order to ensure that you are running Oracle on a stable platform,
which all of our customers are interested in, we have decided to certify
it against distributions that will stay stable for a certain amount of
time. Gento is a very dynamic distributions, people can install both
stable and unstable packages which will, sooner or later, break Oracle.

Besides this, the cost of certifying a distribution is failry high and
there is a lot of work involved on our and on the distributions part.

It is basically a business justification and we don't want to maintain
staff that knows the intricate details of too many distributions.
Currently we also do OS support on redHat AS 2.1 in addition to normal
Oracle support."

I realize Gentoo's management may not have the delusions of grandeur to
become an Oracle-certified distro but I'm just laying down the facts.

Don.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Ron OHara]

In this case I dont want to make revdep-renuild ignore the version - I 
just need it to recompile the version of everything that has been 
installed.   In a production environment, I cant just 'upgrade' many 
versions of  software without regression testing of the applications in 
that environment.
I need to be able to apply security fixes (like the openssl library) and 
still have the installed set of software work.  One change at a time is 
crucial for production systems stability.

Ron

Sami K M Naatanen wrote:

>On Monday 10 November 2003 05:00, Ron OHara wrote:
>
>revdep-rebuild --help and you will see that you can set the revdep-rebuild to 
>ignore the version. You might have problems getting the files, because wget 
>depends from openssl (if you have ssl in your USE). But do emerge -fp wget
>and you get a bunch of URI's download every file and dump them in the
>/PORTDIR/distfiles/ emerge wget.
>
>revdep-rebuild with the version ignoring option. or you can
>first add -- -p to see what emerge would rebuild.
>
>If your portage version is too old (ie your version of revdep-rebuild doesn't 
>support options) then do emerge -fp portage and download all of those files.
>
>You also could try to add link 0.9.6 to the 0.9.7 version of the ssl lib. This 
>should work. Based on the fact, that it seems that the new openssl ebuild 
>misses the link creation according to some people.
>
>  
>
>>Hi,
>>
>>I want to raise an issue resulting from my experience so far in using
>>Gentoo as the basis of production systems. Some may ask why? - but
>>basically 'portage' seems to offer the very best framework for ongoing
>>maintenance/admin of systems, though it's not perfect in that role.
>>
>>In essence, the continuous, easy upgrade capability of portage is great
>>for a development system and should be an excellent mechanism for
>>critical security (and other) upgrades in a production environment (and
>>it is).
>>The problems arise because of the continuous easy upgrades!! - the main
>>benefit is also the main problem.
>>
>>I have just hit a real life hassle with a security upgrade. The history
>>of it goes like this:
>>
>>[background]
>>The example system in trouble is an old P233, and used to be on the end
>>of a dialup link (it's now ADSL).
>>Gentoo has been installed for about 10 months and the last time it was
>>brought completely up to date was about 6 months ago (emerge rsync &&
>>emerge -u world)
>>[/background]
>>
>>
>>[creating a problem]
>>
>>As you have guessed, I've just had some system problems - partly of my
>>own creation, but partly because of how Gentoo operates.
>>
>>My real problem came from doing 'emerge rsync', and then just
>>(selectively) doing 'emerge -u openssl'
>>
>>This installed 'openssl-0.9.7' and removed 'openssl-0.9.6' -
>>unfortunately lots of stuff on the system was compiled and linked
>>against 'openssl-0.9.6' and they promptly broke. IE. Serious outage on a
>>production system.
>>
>>There is a script designed to fix this called 'revdep-rebuild' which
>>scans all the installed binaries for broken dependencies and then
>>recompiles them which should make them link against the nice new
>>'openssl-0.9.7'
>>
>>except!!! - revdep-rebuild carefully tries to recompile the exact
>>versions of software you have installed (good idea) - but the Gentoo
>>central repository has since deleted some of the build scripts for these
>>older versions and when I did the 'emerge rsync', the scripts were also
>>removed from my system. So I ended up where I am now - I have to go
>>through and do 'emerge -u world' and then 'revdep-rebuild' to get it all
>>working... not nice when there are nearly 200 packages to
>>download/recompile on an old P233
>>
>>[/creating a problem]
>>
>>
>>
>>
>>As you can see, I was intending to leave the installed set of packages
>>(and versions) alone. For this machine (and any production system), I
>>dont want to install each and every little patch as it comes along. The
>>machine is 'stable' - so I only want to apply upgrades on a very
>>selective, controlled, manual basis - but still use portage for the
>>package management.
>>This is a very common tactic for 'production' machines, where you want
>>the minimum number of changes to reduce your risks of outage.
>>
>>The trap is that 'emerge rsync' removes old .ebuilds that your installed
>>machine may need if revdep-rebuild is to be able to recovery things
>>after a critical library is rebuilt.
>>In the way portage works, the only time it is safe for 'emerge rsync' to
>>delete ebuilds, is immediately after successfully doing 'emerge -u world'.
>>
>>
>>Is there a way to suppress the 'delete' part of  rsync? Maybe a setting
>>in /etc/make.conf ?
>>
>>That way, even though Gentoo may have removed the relevant (old) ebuild
>>I want, the target machine would have it's local portage version for
>>future recompiles.... I can afford the disk space!!!
>>
>>
>>
>>
>>Regards
>>Ron OHara
>>PS: This is not a 'casual' problem for me - I've convinced a client to
>>use Gentoo for the basis of their deployments and the plan is supposed
>>to be for around 900 sites!! - catering for production software support
>>for the next decade is very relevant to things in this scenario.
>>    
>>
>
>  
>



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 1049 bytes --]

On Tuesday 11 November 2003 21:53, Spider wrote:
>
> For corporate use, we know we can fork it. Thats not  the issue. the
> issue is that we -will- fork things, and we need the baseline to -never
> ever- change. Because errata can then be merged into mainline at the
> corporates own time and developers will.
>
> This is actually a big hamstring with our current distribution model,
> that works great for homeusers and smaller servers, but badly prevents
> anyone from fully taking advantage of the flexibility of Gentoo.

There are certainly thought about this. Currently the main blocking factor of 
this is developer time. At the moment the point that still has more priority 
is general QA before we think we are even able to maintain a fixed tree. Of 
course at the moment that we start offering the first fixed tree it is still 
not advised for production use, but we hope to help gentoo reach the 
enterprise too.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 694 bytes --]

On Tuesday 11 November 2003 22:06, Don Seiler wrote:
> I realize Gentoo's management may not have the delusions of grandeur to
> become an Oracle-certified distro but I'm just laying down the facts.

I personally would not mind it. It sound really cool ;-), however priorities 
are currently not really at oracle certification. If they would ask us, I 
think we would at least consider though. It also depends on the amount of 
work it would cost for us to do what oracle wants. And on what they want. We 
are here first to scratch our own itches, not to do oracles biddings.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Matt Wilson]

On Tue, 2003-11-11 at 21:38, Spider wrote:
> begin  quote
> On Tue, 11 Nov 2003 21:00:57 +0000
> Matt Wilson <matt@mattsscripts.co.uk> wrote:
> 
> btw, didn't you mean to post this to the list?

Apologies, I did, yes (first post woes!)

> > > No, in fact not. 
> > > For Corporate use in this case, we should release a snapshot,
> > > reduced
> > > tree, stable tree, or something like that.
> > > Then -NEVER- -EVER- Change that.
> > 
> > There was the mention of security updates being added, would surely
> > would be beneficial, even essential, to the tree?
> > 
> 
> 
> No, see the post for rationale.
> The -RELEASE- tree should -NEVER EVER- Change. Errata (that is security
> updates and critical bugfixes)  should be released as a -SEPARATE- tree.
> 
> Adding extra emphasis here. 

Fair enough, that makes sense, though I suspect that most
companies/organisations that would use *any* tree would want to keep up
with security releases - making the "release" tree unused - unless the
proposal was that anything that may need essential (e.g. security)
patches went in a separate tree (sorry if this is the case, I missed the
start of this discussion).

--
http://www.mattsscripts.co.uk/
  - A great source for free CGI and stuff

I AM DEATH, NOT TAXES. I TURN UP ONLY ONCE.  (Feet of Clay)


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Spider]

[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]

begin  quote
On Tue, 11 Nov 2003 22:24:14 +0100
Paul de Vrieze <pauldv@gentoo.org> wrote:

> There are certainly thought about this. Currently the main blocking
> factor of this is developer time. At the moment the point that still
> has more priority is general QA before we think we are even able to
> maintain a fixed tree. Of course at the moment that we start offering
> the first fixed tree it is still not advised for production use, but
> we hope to help gentoo reach the enterprise too.


Yes, I understand and support this reasoning, but I also had to state
the issues here for others to know about, people who might not have had
to sit through two months of bickering to get to the point of drafting a
policydocument about what they feel about "open source" before even
considering to -try- a demo setup.


A bit of frustration for me, but okay. I know where that "it strategist"
get their free lunch from.

//Spider


-- 
begin  .signature
This is a .signature virus! Please copy me into your .signature!
See Microsoft KB Article Q265230 for more information.
end

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Spider]

[-- Attachment #1: Type: text/plain, Size: 1079 bytes --]

begin  quote
On Tue, 11 Nov 2003 21:48:26 +0000
Matt Wilson <matt@mattsscripts.co.uk> wrote:



> Fair enough, that makes sense, though I suspect that most
> companies/organisations that would use *any* tree would want to keep
> up with security releases - making the "release" tree unused - unless
> the proposal was that anything that may need essential (e.g. security)
> patches went in a separate tree (sorry if this is the case, I missed
> the start of this discussion).
>

Actually, thats not the case. In the case of a company in this scale, or
doing this sort of work, they will simply -not consider- Gentoo as it
doesn't meet the base prerequesit. That goes for Debian as well, which
also have the "move updates into main tree" mentality that makes
maintainance a hell.

Do take the time to read the other posts on this issue though, if you
don't have them locally, google for "gentoo-dev archives" should help.

//Spider


-- 
begin  .signature
This is a .signature virus! Please copy me into your .signature!
See Microsoft KB Article Q265230 for more information.
end

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 1186 bytes --]

On Tuesday 11 November 2003 22:52, Spider wrote:
> begin  quote
> On Tue, 11 Nov 2003 22:24:14 +0100
>
> Yes, I understand and support this reasoning, but I also had to state
> the issues here for others to know about, people who might not have had
> to sit through two months of bickering to get to the point of drafting a
> policydocument about what they feel about "open source" before even
> considering to -try- a demo setup.
>
>
> A bit of frustration for me, but okay. I know where that "it strategist"
> get their free lunch from.
>

I actually agree with you. Unfortunately I have to tell you that it is very 
likely to happen soon. At the same time I wanted to say that there are people 
working on it and thinking about it. Basically the point is that gentoo gets 
more and more the characteristics of a bulk carrier, and less those of a 
sleek sailing yacht.

Paul

ps. I also feel very sorry for those needing to work with "corporate pollicy" 
although I actualy probably would do the same if I were in the position to 
establish such a pollicy.

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Benjamin Coles]

[-- Attachment #1: Type: text/plain, Size: 3075 bytes --]

I wanted to note a few things about the following thread so far:

1) Oracle Support - I do have to agree with Oracle about not supporting
a dynamic distro such as gentoo, it's a lot of time and money wasted on
silly ventures trying to figure out what latest update broke it. If it
were just updates and very simple to spot a problem then why not get
with the race and support it or simply be left behind.

2) Stable Tree - This is a snapshot, a once a year thing where we will
take our current tree and deem it -stable. It will then go over testing
for the course of 3 months to try and break crack and punch every
possible hole we could think of with that tree. When it's all done we'll
release it as a corporate version. Might as well call it
gentoo-production or gentoo-stable, as you might see that advertising is
the game in the corporate world and a fancy title like gentoo-corporate
will grab some eyes.

3) Our Path to Gentoo-Stable - Gentoo currently has some milestones
setup to where we want to go and what we're doing to get there,
unfortunately -stable isn't on the list until after we can improve a
bunch of smaller areas that need to be attended to and if you haven't
quite figured it out, the rate of gentoo popularity and keeping the
portage tree up to date is no easy task. Personally I'm going to write
up a path so we can get the milestone for -stable going and hopefully we
can give a better date on when it'll be released.

4) Organization - I saw a lot of people saying how easy this way to
split a tree and just call it stable. Ask any company the process they
take just to put out one product, they have an entire division devoted
to that product and many months of labor included. The first thing we
have to do is choose a plan of execution to get this project done, and
then we'll move on until it's finished.

-Benjamin


On Tue, 2003-11-11 at 14:28, Paul de Vrieze wrote:
> On Tuesday 11 November 2003 22:52, Spider wrote:
> > begin  quote
> > On Tue, 11 Nov 2003 22:24:14 +0100
> >
> > Yes, I understand and support this reasoning, but I also had to state
> > the issues here for others to know about, people who might not have had
> > to sit through two months of bickering to get to the point of drafting a
> > policydocument about what they feel about "open source" before even
> > considering to -try- a demo setup.
> >
> >
> > A bit of frustration for me, but okay. I know where that "it strategist"
> > get their free lunch from.
> >
> 
> I actually agree with you. Unfortunately I have to tell you that it is very 
> likely to happen soon. At the same time I wanted to say that there are people 
> working on it and thinking about it. Basically the point is that gentoo gets 
> more and more the characteristics of a bulk carrier, and less those of a 
> sleek sailing yacht.
> 
> Paul
> 
> ps. I also feel very sorry for those needing to work with "corporate pollicy" 
> although I actualy probably would do the same if I were in the position to 
> establish such a pollicy.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Owen Ford]

[-- Attachment #1: Type: text/plain, Size: 642 bytes --]

Several businesses that work for/with love the ideas behind Gentoo but
don't feel that they can rely on such a moving target.  This is the #1
deciding factor in not using gentoo.  #2 happens to be that the install
procedure is not easily scalable (until glis gets stable).

I think that Gentoo-corporate/production/stable is a great idea.  I
would like to get involved.  I have free time and might be able to scare
up a server for testing.

So who is (or going to be) in charge?

-- 
Owen Ford <oford@arghblech.com>

()  ascii ribbon campaign - against html e-mail 
/\                        - against proprietary attachments

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Maintaining production systems - and losing ebuilds

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 4332 bytes --]

On Tue, 2003-11-11 at 22:45, Ron OHara wrote:
> Chris Gianelloni wrote:
> 
> >On Mon, 2003-11-10 at 23:39, Ron OHara wrote:
> >  
> >
> >>Chris Gianelloni wrote:
> >>
> >>    
> >>
> >>>On Sun, 2003-11-09 at 22:46, Ron OHara wrote:
> >>> 
> >>>
> >>>      
> >>>
> >>>>Lisa Seelye wrote:
> >>>>
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>>>On Sun, 2003-11-09 at 22:00, Ron OHara wrote:
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>>>Hi,
> >>>>>>
> >>>>>>I want to raise an issue resulting from my experience so far in using 
> >>>>>>Gentoo as the basis of production systems. Some may ask why? - but 
> >>>>>>basically 'portage' seems to offer the very best framework for ongoing 
> >>>>>>maintenance/admin of systems, though it's not perfect in that role.
> >>>>>>  
> >>>>>>
> >>>>>>       
> >>>>>>
> >>>>>>            
> >>>>>>
> >>>>>There are a couple things you may want to look into.
> >>>>>
> >>>>>First, have you considered setting up your own rsync repository?
> >>>>>Second, how about using PORTAGE_OVERLAY to save ebuilds.
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>     
> >>>>>
> >>>>>          
> >>>>>
> >>>>An rsync repository is another part of the production deployment issues, 
> >>>>(especially for bandwidth issues) but ideally the overall process should 
> >>>>not force me to duplicate the managment effort that already goes into 
> >>>>maintaining the Gentoo portage 'repository'.  That work is already being 
> >>>>done so it seems silly to have to manually administer a downstream 
> >>>>repository just to preserve 'old' ebuilds - and even then, the true 
> >>>>repository of which ebuilds are needed for a specific system is held on 
> >>>>that system .. not on another server.
> >>>>
> >>>>To a degree, the same thing applies to the PORTAGE_OVERLAY setting - 
> >>>>that tree may be a suitable place to preserve older ebuilds that are 
> >>>>being removed from the central portage, but I dont want to maintain it 
> >>>>manually on hundreds of systems.
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>>Two words...
> >>>
> >>>NFS mounts
> >>>
> >>>=]
> >>> 
> >>>
> >>>      
> >>>
> >>>>--
> >>>>gentoo-dev@gentoo.org mailing list
> >>>>   
> >>>>
> >>>>        
> >>>>
> >>Hmmm ... NFS works when the systems are all nicely connected with 
> >>bandwidth, but most of these will be unique nodes on a private microwave 
> >>network (to avoid the local Telco charges .a.k.a 'gouging' - at $0.19 
> >>per megabyte of traffic)
> >>    
> >>
> >
> >That makes a big difference.  In that case, I would run your own rsync
> >mirror for portage and have all the machines sync to your mirror.  I
> >would also make a packages mirror, where you store your packages, and
> >have all the machines pull their packages from that site.  That way
> >nothing gets added or removed from portage, except what you specifiy,
> >and you also get the added upgrade speed and ease of binary packages.
> >
> >  
> >
> I am assuming the deployment of a private rsync mirror (and packages 
> mirror) - BUT - the current setup removes .ebuilds unless I manually 
> intervene. Something I want to avoid. I just want to accumulate ebuilds 
> by default to match the exact version of compiled code on each box. Then 
> any given machine can be recompiled if required, without regard to 
> accessing an external repository.

Well, what I would do is have the rsync mirror *not* mirror the
"official" Gentoo tree into its rsync, but rather into the normal
/usr/portage.  I would then manually move ebuilds for security fixes and
any upgrades I wanted into the rsync tree.  The same would be true of
the packages.  That way you only have *exactly* what you want and have
approved available to the client machines.

e.g. /usr/portage is normal portage tree, /usr/portage/packages is
normal package tree, /var/rsync/portage is YOUR portage tree.  I would
publish the packages via a web server.  The nice thing about this is
that you can standardize on quite a bit of packages.  If you wanted
everyone using KDE, for example, you simply don't copy any of Gnome's
ebuilds to your /var/rsync/portage tree.

-- 
Chris Gianelloni
Developer, Gentoo Linux
Games Team

Is your power animal a pengiun?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]