[gentoo-server] Managing multiple servers.

[gentoo-server] Managing multiple servers.

[la Bigmac]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]




Hello list, 

 

Seems to be a few people recently
wanting to discuss Gentoo as a server :-) so thought I would pose a question that
has been bugging me. 

 

What would you guys recommend to
manage multiple servers and the package versions? 

 

While I have a central emerge
server (rsync) and sync all of my servers to it I still manually update
the packages.

 

Example, openssh how should I be
updating openssh on all of my servers other than logging onto each one in turn
and running emerge openssh.

 

Should I cron schedule an emerge
--update world  and control the repository of packages or is there
a more elegant solution?

 

Regards,

 

Mat. 

 		 	   		  

[-- Attachment #2: Type: text/html, Size: 41514 bytes --]

Re: [gentoo-server] Managing multiple servers.

[Mark Shields]

[-- Attachment #1: Type: text/plain, Size: 1180 bytes --]

On Mon, May 23, 2011 at 5:12 PM, la Bigmac <la_bigmac@hotmail.com> wrote:

>  Hello list,
>
>
>
> Seems to be a few people recently wanting to discuss Gentoo as a server :-)
> so thought I would pose a question that has been bugging me.
>
>
>
> What would you guys recommend to manage multiple servers and the package
> versions?
>
>
>
> While I have a central emerge server (*rsync)* and sync all of my servers
> to it I still manually update the packages.
>
>
>
> Example, openssh how should I be updating openssh on all of my servers
> other than logging onto each one in turn and running emerge openssh.
>
>
>
> Should I cron schedule an *emerge* --*update world  *and control the
> repository of packages or is there a more elegant solution?
>
>
>
> Regards,
>
>
>
> Mat.
>


You can always use clusterssh to send the same command to all of them.

When I was managing multiple servers, I would have a cron job set to
eix-sync daily (the maximum frequency Gentoo recommends syncing), then once
a week I would have emerge -DNpvu world   e-mailed from cron.  I would then
review the e-mail and see if I wanted to upgrade.  If I didn't, I'd mask the
package version and move on.

[-- Attachment #2: Type: text/html, Size: 3338 bytes --]

Re: [gentoo-server] Managing multiple servers.

[Stefan Behte]

Hi,

>> What would you guys recommend to manage multiple servers and the package
>> versions?
Some sort of central deployment (puppet/cfengine/chef).

>> While I have a central emerge server (*rsync)* and sync all of my servers
>> to it I still manually update the packages.

Having a sync mirror is nice, a portage mirror helps too. Don't get
banned on the gentoo rsync mirrors, please don't use them directly. You
might want to keep older, "stable" (by your definition) versions of
rsync and portage, for installing additional servers.

> When I was managing multiple servers, I would have a cron job set to
> eix-sync daily (the maximum frequency Gentoo recommends syncing), then once
> a week I would have emerge -DNpvu world   e-mailed from cron.  I would then
> review the e-mail and see if I wanted to upgrade.  If I didn't, I'd mask the
> package version and move on.
> 
That sounds doable for a few hosts only to me.

Greets,

Craig

Re: [gentoo-server] Managing multiple servers.

[Nils]

[-- Attachment #1: Type: text/plain, Size: 417 bytes --]

>
> When I was managing multiple servers, I would have a cron job set to
> eix-sync daily (the maximum frequency Gentoo recommends syncing), then once
> a week I would have emerge -DNpvu world   e-mailed from cron.  I would then
> review the e-mail and see if I wanted to upgrade.  If I didn't, I'd mask the
> package version and move on.
>

May I ask why would would sync once a day but only run emerge once a week?

[-- Attachment #2: Type: text/html, Size: 591 bytes --]

Re: [gentoo-server] Managing multiple servers.

[Kristjan Kalder]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On 25.05.2011 21:43, Nils wrote:
>
>     When I was managing multiple servers, I would have a cron job set
>     to eix-sync daily (the maximum frequency Gentoo recommends
>     syncing), then once a week I would have emerge -DNpvu world  
>     e-mailed from cron.  I would then review the e-mail and see if I
>     wanted to upgrade.  If I didn't, I'd mask the package version and
>     move on.
>
>
> May I ask why would would sync once a day but only run emerge once a week?
To reduce network traffic for one single rsync session?

-- 
Lugupidamisega,
Kristjan Kalder
+372 56491771


[-- Attachment #2: Type: text/html, Size: 1340 bytes --]

Re: [gentoo-server] Managing multiple servers.

[mRyOuNg]

[-- Attachment #1: Type: text/plain, Size: 732 bytes --]

Or maybe for a glsa-Check -m affected. ?

:: Baptiste Boilet
. (mobile)

On 25 mai 2011, at 21:09, Kristjan Kalder <jolinar1@gmail.com> wrote:

> On 25.05.2011 21:43, Nils wrote:
>> 
>> When I was managing multiple servers, I would have a cron job set to eix-sync daily (the maximum frequency Gentoo recommends syncing), then once a week I would have emerge -DNpvu world   e-mailed from cron.  I would then review the e-mail and see if I wanted to upgrade.  If I didn't, I'd mask the package version and move on.
>> 
>> May I ask why would would sync once a day but only run emerge once a week?
> To reduce network traffic for one single rsync session?
> 
> -- 
> Lugupidamisega,
> Kristjan Kalder
> +372 56491771

[-- Attachment #2: Type: text/html, Size: 1420 bytes --]

Re: [gentoo-server] Managing multiple servers.

[Kristjan Kalder]

[-- Attachment #1: Type: text/plain, Size: 972 bytes --]

Is the glsa-check database (or from where it gets its information) 
updated on a regular basis or how does it work?
If I run the 'glsa-check -t affected' it newer shows me anything.

On 25.05.2011 22:27, mRyOuNg wrote:
> Or maybe for a glsa-Check -m affected. ?
>
> :: Baptiste Boilet
> . (mobile)
>
> On 25 mai 2011, at 21:09, Kristjan Kalder <jolinar1@gmail.com 
> <mailto:jolinar1@gmail.com>> wrote:
>
>> On 25.05.2011 21:43, Nils wrote:
>>>
>>>     When I was managing multiple servers, I would have a cron job
>>>     set to eix-sync daily (the maximum frequency Gentoo recommends
>>>     syncing), then once a week I would have emerge -DNpvu world  
>>>     e-mailed from cron.  I would then review the e-mail and see if I
>>>     wanted to upgrade.  If I didn't, I'd mask the package version
>>>     and move on.
>>>
>>>
>>> May I ask why would would sync once a day but only run emerge once a 
>>> week?
>> To reduce network traffic for one single rsync session?


[-- Attachment #2: Type: text/html, Size: 2189 bytes --]

Re: [gentoo-server] Managing multiple servers.

[mRyOuNg]

[-- Attachment #1: Type: text/plain, Size: 1135 bytes --]

On 05/25/2011 10:12 PM, Kristjan Kalder wrote:
> Is the glsa-check database (or from where it gets its information)
> updated on a regular basis or how does it work?
> If I run the 'glsa-check -t affected' it newer shows me anything.
>

Il I remember well, glsa infos are stored in /usr/portage (metadata or
something)...
glsa-check -t affected should list all unsecure packages...
glsa are updated directly into portage repository, by GLSA team ...

So If GLSA team have infos about an unsecure package, you should be
informed with glsa-check... and a regular emerge --sync...

However, GLSA team seems really busy as there's no new GLSA since
sometimes now ...

To me, I know I love the way Gentoo is working, and the portage
customization (use flags and such) ... but I don't upgrade my packages
so oftenly ... mainly because of time issue.
Something like, every month for my desktops, and every 3/6 months for my
servers ... and due to this, I'm a bit worried about the fact that GLSA
team doesn't give recent updates about security issues...

Anyway, I still sync everyday, and run glsa-check -m affected just to be
sure :)

Cya


[-- Attachment #2: Type: text/html, Size: 1709 bytes --]

Re: [gentoo-server] Managing multiple servers.

[Christian Kauhaus]

Am 25.05.2011 22:12, schrieb Kristjan Kalder:
> Is the glsa-check database (or from where it gets its information) updated
> on a regular basis or how does it work?
> If I run the 'glsa-check -t affected' it newer shows me anything.

Same here. It looks that new GLSAs are released very infrequently. The
other problem ist that they are tied to the portage tree. When I use a
frozen portage tree to get repeatable builds, I'll never get new GLSA.

I would favor check-glsa to use another means of communication (e.g., a RSS
feed).

Regards

Christian

-- 
Dipl.-Inf. Christian Kauhaus <>< · kc@gocept.com · systems administration
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 11 · fax +49 345 1229889 1
Zope and Plone consulting and development

Re: [gentoo-server] Managing multiple servers.

[Stefan Behte]

Hi,

> Same here. It looks that new GLSAs are released very infrequently. The
> other problem ist that they are tied to the portage tree. When I use a
> frozen portage tree to get repeatable builds, I'll never get new GLSA.
> 
> I would favor check-glsa to use another means of communication (e.g., a RSS
> feed).

Yes, I must confess that the security team is a bit shorthanded.
As a solution for check-glsa you could just sync the appropriate folder.

Craig

Re: [gentoo-server] Managing multiple servers.

[Stéphane Guedon]

[-- Attachment #1: Type: Text/Plain, Size: 985 bytes --]

On Wednesday 25 May 2011 21:09:02 Kristjan Kalder wrote:
> On 25.05.2011 21:43, Nils wrote:
> >     When I was managing multiple servers, I would have a cron job set
> >     to eix-sync daily (the maximum frequency Gentoo recommends
> >     syncing), then once a week I would have emerge -DNpvu world
> >     e-mailed from cron.  I would then review the e-mail and see if I
> >     wanted to upgrade.  If I didn't, I'd mask the package version and
> >     move on.
> > 
> > May I ask why would would sync once a day but only run emerge once a
> > week?
> 
> To reduce network traffic for one single rsync session?

if you emerge --sync only the time you plan emerge -uDN world, it's less heavy 
for everyone rather than everyday sync and weekly emerge -u...
-- 
Stéphane Guedon
page web : http://www.22decembre.eu/
carte de visite : http://www.22decembre.eu/downloads/Stephane-Guedon.vcf
clé publique gpg : http://www.22decembre.eu/downloads/Stephane-Guedon.asc

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-server] Best backup service?

[Kristjan Kalder]

Hey,

I'm currently looking for a good backup service so I could do 
centralized backups of all my Windows and Linux clients.
I don't have any tape devices so the backups have to go on a hard drive.
What would you suggest?

-- 
Regards,
Kristjan Kalder

Re: [gentoo-server] Best backup service?

[Jeff Rooney]

[-- Attachment #1: Type: text/plain, Size: 399 bytes --]

Bacula?

Jeff Rooney
jtrooney@nexdlevel.com


On Wed, May 25, 2011 at 3:01 PM, Kristjan Kalder <jolinar1@gmail.com> wrote:

> Hey,
>
> I'm currently looking for a good backup service so I could do centralized
> backups of all my Windows and Linux clients.
> I don't have any tape devices so the backups have to go on a hard drive.
> What would you suggest?
>
> --
> Regards,
> Kristjan Kalder
>
>
>

[-- Attachment #2: Type: text/html, Size: 779 bytes --]

Re: [gentoo-server] Best backup service?

[Denis Bondar]

[-- Attachment #1: Type: text/plain, Size: 395 bytes --]

I'd advise Bacula.
http://bacula.org/

2011/5/26 Kristjan Kalder <jolinar1@gmail.com>

> Hey,
>
> I'm currently looking for a good backup service so I could do centralized
> backups of all my Windows and Linux clients.
> I don't have any tape devices so the backups have to go on a hard drive.
> What would you suggest?
>
> --
> Regards,
> Kristjan Kalder
>
>
>


-- 
Kind regards,
Denis Bondar

[-- Attachment #2: Type: text/html, Size: 760 bytes --]

Re: [gentoo-server] Best backup service?

[John Lowry]

On 05/25/2011 01:17 PM, Denis Bondar wrote:
> I'd advise Bacula.
> http://bacula.org/
> 
> 2011/5/26 Kristjan Kalder <jolinar1@gmail.com <mailto:jolinar1@gmail.com>>
> 
>     Hey,
> 
>     I'm currently looking for a good backup service so I could do
>     centralized backups of all my Windows and Linux clients.
>     I don't have any tape devices so the backups have to go on a hard drive.
>     What would you suggest?
> 
>     -- 
>     Regards,
>     Kristjan Kalder


I heartily third bacula. It can back up to tapes, disk, whatever. It
does not care. You can always start with disk backups and then move to
tapes later.

Re: [gentoo-server] Best backup service?

[flockmock]

On Wednesday 25 May 2011 22:01:12 Kristjan Kalder wrote:
> Hey,
> 
> I'm currently looking for a good backup service so I could do 
> centralized backups of all my Windows and Linux clients.
> I don't have any tape devices so the backups have to go on a hard drive.
> What would you suggest?
> 
> 

bacula, if you want a high-end solution, but keep in mind, that support for backups on disks is implemented via a helper script "vchanger", to emulate tape drives, which makes the whole configuration a little cumbersome and restoring backups is not straightforward, if your database (yes, every file gets recorded separately) is lost (which happend to me). Now I use a much simpler solution: sarab 
I don't know anything about amanda...

Re: [gentoo-server] Best backup service?

[Homer Parker]

On Wed, 2011-05-25 at 23:01 +0300, Kristjan Kalder wrote:
> Hey,
> 
> I'm currently looking for a good backup service so I could do 
> centralized backups of all my Windows and Linux clients.
> I don't have any tape devices so the backups have to go on a hard drive.
> What would you suggest?
> 

	I like BackupPC. 

-- 
Homer Parker <hparker@gentoo.org>

Re: [gentoo-server] Best backup service?

[R. David Murray]

On Wed, 25 May 2011 23:01:12 +0300, Kristjan Kalder <jolinar1@gmail.com> wrote:
> I'm currently looking for a good backup service so I could do 
> centralized backups of all my Windows and Linux clients.
> I don't have any tape devices so the backups have to go on a hard drive.
> What would you suggest?

I started out with bacula, but as someone else said the disk backup
support is cumbersome.  I've since switched to rsync-backup and I love
it.  Combined with one of the web interfaces it basically gives you time
machine for your linux boxes....

--
R. David Murray           http://www.bitdance.com

Re: [gentoo-server] Best backup service?

[Kristjan Kalder]

But is it capable of backing up my windows machines?
At the time when I asked the question I was reading baculas and amandas 
documentations and could not decide. Now I am even more confused. :)

On 26.05.2011 6:53, R. David Murray wrote:
> On Wed, 25 May 2011 23:01:12 +0300, Kristjan Kalder<jolinar1@gmail.com>  wrote:
>> I'm currently looking for a good backup service so I could do
>> centralized backups of all my Windows and Linux clients.
>> I don't have any tape devices so the backups have to go on a hard drive.
>> What would you suggest?
> I started out with bacula, but as someone else said the disk backup
> support is cumbersome.  I've since switched to rsync-backup and I love
> it.  Combined with one of the web interfaces it basically gives you time
> machine for your linux boxes....
>
> --
> R. David Murray           http://www.bitdance.com
>

AW: [gentoo-server] Best backup service?

[Zemke, Kai]

Rsync is also available for windows:
http://rsync.samba.org/download.html
So it should also be possible to backup your windows machines.
Im backing up about 20 gentoo servers via rsync today. Since its only used in our datacenter where only gentoo runs I have no experience in using rsync on windows but it looks possible. 

Mit freundlichen Grüßen
Kai Zemke

===========================================================
Email: kai.zemke@smartnet.de
Telefon: +49(40)-5540-0 Telefax: +49(40)-5540-1040 Direkt: +49(40)-5540-1105
===========================================================


-----Ursprüngliche Nachricht-----
Von: Kristjan Kalder [mailto:jolinar1@gmail.com] 
Gesendet: Donnerstag, 26. Mai 2011 06:50
An: gentoo-server@lists.gentoo.org
Betreff: Re: [gentoo-server] Best backup service?

But is it capable of backing up my windows machines?
At the time when I asked the question I was reading baculas and amandas documentations and could not decide. Now I am even more confused. :)

On 26.05.2011 6:53, R. David Murray wrote:
> On Wed, 25 May 2011 23:01:12 +0300, Kristjan Kalder<jolinar1@gmail.com>  wrote:
>> I'm currently looking for a good backup service so I could do 
>> centralized backups of all my Windows and Linux clients.
>> I don't have any tape devices so the backups have to go on a hard drive.
>> What would you suggest?
> I started out with bacula, but as someone else said the disk backup 
> support is cumbersome.  I've since switched to rsync-backup and I love 
> it.  Combined with one of the web interfaces it basically gives you 
> time machine for your linux boxes....
>
> --
> R. David Murray           http://www.bitdance.com
>
**********************************************************************************************
IMPORTANT: The contents of this email and any attachments are confidential. They are intended for the 
named recipient(s) only.
If you have received this email in error, please notify the system manager or the sender immediately and do 
not disclose the contents to anyone or make copies thereof.
*** eSafe scanned this email for viruses, vandals, and malicious content. ***
**********************************************************************************************

Re: [gentoo-server] Best backup service?

[Patrick Nagel]

[-- Attachment #1: Type: text/plain, Size: 1393 bytes --]

Hi,

I'm also using rdiff-backup extensively. And yes, it can backup Windows
boxes, but there are known issues when the target file system is NTFS,
so better avoid that. Should be fine with NTFS as source file system.

Patrick.

P.S.: Btw., great to see this list come to life again :)

On 2011-05-26 12:50, Kristjan Kalder wrote:
> But is it capable of backing up my windows machines?
> At the time when I asked the question I was reading baculas and amandas
> documentations and could not decide. Now I am even more confused. :)
> 
> On 26.05.2011 6:53, R. David Murray wrote:
>> On Wed, 25 May 2011 23:01:12 +0300, Kristjan
>> Kalder<jolinar1@gmail.com>  wrote:
>>> I'm currently looking for a good backup service so I could do
>>> centralized backups of all my Windows and Linux clients.
>>> I don't have any tape devices so the backups have to go on a hard drive.
>>> What would you suggest?
>> I started out with bacula, but as someone else said the disk backup
>> support is cumbersome.  I've since switched to rsync-backup and I love
>> it.  Combined with one of the web interfaces it basically gives you time
>> machine for your linux boxes....
>>
>> -- 
>> R. David Murray           http://www.bitdance.com
>>
> 


-- 
Key ID: 0x86E346D4            http://patrick-nagel.net/key.asc
Fingerprint: 7745 E1BE FA8B FBAD 76AB 2BFC C981 E686 86E3 46D4


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-server] Best backup service?

[R. David Murray]

On Thu, 26 May 2011 07:50:22 +0300, Kristjan Kalder <jolinar1@gmail.com> wrote:
> But is it capable of backing up my windows machines?
> At the time when I asked the question I was reading baculas and amandas 
> documentations and could not decide. Now I am even more confused. :)

Yeah, I used rsync-backup to back up a windows box for a while (until I
got rid of the windows box).  The only tricky bit I ran into was that
the XP SP2 box had a bug in the Windows equivalent of cron that
prevented me from automating the backup.  It wasn't an important enough
box for me to invest in an alternate cron solution so I just manually
clicked the little backup icon I made for my rsync-backup script every
once in a while....a windows server should have no such issues :) 

--
R. David Murray           http://www.bitdance.com

Re: [gentoo-server] Best backup service?

[Konstantin]

Hello Kristjan,

Wednesday, May 25, 2011, 23:01:12, Kristjan Kalder wrote:
> I'm currently looking for a good backup service so I could do
> centralized backups of all my Windows and Linux clients.
> I don't have any tape devices so the backups have to go on a hard drive.
> What would you suggest?

Bacula + webacula (as web interface).

-- 
Konstantin

Re: [gentoo-server] Best backup service?

[nirn]

you can try duplicity :)
http://duplicity.nongnu.org/

2011/5/26 Konstantin <konstantin@astafjev.com>:
> Hello Kristjan,
>
> Wednesday, May 25, 2011, 23:01:12, Kristjan Kalder wrote:
>> I'm currently looking for a good backup service so I could do
>> centralized backups of all my Windows and Linux clients.
>> I don't have any tape devices so the backups have to go on a hard drive.
>> What would you suggest?
>
> Bacula + webacula (as web interface).
>
> --
> Konstantin
>
>
>

Re: [gentoo-server] Managing multiple servers.

[Jean-François Maeyhieux]

Hi !

I think you may be use this old script as i get used several years ago:

website: http://www.panhorst.com/glcu/
ebuild: http://bugs.gentoo.org/show_bug.cgi?id=101827


Hopping this script could help you... It manage daily update
(sync,build) and report via cron/mail. So you've just to install
pre-built package that have been prepared on a daily frequency when you
decide it's ok to do it without lost time. A revdep-rebuild and commit
of new configuration file using a configured dispatch-conf later, your
machine is update.




Le mercredi 25 mai 2011 à 20:43 +0200, Nils a écrit :
>         When I was managing multiple servers, I would have a cron job
>         set to eix-sync daily (the maximum frequency Gentoo recommends
>         syncing), then once a week I would have emerge -DNpvu world
>         e-mailed from cron.  I would then review the e-mail and see if
>         I wanted to upgrade.  If I didn't, I'd mask the package
>         version and move on.
> 
> May I ask why would would sync once a day but only run emerge once a
> week?

-- 
------------------------------------
  Jean-François Maeyhieux
     Atanar Technologies
    http://www.atanar.com
 Phone: +33 6 59 31 55 63
------------------------------------

Re: [gentoo-server] Managing multiple servers.

[Sven Vermeulen]

[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]

2011/5/30 Jean-François Maeyhieux <b4b1@free.fr>

> I think you may be use this old script as i get used several years ago:
>
> website: http://www.panhorst.com/glcu/
> ebuild: http://bugs.gentoo.org/show_bug.cgi?id=101827
>
>
> Hopping this script could help you... It manage daily update
> (sync,build) and report via cron/mail. So you've just to install
> pre-built package that have been prepared on a daily frequency when you
> decide it's ok to do it without lost time. A revdep-rebuild and commit
> of new configuration file using a configured dispatch-conf later, your
> machine is update.
>
>
I wrote (and still maintain) a package called cvechecker (
http://cvechecker.sourceforge.net) whose purpose is to scan the system for
installed software (or you use a simple file that tells the application what
is installed so systemwide scans aren't needed then anymore) and pull in
information from NVD about CVE entries. It then matches the CVE entries with
the detected software/versions on your system and report which ones might be
affected by a known vulnerability.

Wkr,
  Sven Vermeulen

[-- Attachment #2: Type: text/html, Size: 1602 bytes --]

Re: [gentoo-server] Managing multiple servers.

[David]

[-- Attachment #1: Type: text/plain, Size: 1441 bytes --]

Since Gentoo does not report security issues very frequently, having a
Gentoo package/ebuild for 'cvechecker' seems ideal.

Perhaps 'cvechecker' would also help the Gentoo team that announces
vulnerabilities.





2011/6/2 Sven Vermeulen <sven.vermeulen@siphos.be>

> 2011/5/30 Jean-François Maeyhieux <b4b1@free.fr>
>
>> I think you may be use this old script as i get used several years ago:
>>
>> website: http://www.panhorst.com/glcu/
>> ebuild: http://bugs.gentoo.org/show_bug.cgi?id=101827
>>
>>
>> Hopping this script could help you... It manage daily update
>> (sync,build) and report via cron/mail. So you've just to install
>> pre-built package that have been prepared on a daily frequency when you
>> decide it's ok to do it without lost time. A revdep-rebuild and commit
>> of new configuration file using a configured dispatch-conf later, your
>> machine is update.
>>
>>
> I wrote (and still maintain) a package called cvechecker (
> http://cvechecker.sourceforge.net) whose purpose is to scan the system for
> installed software (or you use a simple file that tells the application what
> is installed so systemwide scans aren't needed then anymore) and pull in
> information from NVD about CVE entries. It then matches the CVE entries with
> the detected software/versions on your system and report which ones might be
> affected by a known vulnerability.
>
> Wkr,
>   Sven Vermeulen
>

[-- Attachment #2: Type: text/html, Size: 2202 bytes --]

Re: [gentoo-server] Managing multiple servers.

[Christian Kauhaus]

Am 02.06.2011 21:31, schrieb Sven Vermeulen:
> I wrote (and still maintain) a package called cvechecker
> (http://cvechecker.sourceforge.net) whose purpose is to scan the system for
> installed software (or you use a simple file that tells the application what
> is installed so systemwide scans aren't needed then anymore) and pull in
> information from NVD about CVE entries. It then matches the CVE entries with
> the detected software/versions on your system and report which ones might be
> affected by a known vulnerability.

This looks really interesting. As an admin in a larger installation, I must 
evaluate the impact of possible security problems on our infrastructure by 
myself anyway. This tool could give a good basis in form of early warnings.

Has anyone already written an ebuild?

Regards

Christian

-- 
Dipl.-Inf. Christian Kauhaus <>< · kc@gocept.com · systems administration
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 11 · fax +49 345 1229889 1
Zope and Plone consulting and development

Re: [gentoo-server] Managing multiple servers.

[Joost Roeleveld]

On Friday 03 June 2011 11:37:28 Christian Kauhaus wrote:
> Am 02.06.2011 21:31, schrieb Sven Vermeulen:
> > I wrote (and still maintain) a package called cvechecker
> > (http://cvechecker.sourceforge.net) whose purpose is to scan the system
> > for installed software (or you use a simple file that tells the
> > application what is installed so systemwide scans aren't needed then
> > anymore) and pull in information from NVD about CVE entries. It then
> > matches the CVE entries with the detected software/versions on your
> > system and report which ones might be affected by a known
> > vulnerability.
> 
> This looks really interesting. As an admin in a larger installation, I must
> evaluate the impact of possible security problems on our infrastructure by
> myself anyway. This tool could give a good basis in form of early warnings.
> 
> Has anyone already written an ebuild?

Yes, the location for this is listed on the website:
http://cvechecker.sourceforge.net/install.html#gentoo

--
Joost

Re: [gentoo-server] Managing multiple servers.

[Christian Kauhaus]

Am 03.06.2011 12:49, schrieb Joost Roeleveld:
> Yes, the location for this is listed on the website:
> http://cvechecker.sourceforge.net/install.html#gentoo

Right... I'd better had a closer look at the website. Thanks anyway. I'll give 
it a try.

Regards

Christian

-- 
Dipl.-Inf. Christian Kauhaus <>< · kc@gocept.com · systems administration
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 11 · fax +49 345 1229889 1
Zope and Plone consulting and development

Re: [gentoo-server] Managing multiple servers.

[kashani]

On 5/23/2011 3:12 PM, la Bigmac wrote:
> Hello list,
>
> Seems to be a few people recently wanting to discuss Gentoo as a server
> :-) so thought I would pose a question that has been bugging me.
>
> What would you guys recommend to manage multiple servers and the package
> versions?
>
> While I have a central emerge server (*rsync)* and sync all of my
> servers to it I still manually update the packages.
>
> Example, openssh how should I be updating openssh on all of my servers
> other than logging onto each one in turn and running emerge openssh.
>
> Should I cron schedule an /emerge/ --/update world /and control the
> repository of packages or is there a more elegant solution?

	I've become a huge Puppet nerd over the last year. I'm not managing 
Gentoo on it, but it's supported and Puppet Labs does seem to fix Gentoo 
bugs in a reasonable time.

First you'll need Ruby 1.8.7 as 1.9.2 support in Ruby isn't quite there 
yet. I'd also run unstable for Puppet and Facter. You're better off 
jumping in at 2.6.x than 0.25.x.

Puppet requires facter which is very cool in it's own right. It's local 
discovery of the OS and those facts about your system can be used in 
templates to make decisions. Here's an example for setting higher 
thresholds on my large machines.

<% if processorcount.to_i >= 12 then -%>

and here's an example of a module to make sure sudo is the latest 
version and add a config file for my local sudoers additions.

class sudo {
   package { "sudo": ensure => latest, }

   file { "/etc/sudoers.d/my_additions":
     ensure  => present,
     owner   => root, group => root, mode => 440,
     require => Package["sudo"],
     source  => "puppet:///modules/sudo/my_additions",
   }
}

	In order to make this work you'd really need to have modules for each 
package in your world file and set ensure => latest rather than just 
present. However it does make it easy to keep configs, users, settings, 
which packages in sync across machines.

	That's Puppet in a very very tiny nutshell. They are some unique 
challenges with using it well with Gentoo, but it would ultimately make 
your system easier to reproduce.

kashani

Re: [gentoo-server] Managing multiple servers.

[Fabiano - deStilaDo]

Some more ideas:

- Local copy dist files, two very easy ways to do without mirroring a
huge official gentoo server:
  - Networked DISTDIR
  - First server on GENTOO_MIRRORS, like "http://10.0.0.2/gentoo"
    - This is interesting if you have a "master" gentoo server and
other misc gentoos, for example I do this on my laptop, if I am on my
local net it tries my local server first, the distfiles I want are
usually there, if not, portage gets s 404 a tries the next server. The
same happens when I am away from my local net.

- Use you own binary packages: This saves some nice compile time, but
the binary optimization has to be a common denominator for all the
architectures in use, or have different binary repositories for
different arches if they are "really" different (i.e. incompatible).
For example, if you have intel and amd server you can optimize to
i686. I like better this approach on more homogeneous setups, like
everything optimized for say core 2.
  - Other consideration here is that when you use binary packages you
fix the use flags for that packages, but gentoo handles this very
nicely: when your use flags match the binary packages it uses the
binary, if you flags are different it use binary for the packages that
has the same flags and compiles only the packages that differs. I find
this very good, as in my server more the 95% of packages usually don't
need to be recompiled.
  - If you a have "master" gentoo server for bin packages, you can use
distcc on it to have the packages build very fast (distributed grid
compilation) and only one time, as the other servers then just
download a bin copy and unpack/install. This "master" server or
compiling server does not necessary need to have all the bin packages
installed, as you can use --buildpkgonly.

- Reduce upgrade downtime building binary packages before install and
have a revert point: some services are safe to upgrade and restart
after emerge of new versions are complete, but what if the new version
does not start for some reason? And for service that you cannot
upgrade while running, it is like this: service stop, emerge take some
compile time = downtime, service start.
  Everyone knows that on a perfect world you should not be upgrading
production servers directly, but test the upgrades on testing
servers/environments and only then put the well tested stuff in
production. But lets face it: just a few of us are able to do that in
real world, we are usually overloaded/underpaid sysadmins, have time
constrains, lack of man power, etc. So what is feasible to do being
short on time/resources?
 - Here goes my favourite approach:
   - First backup every affected package with:quickpkg --include-config=y
     - This makes it very easy to revert a unsuccessfully upgrade and
usually is sufficient to revert, but special attention must me given
to programs/services that uses files not save as config files (like
databases for examples).
     - emerge with  --buildpkgonly, this way a bin package is built
but not installed, while the services are running.
     - now, the upgrade is much faster: service stop, emerge bin
package = very fast tar unpack, service start. If service does not
start, emerge very fast unpack time of previous binary backup version,
service start.
     - this can be easy automated with shell scripts (or say,
semi-automated, as the should ask for confirmation on critical
operations)

- Mail GLSA affected: it was mentioned on this list before to cron
emerge update (or eix-sync). After every update I'd add glsa-check to
e-mail me affected packages (security is never too much =)

- Gentoo server "template": many like to have a stage4 to backup or
replicate servers and customize. This is good to have cd/dvd copy in
case of catastrophic raid/backup servers failure, but I enjoy other
approach as well: I have a "template" root of a generic gentoo in one
of my file servers, I find this very handy and flexible. How I use it:
  - Need new server: boot with gentoo minimal (or better sysresccd)
  - Partition it the way is more appropriate to that server (which is
usually very different among all my servers), usually this is done on
top of some raid.
  - Mount partitions, rsync the "tamplate" server to mounted partitions
  - Change unique configs, like hostname and ip
  - chroot, check if kernel config is appropriate for that machine, if
not ajust and recompile
  - grub install
  - reboot
  Enjoy a new server up and running with most of the things already
configured to you linking. I like this approach because I can change
things directly on the "template" server which I think should apply to
all new servers, and also I very easy regularly update it, just chroot
and emerge world.
  - You can also use this approach to clone a running server, but then
you need a few more tricks, specially to rsync special dirs, like
/dev, temporaries, exclude ssh keys, etc. I can provides my details on
how I do this if someone is interested.

- Versioned configs: you can put config dirs (like /etc) under version
control, like subversion or git. This makes it easy to track changes
and do reverts if needed. In case of polytheistic environments (you
are not the only god, there are other sysadmins) this is also a good
way to track who changed what, why and when.


So, and you guys, what are you gentoo-server tricks?

Regards,

Fabiano.



On Mon, May 23, 2011 at 9:40 PM, kashani <kashani-list@badapple.net> wrote:
> On 5/23/2011 3:12 PM, la Bigmac wrote:
>>
>> Hello list,
>>
>> Seems to be a few people recently wanting to discuss Gentoo as a server
>> :-) so thought I would pose a question that has been bugging me.
>>
>> What would you guys recommend to manage multiple servers and the package
>> versions?
>>
>> While I have a central emerge server (*rsync)* and sync all of my
>> servers to it I still manually update the packages.
>>
>> Example, openssh how should I be updating openssh on all of my servers
>> other than logging onto each one in turn and running emerge openssh.
>>
>> Should I cron schedule an /emerge/ --/update world /and control the
>> repository of packages or is there a more elegant solution?
>
>        I've become a huge Puppet nerd over the last year. I'm not managing
> Gentoo on it, but it's supported and Puppet Labs does seem to fix Gentoo
> bugs in a reasonable time.
>
> First you'll need Ruby 1.8.7 as 1.9.2 support in Ruby isn't quite there yet.
> I'd also run unstable for Puppet and Facter. You're better off jumping in at
> 2.6.x than 0.25.x.
>
> Puppet requires facter which is very cool in it's own right. It's local
> discovery of the OS and those facts about your system can be used in
> templates to make decisions. Here's an example for setting higher thresholds
> on my large machines.
>
> <% if processorcount.to_i >= 12 then -%>
>
> and here's an example of a module to make sure sudo is the latest version
> and add a config file for my local sudoers additions.
>
> class sudo {
>  package { "sudo": ensure => latest, }
>
>  file { "/etc/sudoers.d/my_additions":
>    ensure  => present,
>    owner   => root, group => root, mode => 440,
>    require => Package["sudo"],
>    source  => "puppet:///modules/sudo/my_additions",
>  }
> }
>
>        In order to make this work you'd really need to have modules for each
> package in your world file and set ensure => latest rather than just
> present. However it does make it easy to keep configs, users, settings,
> which packages in sync across machines.
>
>        That's Puppet in a very very tiny nutshell. They are some unique
> challenges with using it well with Gentoo, but it would ultimately make your
> system easier to reproduce.
>
> kashani
>
>

Re: [gentoo-server] Managing multiple servers.

[Pandu Poluan]

WHOA... lots of nice ideas here...

On Tue, May 24, 2011 at 09:44, Fabiano - deStilaDo
<fabianoengler@gmail.com> wrote:
> Some more ideas:
>
> - Local copy dist files, two very easy ways to do without mirroring a
> huge official gentoo server:
>  - Networked DISTDIR
>  - First server on GENTOO_MIRRORS, like "http://10.0.0.2/gentoo"
>

Yup, that's what I did on my in-company Gentoo servers. For the
cloud-based ones, I just use the Gentoo mirrors (they have *huge*
bandwidth and I'm not capped/throttled).

> - Use you own binary packages: This saves some nice compile time, but
> the binary optimization has to be a common denominator for all the
> architectures in use, or have different binary repositories for
> different arches if they are "really" different (i.e. incompatible).
> For example, if you have intel and amd server you can optimize to
> i686. I like better this approach on more homogeneous setups, like
> everything optimized for say core 2.

Core 2 optimizations work for AMD Opterons?

>  - Here goes my favourite approach:
>   - First backup every affected package with:quickpkg --include-config=y
>     - This makes it very easy to revert a unsuccessfully upgrade and
> usually is sufficient to revert, but special attention must me given
> to programs/services that uses files not save as config files (like
> databases for examples).
>     - emerge with  --buildpkgonly, this way a bin package is built
> but not installed, while the services are running.
>     - now, the upgrade is much faster: service stop, emerge bin
> package = very fast tar unpack, service start. If service does not
> start, emerge very fast unpack time of previous binary backup version,
> service start.
>     - this can be easy automated with shell scripts (or say,
> semi-automated, as the should ask for confirmation on critical
> operations)
>

Very nice tip, thanks!

Will certainly do that for my next updates :-)

> - Versioned configs: you can put config dirs (like /etc) under version
> control, like subversion or git. This makes it easy to track changes
> and do reverts if needed. In case of polytheistic environments (you
> are not the only god, there are other sysadmins) this is also a good
> way to track who changed what, why and when.
>

Interesting... how do I put /etc under svn/git?


Rgds,
-- 
Pandu E Poluan
~ IT Optimizer ~
Visit my Blog: http://pepoluan.posterous.com
Google Talk:    pepoluan
Y! messenger: pepoluan
MSN / Live:      pepoluan@hotmail.com (do not send email here)
Skype:            pepoluan
More on me:  My LinkedIn Account  My Facebook Account

Re: [gentoo-server] Managing multiple servers.

[Fabiano - deStilaDo]

On Tue, May 24, 2011 at 2:32 AM, Pandu Poluan <pandu@poluan.info> wrote:
> WHOA... lots of nice ideas here...
>
> On Tue, May 24, 2011 at 09:44, Fabiano - deStilaDo
> <fabianoengler@gmail.com> wrote:
>> For example, if you have intel and amd server you can optimize to
>> i686. I like better this approach on more homogeneous setups, like
>> everything optimized for say core 2.
>
> Core 2 optimizations work for AMD Opterons?

By homogeneous setup I meant, if all servers are Core 2, you can
obviously take advantage of this by optimizing everything to Core 2.

But since you asked if Core 2 optimizations work for AMD Opteron, well
it /may/, depends. There are two levels of instruction (cpu)
optimization on GCC, -march and -mtune.

-march controls what instructions the compiler can produce, so this
breaks binary compatibility. For example, trying to run a binary
compiled with -march=core2 on an AMD processor should fail, as the
binary may have instructions not implemented by AMD.

And there is -mtune optimization level, this optimization is
restricted by the instruction set specified by -march, and thus is
binary compatible with it. For example, you can compile the binaries
with -march=i686 and -mtune=core2, now it run on both Intel and AMD,
but will be more optimized for Core 2 processors but should run on any
processor that implements i686 and above, like k6 and pentium pro.


Fabiano.

Re: [gentoo-server] Managing multiple servers.

[kashani]

On 5/23/2011 3:12 PM, la Bigmac wrote:
> Hello list,
>
> Seems to be a few people recently wanting to discuss Gentoo as a server
> :-) so thought I would pose a question that has been bugging me.
>
> What would you guys recommend to manage multiple servers and the package
> versions?
>
> While I have a central emerge server (*rsync)* and sync all of my
> servers to it I still manually update the packages.
>
> Example, openssh how should I be updating openssh on all of my servers
> other than logging onto each one in turn and running emerge openssh.
>
> Should I cron schedule an /emerge/ --/update world /and control the
> repository of packages or is there a more elegant solution?

	I've become a huge Puppet nerd over the last year. I'm not managing 
Gentoo on it, but it's supported and Puppet Labs does seem to fix Gentoo 
bugs in a reasonable time.

First you'll need Ruby 1.8.7 as 1.9.2 support in Ruby isn't quite there 
yet. I'd also run unstable for Puppet and Facter. You're better off 
jumping in at 2.6.x than 0.25.x.

	Puppet requires Facter which is very cool in it's own right. It's local 
discovery of the OS and those facts about your system can be used in 
templates to make decisions. Just run facter from the command line to 
see what's available. Here's an example for setting higher thresholds on 
my large machines where processorcount is a fact.

<% if processorcount.to_i >= 12 then -%>

and here's an example of a module to make sure sudo is the latest 
version and add a config file for my local sudoers additions.

class sudo {
   package { "sudo": ensure => latest, }

   file { "/etc/sudoers.d/my_additions":
     ensure  => present,
     owner   => root, group => root, mode => 440,
     require => Package["sudo"],
     source  => "puppet:///modules/sudo/my_additions",
   }
}

	In order to make this work you'd really need to have modules for each 
package in your world file and set ensure => latest rather than just 
present. However it does make it easy to keep configs, users, settings, 
which packages in sync across machines. You can also specify ensure => 
"2.1.2-r12" as a version instead of present or latest.

	That's Puppet in a very very tiny nutshell. They are some unique 
challenges with using it well with Gentoo, but it would ultimately make 
your system easier to reproduce. You would still need to decide how and 
when to run Puppet on each node.

kashani

Re: [gentoo-server] Managing multiple servers.

[Matthew Marlowe]

> 
> 	I've become a huge Puppet nerd over the last year. I'm not managing
> Gentoo on it, but it's supported and Puppet Labs does seem to fix Gentoo
> bugs in a reasonable time.
> 

+1 for puppet

Some other things of note:
* #puppet channel on freenode is very active and friendly.
* Puppet devs seem to be open to accepting "gentoo specific" patches, although 
they admit their user base is much more focused on binary distributions and 
they may need others to spearhead gentoo specific work.
* A good amount of puppet development occurs on github and can be followed
* If there isn't already a gentoo specific module for something you want, it 
isn't hard at all to write your own or to fork from someone else's code via 
github/etc

In my mind, puppet is a very good match for gentoo servers, because once again 
it allows extensive customizations/flexibility, is community driven, and the 
user base is very active and growing (pretty much the reasons I chose to use 
gentoo for servers in the first place).  I've also been using it for a few 
years now, and while I have to constantly work around the binary distribution 
biases inherit to it, it has yet to let me down and is generally silently 
reliable.  That it integrates ruby and git are also, in my mind, advantages.

Matt

Re: [gentoo-server] Managing multiple servers.

[Ramon van Alteren]

Hey,

This list seems to have woken up suddenly again, good news :)

On Tue, May 24, 2011 at 00:12, la Bigmac <la_bigmac@hotmail.com> wrote:
> Seems to be a few people recently wanting to discuss Gentoo as a server :-)
> so thought I would pose a question that has been bugging me.
>
> What would you guys recommend to manage multiple servers and the package
> versions?

We manage 3000+ servers and use puppet for that, but it is still
painful with gentoo at times.
Especially the moving portage tree forces us to keep a local frozen
version that retains both the ebuilds and the distfiles. That is not
so much of a problem, it is the unfreezing that is causing us grief.

In addition to puppet and our own frozen portage snapshot+overlay, we
use agaffneys install scripting to install servers over the net + a
standard tftp + dhcp netbooting setup.

> While I have a central emerge server (rsync) and sync all of my servers to
> it I still manually update the packages.
>
> Example, openssh how should I be updating openssh on all of my servers other
> than logging onto each one in turn and running emerge openssh.

Puppet takes care of that for us and this is a major relief,  having
useflag support in the puppet gentoo package provider would be nice,
but not really necessary. I'd prefer having useflag awareness in
binpkgs and the ability to produce different binpkgs for different
useflag sets in portage.

> Should I cron schedule an emerge --update world  and control the repository
> of packages or is there a more elegant solution?

I've never dared to schedule an emerge --update world on my servers
even though I control the repository, I'm just not that confident of
my own abilities :)

I'm / we are currently working on a new setup for our frozen snapshot
+ overlay + distfiles mirror.
Ideally we're shooting for a setup were we continuously test the
system set of packages for compilation and maybe a few functional
test-cases (can I ssh into the server comes to mind :)

This would allow us to stay closer to $UPSTREAM, in this case gentoo
and avoid the large maintenance nightmare we have now. If possible we
would possibly be looking to open up the infrastructure as open source
and allow others to use it. However that is a future hope, not a
promise :)

Regards,

Ramon van Alteren
Senior System Engineer Hyves.nl

Re: [gentoo-server] Managing multiple servers.

[Pandu Poluan]

On Tue, May 24, 2011 at 14:37, Ramon van Alteren <ramon@vanalteren.nl> wrote:
> Hey,
>
> This list seems to have woken up suddenly again, good news :)
>

About time, I should say...

There have been.... let's say, 'doubts' as to the suitability of
Gentoo as servers.

> On Tue, May 24, 2011 at 00:12, la Bigmac <la_bigmac@hotmail.com> wrote:
>> While I have a central emerge server (rsync) and sync all of my servers to
>> it I still manually update the packages.
>>
>> Example, openssh how should I be updating openssh on all of my servers other
>> than logging onto each one in turn and running emerge openssh.
>
> Puppet takes care of that for us and this is a major relief,  having
> useflag support in the puppet gentoo package provider would be nice,
> but not really necessary. I'd prefer having useflag awareness in
> binpkgs and the ability to produce different binpkgs for different
> useflag sets in portage.
>

So, do you think it will be wise to create a management tool
explicitly for Gentoo (with its quirks such as ~masks, USE flags,
portage/env, and so-on), or just rely on Puppet?

Rgds,
-- 
Pandu E Poluan
~ IT Optimizer ~
Visit my Blog: http://pepoluan.posterous.com
Google Talk:    pepoluan
Y! messenger: pepoluan
MSN / Live:      pepoluan@hotmail.com (do not send email here)
Skype:            pepoluan
More on me:  My LinkedIn Account  My Facebook Account

Re: [gentoo-server] Managing multiple servers.

[Robert Bridge]

On Tue, May 24, 2011 at 9:21 AM, Pandu Poluan <pandu@poluan.info> wrote:
> So, do you think it will be wise to create a management tool
> explicitly for Gentoo (with its quirks such as ~masks, USE flags,
> portage/env, and so-on), or just rely on Puppet?

Without commenting on the wisdom of creating a gentoo specific
management tool, one observation I would make is that Puppet and Chef
are both written in Ruby. For some, this won't be an issue, but for
others, the requirement for Ruby and Python on production servers will
be a concern.

Re: [gentoo-server] Managing multiple servers.

[Pandu Poluan]

On Tue, May 24, 2011 at 16:19, Robert Bridge <robert@robbieab.com> wrote:
> On Tue, May 24, 2011 at 9:21 AM, Pandu Poluan <pandu@poluan.info> wrote:
>> So, do you think it will be wise to create a management tool
>> explicitly for Gentoo (with its quirks such as ~masks, USE flags,
>> portage/env, and so-on), or just rely on Puppet?
>
> Without commenting on the wisdom of creating a gentoo specific
> management tool, one observation I would make is that Puppet and Chef
> are both written in Ruby. For some, this won't be an issue, but for
> others, the requirement for Ruby and Python on production servers will
> be a concern.
>
>

Then again, you can expect Python to exist on a Gentoo server, no?

-- 
Pandu E Poluan
~ IT Optimizer ~
Visit my Blog: http://pepoluan.posterous.com

Re: [gentoo-server] Managing multiple servers.

[kashani]

On 5/24/2011 2:19 AM, Robert Bridge wrote:
> On Tue, May 24, 2011 at 9:21 AM, Pandu Poluan<pandu@poluan.info>  wrote:
>> So, do you think it will be wise to create a management tool
>> explicitly for Gentoo (with its quirks such as ~masks, USE flags,
>> portage/env, and so-on), or just rely on Puppet?
>
> Without commenting on the wisdom of creating a gentoo specific
> management tool, one observation I would make is that Puppet and Chef
> are both written in Ruby. For some, this won't be an issue, but for
> others, the requirement for Ruby and Python on production servers will
> be a concern.
>

	Portage and RHEL need Python. Having your configuration management 
system require a high level language doesn't seem like much of a 
stretch. Access to a full language within my templates has increased the 
things I no longer have to manage. For example in my Mysql template I 
pump the IP through a function that spits out a 32bit integer to set 
server_id.

server_id = <%= ipaddress.split('.').inject(0) {|total,value| (total << 
8 ) + value.to_i} %>

I'm not sure I could do it in Bash and I was able to write it myself 
rather than waiting for someone to add it to the management system.

	I can't imagine any management system without the equivalent of a full 
language, but maybe I'm not imaginative enough.

kashani

Re: [gentoo-server] Managing multiple servers.

[Ramon van Alteren]

Hi,

On Tue, May 24, 2011 at 10:21, Pandu Poluan <pandu@poluan.info> wrote:
> On Tue, May 24, 2011 at 14:37, Ramon van Alteren <ramon@vanalteren.nl> wrote:
>> This list seems to have woken up suddenly again, good news :)
> About time, I should say...

Yeah :)
>
> There have been.... let's say, 'doubts' as to the suitability of
> Gentoo as servers.

And they are well-founded in many cases IMHO, not many shops have the
expertise and the guts to deal with a moving target such as the
portage tree is and it will bite them eventually. On top of that I
think there are very few shops that need the flexibility and
malleability of gentoo. So that seems like a nice fit.

I have always viewed gentoo as a developers distro which allows you to
stay on the bleeding edge with as little effort as possible. If you do
not need that functionality, use *fill in favorite distro name here*

>> On Tue, May 24, 2011 at 00:12, la Bigmac <la_bigmac@hotmail.com> wrote:
>>> While I have a central emerge server (rsync) and sync all of my servers to
>>> it I still manually update the packages.
>>>
>>> Example, openssh how should I be updating openssh on all of my servers other
>>> than logging onto each one in turn and running emerge openssh.
>>
>> Puppet takes care of that for us and this is a major relief,  having
>> useflag support in the puppet gentoo package provider would be nice,
>> but not really necessary. I'd prefer having useflag awareness in
>> binpkgs and the ability to produce different binpkgs for different
>> useflag sets in portage.
>
> So, do you think it will be wise to create a management tool
> explicitly for Gentoo (with its quirks such as ~masks, USE flags,
> portage/env, and so-on), or just rely on Puppet?

No, i think it would pay off to take a look at adding a specific
provider in puppet for portage that exposes more of the unique
functionality of portage to the puppet manifest writer.

Ramon van Alteren
Senior System Engineer Hyves.nl

Re: [gentoo-server] Managing multiple servers.

[Hacking Network Solutions - Gentoo List Subscriptions]

On Tue, 2011-05-24 at 09:37 +0200, Ramon van Alteren wrote:
> Hey,
> 
> This list seems to have woken up suddenly again, good news :)
> 

I would agree, it's always nice to have a useful Gentoo oriented
discussion, especially about something which isn't desktop related.

> On Tue, May 24, 2011 at 00:12, la Bigmac <la_bigmac@hotmail.com> wrote:
> > Seems to be a few people recently wanting to discuss Gentoo as a server :-)
> > so thought I would pose a question that has been bugging me.
> >
> > What would you guys recommend to manage multiple servers and the package
> > versions?
> 
> We manage 3000+ servers and use puppet for that, but it is still
> painful with gentoo at times.
> Especially the moving portage tree forces us to keep a local frozen
> version that retains both the ebuilds and the distfiles. That is not
> so much of a problem, it is the unfreezing that is causing us grief.
> 
> In addition to puppet and our own frozen portage snapshot+overlay, we
> use agaffneys install scripting to install servers over the net + a
> standard tftp + dhcp netbooting setup.
> 
> > While I have a central emerge server (rsync) and sync all of my servers to
> > it I still manually update the packages.
> >

I hope this shameless plug for my company's website doesn't offend but
we provide a set of packages and some documentation for those wishing to
automate the process of building and testing binary packages on a
central server.  More information can be found at....

http://www.mad-hacking.net/documentation/linux/deployment/buildserver/

> > Example, openssh how should I be updating openssh on all of my servers other
> > than logging onto each one in turn and running emerge openssh.
> 
> Puppet takes care of that for us and this is a major relief,  having
> useflag support in the puppet gentoo package provider would be nice,
> but not really necessary. I'd prefer having useflag awareness in
> binpkgs and the ability to produce different binpkgs for different
> useflag sets in portage.
> 
> > Should I cron schedule an emerge --update world  and control the repository
> > of packages or is there a more elegant solution?
> 
> I've never dared to schedule an emerge --update world on my servers
> even though I control the repository, I'm just not that confident of
> my own abilities :)

This is usually a bad idea.  Quite often an "emerge --update world" will
leave a system in an unstable state until configuration files are
updated and reverse dependencies checked.  This can be ameliorated by
using a system as described in the link above as all packages which have
been rebuilt on the "build server" will be updated by an "emerge
--update world".  Configuration files will still be a problem however.

> I'm / we are currently working on a new setup for our frozen snapshot
> + overlay + distfiles mirror.
> Ideally we're shooting for a setup were we continuously test the
> system set of packages for compilation and maybe a few functional
> test-cases (can I ssh into the server comes to mind :)
> 
> This would allow us to stay closer to $UPSTREAM, in this case gentoo
> and avoid the large maintenance nightmare we have now. If possible we
> would possibly be looking to open up the infrastructure as open source
> and allow others to use it. However that is a future hope, not a
> promise :)
> 
> Regards,
> 
> Ramon van Alteren
> Senior System Engineer Hyves.nl
> 

Re: [gentoo-server] Managing multiple servers.

[Christian Kauhaus]

Am 24.05.2011 09:37, schrieb Ramon van Alteren:
> In addition to puppet and our own frozen portage snapshot+overlay, we
> use agaffneys install scripting to install servers over the net + a
> standard tftp + dhcp netbooting setup.

We have a lot of Gentoo servers at our company, too. To get stable installs 
and avoid too frequent updates, we also freeze the portage tree. The problem 
is that some distfiles disappear really soon on the upstream mirrors. Does 
anyone know of a fallback mirror which keeps distfiles longer as usual? If 
there is none, we could possibly provide one by ourselves...

Regards

Christian

-- 
Dipl.-Inf. Christian Kauhaus <>< · kc@gocept.com · systems administration
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 11 · fax +49 345 1229889 1
Zope and Plone consulting and development

Re: [gentoo-server] Managing multiple servers.

[Dawid Węgliński]

On Tuesday 24 of May 2011 16:45:08 Christian Kauhaus wrote:
> Am 24.05.2011 09:37, schrieb Ramon van Alteren:
> > In addition to puppet and our own frozen portage snapshot+overlay, we
> > use agaffneys install scripting to install servers over the net + a
> > standard tftp + dhcp netbooting setup.
> 
> We have a lot of Gentoo servers at our company, too. To get stable installs
> and avoid too frequent updates, we also freeze the portage tree. The
> problem is that some distfiles disappear really soon on the upstream
> mirrors. Does anyone know of a fallback mirror which keeps distfiles
> longer as usual? If there is none, we could possibly provide one by
> ourselves...
> 
> Regards
> 
> Christian

If you have your own rsync server just drop --delete option from scipt, so 
rsync will not remove files that were deleted upstream.

-- 
Cheers
Dawid Węgliński

Re: [gentoo-server] Managing multiple servers.

[Ramon van Alteren]

Hi,

On Tue, May 24, 2011 at 16:45, Christian Kauhaus <kc@gocept.com> wrote:
> Am 24.05.2011 09:37, schrieb Ramon van Alteren:
>>
>> In addition to puppet and our own frozen portage snapshot+overlay, we
>> use agaffneys install scripting to install servers over the net + a
>> standard tftp + dhcp netbooting setup.
>
> We have a lot of Gentoo servers at our company, too. To get stable installs
> and avoid too frequent updates, we also freeze the portage tree. The problem
> is that some distfiles disappear really soon on the upstream mirrors. Does
> anyone know of a fallback mirror which keeps distfiles longer as usual? If
> there is none, we could possibly provide one by ourselves...

Not that I know off, although most of the time you can find older
releases by a project by hand if you dig through the archives etc.
We keep our own stuff on a local mirror. Because we use catalyst to
build a stage4 package set, we download most of the sources anyway
during the build process, it is fairly easy to turn that into a local
mirror

You can also setup a private distfile mirror and not --delete, that
gives you more or less indefinite history of mirror contents.
Given todays hd prices, this will go a long way :)

Ramon
Senior System Engineer Hyves.nl

Re: [gentoo-server] Managing multiple servers.

[Christian Kauhaus]

Am 24.05.2011 23:06, schrieb Ramon van Alteren:
> You can also setup a private distfile mirror and not --delete, that
> gives you more or less indefinite history of mirror contents.
> Given todays hd prices, this will go a long way :)

This is probably the way to go.

Regards

Christian

-- 
Dipl.-Inf. Christian Kauhaus <>< · kc@gocept.com · systems administration
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 11 · fax +49 345 1229889 1
Zope and Plone consulting and development

RE: [gentoo-server] Managing multiple servers.

[Pandu Poluan]

-original message-
Subject: Re: [gentoo-server] Managing multiple servers.
From: kashani <kashani-list@badapple.net>
Date: 2011-05-24 07:40

On 5/23/2011 3:12 PM, la Bigmac wrote:
>> Hello list,
>>
>> Seems to be a few people recently wanting to discuss Gentoo as a server
>> :-) so thought I would pose a question that has been bugging me.
>>
>> What would you guys recommend to manage multiple servers and the package
>> versions?
>>
>> While I have a central emerge server (*rsync)* and sync all of my
>> servers to it I still manually update the packages.
>>
>> Example, openssh how should I be updating openssh on all of my servers
>> other than logging onto each one in turn and running emerge openssh.
>>
>> Should I cron schedule an /emerge/ --/update world /and control the
>> repository of packages or is there a more elegant solution?
>
>I've become a huge Puppet nerd over the last year. I'm not managing 
>Gentoo on it, but it's supported and Puppet Labs does seem to fix Gentoo 
>bugs in a reasonable time.
>
>First you'll need Ruby 1.8.7 as 1.9.2 support in Ruby isn't quite there 
>yet. I'd also run unstable for Puppet and Facter. You're better off 
>jumping in at 2.6.x than 0.25.x.
>
>Puppet requires facter which is very cool in it's own right. It's local 
>discovery of the OS and those facts about your system can be used in 
>templates to make decisions. Here's an example for setting higher 
>thresholds on my large machines.
>
><% if processorcount.to_i >= 12 then -%>
>
>and here's an example of a module to make sure sudo is the latest 
>version and add a config file for my local sudoers additions.
>
>class sudo {
>   package { "sudo": ensure => latest, }
>
>   file { "/etc/sudoers.d/my_additions":
>     ensure  => present,
>     owner   => root, group => root, mode => 440,
>     require => Package["sudo"],
>     source  => "puppet:///modules/sudo/my_additions",
>   }
>}
>
>In order to make this work you'd really need to have modules for each 
>package in your world file and set ensure => latest rather than just 
>present. However it does make it easy to keep configs, users, settings, 
>which packages in sync across machines.
>
>That's Puppet in a very very tiny nutshell. They are some unique 
>challenges with using it well with Gentoo, but it would ultimately make 
>your system easier to reproduce.
>
>kashani

Interesting...

Now, I've never used Puppet before, so please forgive my apparently ignorant questions:

1. How does it handle differing USE flags? Or portage/env variables?

2. How does it update the systems it maintains? I mean, like doing compiling stuffs and such?

3. Does it use SLOTs to enable rollback, just in case?

I'm looking at 8 Gentoo servers -- soon to be 10 -- and I *will* seriously evaluate *anything* that promises to assist me with managing/maintaining them.

Anecdote time: just last week, I tried deploying a Squid proxy. I don't know what happened, but I failed doing that on two other distros; one has a buggy package, and the other just... did not work. It might be my mistake, but I was pressed for time. So I did a 'stage4' installation of Gentoo, emerged squid... and it Just Works(tm). Now I'm setting up two more proxies with Gentoo.

(You might've seen my question regarding configure flags back in [gentoo-user]. Yes, that's me building the 2nd proxy.)

Rgds,
--
Pandu E Poluan
~ IT Optimizer ~

Sent from Nokia E72-1

Re: [gentoo-server] Managing multiple servers.

[Fabiano - deStilaDo]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

On May 24, 2011 5:06 AM, "Ramon van Alteren" <ramon@vanalteren.nl> wrote:
> I'm / we are currently working on a new setup for our frozen snapshot
> + overlay + distfiles mirror.
> Ideally we're shooting for a setup were we continuously test the
> system set of packages for compilation and maybe a few functional
> test-cases (can I ssh into the server comes to mind :)
>
> This would allow us to stay closer to $UPSTREAM, in this case gentoo
> and avoid the large maintenance nightmare we have now. If possible we
> would possibly be looking to open up the infrastructure as open source
> and allow others to use it. However that is a future hope, not a
> promise :)

Did you take a look on the tools that gentoo devs use? If not, you may find
some useful stuff for what you are doing, or at least inspiring, they have
tools for managing repos, automated building/testing of the tree, etc.

__
Fabiano.

/* sent from an android device */

[-- Attachment #2: Type: text/html, Size: 1111 bytes --]