[gentoo-dev] implementation details for GLEP 41

[gentoo-dev] implementation details for GLEP 41

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 1632 bytes --]

Ignoring the yellow star issue, there are a few implementation
concerns/impossibilities with GLEP 41 in its current form. 

For instance, the way GLEP 41 suggests doing r/o cvs is not going to work.
It suggests using a single account and placing an SSH key for each arch
tester in that account's ~/.ssh/authorized_keys file.

There are no provisions for key management and I cannot see an easy way to
handle it.  It's easy to add new keys, but how do we clean out old keys for
retired arch testers?  (including arch testers that "retire" without ever
informing us)  SSH doesn't log key ID as near as I can tell, so we have no
way of tracking what keys are used and how often.  Also, how do we
definitively correlate an SSH key with an arch tester?  

Now, the same question for email -- how do we manage aliases, especially
for inactive, retired and semi-retired arch testers?  We could track usage
in logs, but between mailing list subscriptions, bugzilla notifications and
all sorts of other automated emails, that's not an accurate representation
of whether an email alias is actively used or not.

I talked to Lance and neither he nor I were consulted about this GLEP and
how feasible the implementation is.  We both are quite concerned about the
issues that I've outlined above as well as others.  

This isn't a "we're refusing to implement this GLEP" email, btw, though I'm
sure some of you will take it as such.  It is, however, a "we were never
consulted regarding implementation details, so there are still issues that
need to be worked out before this GLEP can go anywhere" email.  

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Danny van Dyk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Kurt Lieber schrieb:
| Ignoring the yellow star issue, there are a few implementation
| concerns/impossibilities with GLEP 41 in its current form.
|
| For instance, the way GLEP 41 suggests doing r/o cvs is not going to work.
| It suggests using a single account and placing an SSH key for each arch
| tester in that account's ~/.ssh/authorized_keys file.
|
| There are no provisions for key management and I cannot see an easy way to
| handle it.  It's easy to add new keys, but how do we clean out old
keys for
| retired arch testers?  (including arch testers that "retire" without ever
| informing us)  SSH doesn't log key ID as near as I can tell, so we have no
| way of tracking what keys are used and how often.  Also, how do we
| definitively correlate an SSH key with an arch tester?
Do we have to? Nobody has to track how often an Arch Tester uses RO
access to CVS, as you don't need that information. RO CVS access is a
service to the ATs. Their work is pretty much outside CVS...

| Now, the same question for email -- how do we manage aliases, especially
| for inactive, retired and semi-retired arch testers?  We could track usage
| in logs, but between mailing list subscriptions, bugzilla
notifications and
| all sorts of other automated emails, that's not an accurate representation
| of whether an email alias is actively used or not.
Afaik the gentoo.org address is only a forward to their normal adress,
so one can hardly speak 'active usage'. You simply can't actively use
it! On the other hand, tracking down how active/inactive a AT/HT is
falls under the project the AT/HT is associated with, or the AT/HT
Project (hparker) as last resort. So if he says 'AT foo is inactive',
he's to be removed from email forwarding and CVS RO Access. I really
don't see the problem here.

Danny
- --
Danny van Dyk <kugelfang@gentoo.org>
Gentoo/AMD64 Project, Gentoo Scientific Project
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDf2eVaVNL8NrtU6IRAoyTAJ0ey3mRDulIHz2KMtZjCM0zWEOKWwCffHsx
pcnKGFfZ9OoXBRV2RhKKAOU=
=vTjI
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] implementation details for GLEP 41

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 2122 bytes --]

On Sat, Nov 19, 2005 at 06:57:41PM +0100 or thereabouts, Danny van Dyk wrote:
> | There are no provisions for key management and I cannot see an easy way to
> | handle it.  It's easy to add new keys, but how do we clean out old keys for
> | retired arch testers?  (including arch testers that "retire" without ever
> | informing us)  SSH doesn't log key ID as near as I can tell, so we have no
> | way of tracking what keys are used and how often.  Also, how do we
> | definitively correlate an SSH key with an arch tester?
>
> Do we have to? Nobody has to track how often an Arch Tester uses RO
> access to CVS, as you don't need that information. RO CVS access is a
> service to the ATs. Their work is pretty much outside CVS...

Yes, we have to.  If someone retires, their access needs to be revoked.

> | Now, the same question for email -- how do we manage aliases, especially
> | for inactive, retired and semi-retired arch testers?  We could track usage
> | in logs, but between mailing list subscriptions, bugzilla
> notifications and
> | all sorts of other automated emails, that's not an accurate representation
> | of whether an email alias is actively used or not.
> Afaik the gentoo.org address is only a forward to their normal adress,
> so one can hardly speak 'active usage'. You simply can't actively use
> it! On the other hand, tracking down how active/inactive a AT/HT is
> falls under the project the AT/HT is associated with, or the AT/HT
> Project (hparker) as last resort. So if he says 'AT foo is inactive',
> he's to be removed from email forwarding and CVS RO Access. I really
> don't see the problem here.

Because, in practice, this doesn't happen.  Accounts (or, in this case,
email addresses) stay around until someone gets enough of a bee under their
bonnet to do somethig about it.  Since there's no pain or cost for the
AT/HT project lead, there's no reason for them to be vigilant about
tracking activity.  Plus, assuming we have a large number of these testers,
how are people going to know whether or not one specific arch tester is
active?  That's not an acceptable solution.

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Simon Stelling]

Kurt Lieber wrote:
> Because, in practice, this doesn't happen.  Accounts (or, in this case,
> email addresses) stay around until someone gets enough of a bee under their
> bonnet to do somethig about it.  Since there's no pain or cost for the
> AT/HT project lead, there's no reason for them to be vigilant about
> tracking activity.  Plus, assuming we have a large number of these testers,
> how are people going to know whether or not one specific arch tester is
> active?  That's not an acceptable solution.

Uhm, does that implicitly mean there is such a tracking method for devs (where 
dev = dev/staff/whatever)? There are devs who don't have commit permissions to 
any cvs repo, how is their activity tracked?

In the AT case it wouldn't be so hard to check their activity. !seen on IRC and 
a bugzilla query printing out bugs where they made a comment should be enough, IMHO.

-- 
Simon Stelling
Gentoo/AMD64 Operational Co-Lead
blubb@gentoo.org
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] implementation details for GLEP 41

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 2303 bytes --]

On Sat, Nov 19, 2005 at 05:06:15PM +0000, Kurt Lieber wrote:
> For instance, the way GLEP 41 suggests doing r/o cvs is not going to work.
> It suggests using a single account and placing an SSH key for each arch
> tester in that account's ~/.ssh/authorized_keys file.
text in question

"Get read-only access to the gentoo-x86 repository. This doesn't have 
to be individual accounts, a single account, without a shell, with all 
of their keys will be sufficiant."

Note the "doesn't have to be" and "will be sufficient", it's left open 
to how y'all want to implement it.

> There are no provisions for key management and I cannot see an easy way to
> handle it.  It's easy to add new keys, but how do we clean out old keys for
> retired arch testers?  (including arch testers that "retire" without ever
> informing us)  SSH doesn't log key ID as near as I can tell, so we have no
> way of tracking what keys are used and how often.  Also, how do we
> definitively correlate an SSH key with an arch tester?  
> 
> Now, the same question for email -- how do we manage aliases, especially
> for inactive, retired and semi-retired arch testers?  We could track usage
> in logs, but between mailing list subscriptions, bugzilla notifications and
> all sorts of other automated emails, that's not an accurate representation
> of whether an email alias is actively used or not.
> 
> I talked to Lance and neither he nor I were consulted about this GLEP and
> how feasible the implementation is.  We both are quite concerned about the
> issues that I've outlined above as well as others.  
> 
> This isn't a "we're refusing to implement this GLEP" email, btw, though I'm
> sure some of you will take it as such.  It is, however, a "we were never
> consulted regarding implementation details, so there are still issues that
> need to be worked out before this GLEP can go anywhere" email.  

Cvs concerns above are all based upon doing single account for cvs ro; 
again, it's stated as an option (iow, the option is left up to y'all).

It's not mandating anything on you for cvs, reread it if you don't 
believe me.  It's stating the base, that they only need the users to 
have cvs ro access...

Either way, it's word games, and yes, it's kind of retarded.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Sven Vermeulen]

[-- Attachment #1: Type: text/plain, Size: 772 bytes --]

On Sat, Nov 19, 2005 at 05:06:15PM +0000, Kurt Lieber wrote:
> Now, the same question for email -- how do we manage aliases, especially
> for inactive, retired and semi-retired arch testers?  We could track usage
> in logs, but between mailing list subscriptions, bugzilla notifications and
> all sorts of other automated emails, that's not an accurate representation
> of whether an email alias is actively used or not.

Isn't this an issue that also exists for the Gentoo developers in general?

Wkr,
      Sven Vermeulen


-- 
  Gentoo Foundation Trustee          |  http://foundation.gentoo.org
  Gentoo Documentation Project Lead  |  http://www.gentoo.org/proj/en/gdp
  Gentoo Council Member  

  The Gentoo Project   <<< http://www.gentoo.org >>>

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 975 bytes --]

On Sat, Nov 19, 2005 at 08:03:55PM +0100 or thereabouts, Sven Vermeulen wrote:
> On Sat, Nov 19, 2005 at 05:06:15PM +0000, Kurt Lieber wrote:
> > Now, the same question for email -- how do we manage aliases, especially
> > for inactive, retired and semi-retired arch testers?  We could track usage
> > in logs, but between mailing list subscriptions, bugzilla notifications and
> > all sorts of other automated emails, that's not an accurate representation
> > of whether an email alias is actively used or not.
> 
> Isn't this an issue that also exists for the Gentoo developers in general?

Not as much since we can track things like last cvs commit, last login to
toucan, etc.  But it does exist to some extent.

That does not, however, make it acceptable to further exacerbate the
problem.  We're working towards improving the procedures and controls we
have in place today.  We're not going to implement something that will move
us backwards.

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

On Sat, Nov 19, 2005 at 07:14:03PM +0000, Kurt Lieber wrote:
> On Sat, Nov 19, 2005 at 08:03:55PM +0100 or thereabouts, Sven Vermeulen wrote:
> > Isn't this an issue that also exists for the Gentoo developers in general?
> 
> Not as much since we can track things like last cvs commit, last login to
> toucan, etc.  But it does exist to some extent.
> 
> That does not, however, make it acceptable to further exacerbate the
> problem.  We're working towards improving the procedures and controls we
> have in place today.  We're not going to implement something that will move
> us backwards.

I'll again point out that the glep doesn't actually mandate it, states 
it's the lowest common denominator that's acceptable.

Stop pointing at one interpretation of it that sucks, when the glep 
_does_ leave it open to you how to implement it.  It's a waste of 
people's time and bandwidth, and is a bit disenguous.
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]

On Sat, Nov 19, 2005 at 01:51:15PM -0600 or thereabouts, Brian Harring wrote:
> I'll again point out that the glep doesn't actually mandate it, states 
> it's the lowest common denominator that's acceptable.

And I'll point out that there's more than one issue that we're concerned
with here.
  
> Stop pointing at one interpretation of it that sucks, when the glep 
> _does_ leave it open to you how to implement it.  It's a waste of 
> people's time and bandwidth, and is a bit disenguous.

I'm trying to find a solution to the issues as I see them.  Telling me I'm
wasting people's time and bandwidth doesn't seem conducive to working
together towards a resolution to this all.  If you're going to say, "it was
passed, you guys just have to find a way to implement it.  now please stop
bothering us" then I'm going to come up with an implementation plan that
looks something like the following:

* all SSH keys and email addresses for arch testers will auto-expire after
  60 days.  If an arch tester needs to have continued access, a gentoo dev
  will have to re-submit the key and recreate the alias for that arch
  tester every 60 days.

That meets the requirements of the GLEP down to the letter and also
satisfies infra concerns around key management.  However, it's a crappy
solution.

So, I'd much rather work together towards finding a better one.  

--kurt


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Lares Moreau]

[-- Attachment #1: Type: text/plain, Size: 922 bytes --]

On Sat, 2005-11-19 at 22:03 +0000, Kurt Lieber wrote:
> I'm going to come up with an implementation plan that
> looks something like the following:
> 
> * all SSH keys and email addresses for arch testers will auto-expire after
>   60 days.  If an arch tester needs to have continued access, a gentoo dev
>   will have to re-submit the key and recreate the alias for that arch
>   tester every 60 days.

Just a thought, auto-expire after 60 days of non-use. That only applies
to ro CVS access, but perhaps a resonable implimentation.

Email is a different issue, which I do not have a 'firm' opinion on.

-- 
Lares Moreau <lares.moreau@gmail.com>  | LRU: 400755 http://counter.li.org
Gentoo x86 Arch Tester                 |               ::0 Alberta, Canada
Public Key: 0D46BB6E @ subkeys.pgp.net |           Encrypted Mail Prefered
Key fingerprint = 0CA3 E40D F897 7709 3628  C5D4 7D94 483E 0D46 BB6E

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 2191 bytes --]

On Sat, Nov 19, 2005 at 10:03:58PM +0000, Kurt Lieber wrote:
> On Sat, Nov 19, 2005 at 01:51:15PM -0600 or thereabouts, Brian Harring wrote:
> > Stop pointing at one interpretation of it that sucks, when the glep 
> > _does_ leave it open to you how to implement it.  It's a waste of 
> > people's time and bandwidth, and is a bit disenguous.
> 
> I'm trying to find a solution to the issues as I see them.  Telling me I'm
> wasting people's time and bandwidth doesn't seem conducive to working
> together towards a resolution to this all.  If you're going to say, "it was
> passed, you guys just have to find a way to implement it.  now please stop
> bothering us" then I'm going to come up with an implementation plan that
> looks something like the following:
> 
> * all SSH keys and email addresses for arch testers will auto-expire after
>   60 days.  If an arch tester needs to have continued access, a gentoo dev
>   will have to re-submit the key and recreate the alias for that arch
>   tester every 60 days.
> 
> That meets the requirements of the GLEP down to the letter and also
> satisfies infra concerns around key management.  However, it's a crappy
> solution.
> 
> So, I'd much rather work together towards finding a better one.

Simple solution, that I've repeatedly pointed at.  Use the existing 
ldap setup.  It's not infra's responsibility to add their accounts nor 
disable them (that is left in the air as stated, although I'd expect 
it'll fall on devrels head).  Infra doesn't even do retirement beyond 
when _devrel_ asks them to.  If that process is slow, ask for help and 
someone will chip in and improve it (mainly to minimize bottleneck 
involved).

A simple script handling a pull from ldap sshPubKey attribute 
updating $USER/.ssh/authorized_keys on lark, you've got the cvs ro 
issue licked.  Doesn't require anything crazy/new, and could be 
implemented in no time- no infra overhead beyond an initial setup cost 
for cvs, which I would be willing to implement myself.

It's minor to do it within existing framework, which is why I've 
stated it's daft pointing at the minimal requirement as admin hell.
~harring


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 1307 bytes --]

On Sat, Nov 19, 2005 at 05:06:15PM +0000 or thereabouts, Kurt Lieber wrote:
> For instance, the way GLEP 41 suggests doing r/o cvs is not going to work.

So, in the interests of trying to find a solution to this particular
problem...

As I understand the GLEP, the main requirement here is to give the arch
testers faster access to the ebuilds in CVS.  Is this accurate?  

If so, is there any reason we have to use CVS?  Lance and I are both
concerned about the extra load that another 50-100 people (extrapolating
from the 20 folks that amd64 says they have currently) will place on
cvs.g.o and I'd rather break this out onto a separate server.

One proposal being discussed is setting up a dedicated rsync server for
this purpose that a) syncs from CVS more frequently and b) has no ban
limits imposed on it.  Arch testers would have some way of authenticating
to the box and being able to sync as frequently as they wanted to.  Current
goal is to have all data in this new repository within 30 minutes of it
hitting CVS.  (current average is about 1 hour)

If the requirement is for r/o CVS access to the same CVS server that the
pure-blooded developers use (sorry, couldn't resist) then it may require
upgrades to our existing server and/or purchasing a new server.  

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Dan Meltzer]

On 11/19/05, Kurt Lieber <klieber@gentoo.org> wrote:
> On Sat, Nov 19, 2005 at 05:06:15PM +0000 or thereabouts, Kurt Lieber wrote:
> > For instance, the way GLEP 41 suggests doing r/o cvs is not going to work.
>
> So, in the interests of trying to find a solution to this particular
> problem...
>
> As I understand the GLEP, the main requirement here is to give the arch
> testers faster access to the ebuilds in CVS.  Is this accurate?

This is the main idea behind it I believe.
>
> If so, is there any reason we have to use CVS?  Lance and I are both
> concerned about the extra load that another 50-100 people (extrapolating
> from the 20 folks that amd64 says they have currently) will place on
> cvs.g.o and I'd rather break this out onto a separate server.

Funy, I was just pondering that myself...  is authenticated rsync
really possible?  I was thinking about some sort of on-the-fly access
list updating based on a daemon running on AT's computers, that sent
current IP.. but that sounded kind of messy, I suppose you are better
at figuring this out than I am, though :)

The only downside to this that I can see would be the lack of
history... FEX an upgraded -rX ebuild breaks something, I could test
against previous -rX's in turn to find out exactly which broke it, and
other history like stuff.  This may or may not be necessary/helpful,
hard to say without it having happened :)

Thanks for coming back and thinking about the implementation,
reguardless of which way its done.
>
> One proposal being discussed is setting up a dedicated rsync server for
> this purpose that a) syncs from CVS more frequently and b) has no ban
> limits imposed on it.  Arch testers would have some way of authenticating
> to the box and being able to sync as frequently as they wanted to.  Current
> goal is to have all data in this new repository within 30 minutes of it
> hitting CVS.  (current average is about 1 hour)
>
> If the requirement is for r/o CVS access to the same CVS server that the
> pure-blooded developers use (sorry, couldn't resist) then it may require
> upgrades to our existing server and/or purchasing a new server.
>
> --kurt
>
>
>

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] implementation details for GLEP 41

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 677 bytes --]

On Sat, Nov 19, 2005 at 04:30:53PM -0600 or thereabouts, Brian Harring wrote:
> Infra doesn't even do retirement beyond when _devrel_ asks them to.  If
> that process is slow, ask for help and someone will chip in and improve
> it (mainly to minimize bottleneck involved).

OK, fine.  Devrel does not have an established track record of retiring
devs who are otherwise inactive.  Please fix this.  Please also get an
agreement from them that they're going to be willing to take on the
additional load of these arch testers.  Then please articulate the process
that will be followed to ensure they're actively tracked and retired
if/when they fall off the map.

Thanks.

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1199 bytes --]

On Sat, Nov 19, 2005 at 10:47:37PM +0000, Kurt Lieber wrote:
> On Sat, Nov 19, 2005 at 04:30:53PM -0600 or thereabouts, Brian Harring wrote:
> > Infra doesn't even do retirement beyond when _devrel_ asks them to.  If
> > that process is slow, ask for help and someone will chip in and improve
> > it (mainly to minimize bottleneck involved).
> 
> OK, fine.  Devrel does not have an established track record of retiring
> devs who are otherwise inactive.  Please fix this.  Please also get an
> agreement from them that they're going to be willing to take on the
> additional load of these arch testers.  Then please articulate the process
> that will be followed to ensure they're actively tracked and retired
> if/when they fall off the map.

Devrel doesn't have much issues in actually retiring a dev from where 
I'm sitting.

The problem is in detection- an infra issue that could be solved by 
either allowing normal devrel people to run the detection scripts 
themselves (rather then asking infra to do so), or in modifying the 
commit hooks so it pushes info into ldap (which we can access).

Again... setup cost (something I'm more then willing to implement).
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 843 bytes --]

On Sat, Nov 19, 2005 at 05:44:41PM -0500 or thereabouts, Dan Meltzer wrote:
> Funy, I was just pondering that myself...  is authenticated rsync
> really possible?

Yes, it has its own auth mechanism.  We actually use it for some automated
cron jobs that we have.

> The only downside to this that I can see would be the lack of
> history... FEX an upgraded -rX ebuild breaks something, I could test
> against previous -rX's in turn to find out exactly which broke it, and
> other history like stuff.  This may or may not be necessary/helpful,
> hard to say without it having happened :)

So, can other arch testers please pitch in with their $.02?  If we gave you
rsync instead of CVS, would that be sufficient?  Or do you need the
revision history, etc. of CVS?

And, any objections to a ~30 minute delay between CVS<->this solution?

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Dan Meltzer]

On 11/19/05, Kurt Lieber <klieber@gentoo.org> wrote:
> On Sat, Nov 19, 2005 at 05:44:41PM -0500 or thereabouts, Dan Meltzer wrote:
> > Funy, I was just pondering that myself...  is authenticated rsync
> > really possible?
>
> Yes, it has its own auth mechanism.  We actually use it for some automated
> cron jobs that we have.
>
> > The only downside to this that I can see would be the lack of
> > history... FEX an upgraded -rX ebuild breaks something, I could test
> > against previous -rX's in turn to find out exactly which broke it, and
> > other history like stuff.  This may or may not be necessary/helpful,
> > hard to say without it having happened :)
>
> So, can other arch testers please pitch in with their $.02?  If we gave you
> rsync instead of CVS, would that be sufficient?  Or do you need the
> revision history, etc. of CVS?
>
> And, any objections to a ~30 minute delay between CVS<->this solution?
30 minutes is much better than 24 hours :)
>
> --kurt
>
>
>

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] implementation details for GLEP 41

[Dan Meltzer]

Sorry for two mails in a row.. but out of curiosity, instead of using
30 minute rsync, why not 30 minute mirror of cvs? KDE does this fairly
well, they even have it something like every 5 minutes, is there any
reason mirrored cvs is not possible//feasible? is this something svn
has gotten better at?

On 11/19/05, Kurt Lieber <klieber@gentoo.org> wrote:
> On Sat, Nov 19, 2005 at 05:44:41PM -0500 or thereabouts, Dan Meltzer wrote:
> > Funy, I was just pondering that myself...  is authenticated rsync
> > really possible?
>
> Yes, it has its own auth mechanism.  We actually use it for some automated
> cron jobs that we have.
>
> > The only downside to this that I can see would be the lack of
> > history... FEX an upgraded -rX ebuild breaks something, I could test
> > against previous -rX's in turn to find out exactly which broke it, and
> > other history like stuff.  This may or may not be necessary/helpful,
> > hard to say without it having happened :)
>
> So, can other arch testers please pitch in with their $.02?  If we gave you
> rsync instead of CVS, would that be sufficient?  Or do you need the
> revision history, etc. of CVS?
>
> And, any objections to a ~30 minute delay between CVS<->this solution?
>
> --kurt
>
>
>

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] implementation details for GLEP 41

[Tres Melton]

[-- Attachment #1: Type: text/plain, Size: 802 bytes --]

On Sat, 2005-11-19 at 22:47 +0000, Kurt Lieber wrote:
> OK, fine.  Devrel does not have an established track record of retiring
> devs who are otherwise inactive.  Please fix this.  Please also get an
> agreement from them that they're going to be willing to take on the
> additional load of these arch testers.  Then please articulate the process
> that will be followed to ensure they're actively tracked and retired
> if/when they fall off the map.

I don't know about retiring devs but the AT's should fall under the
jurisdiction of the Head/lead/strategic lead/whatever the dude in charge
of the AT's for a platform calls themselves.  For amd64 that is hparker,
dang, blubb, and maybe another one or two developers.

> Thanks.
> 
> --kurt
-- 
Tres Melton
IRC & Gentoo: RiverRat

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 592 bytes --]

On Sat, Nov 19, 2005 at 04:52:08PM -0600 or thereabouts, Brian Harring wrote:
> Devrel doesn't have much issues in actually retiring a dev from where 
> I'm sitting.

Then I guess we'll disagree on this.

> The problem is in detection- an infra issue that could be solved by 
> either allowing normal devrel people to run the detection scripts 
> themselves (rather then asking infra to do so)

First I've heard of this request.  Has a bug been submitted for it?  It's
easy enough to set up some cron jobs to run scripts and email output to an
alias or mailing list.  

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]

Tres Melton wrote:
> On Sat, 2005-11-19 at 22:47 +0000, Kurt Lieber wrote:
> 
>>OK, fine.  Devrel does not have an established track record of retiring
>>devs who are otherwise inactive.  Please fix this.  Please also get an
>>agreement from them that they're going to be willing to take on the
>>additional load of these arch testers.  Then please articulate the process
>>that will be followed to ensure they're actively tracked and retired
>>if/when they fall off the map.
> 
> 
> I don't know about retiring devs but the AT's should fall under the
> jurisdiction of the Head/lead/strategic lead/whatever the dude in charge
> of the AT's for a platform calls themselves.  For amd64 that is hparker,
> dang, blubb, and maybe another one or two developers.

I see this as something that devrel would take care of since they
already do this for developers. They already have the tools/access to
the places for such things. Would rather not have another set of folks
with that access.

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 714 bytes --]

On Sat, Nov 19, 2005 at 05:59:46PM -0500 or thereabouts, Dan Meltzer wrote:
> Sorry for two mails in a row.. but out of curiosity, instead of using
> 30 minute rsync, why not 30 minute mirror of cvs? KDE does this fairly
> well, they even have it something like every 5 minutes, is there any
> reason mirrored cvs is not possible//feasible? is this something svn
> has gotten better at?

We have a well-established rsync infrastructure in place and rsync has its
own authentication that will support per-user auth (something that we
require)  I'm not opposed to using cvs, but unless there's a strong need
for it, I'd rather stick with rsync.  (which is why I'm asking here to see
what the need for it is)

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Simon Stelling]

Lance Albertson wrote:
> I see this as something that devrel would take care of since they
> already do this for developers. They already have the tools/access to
> the places for such things. Would rather not have another set of folks
> with that access.

So do I. Hint: Homer Parker is a devrel member ;)

-- 
Simon Stelling
Gentoo/AMD64 Operational Co-Lead
blubb@gentoo.org
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] implementation details for GLEP 41

[Lares Moreau]

[-- Attachment #1: Type: text/plain, Size: 1132 bytes --]

On Sat, 2005-11-19 at 22:56 +0000, Kurt Lieber wrote:
> So, can other arch testers please pitch in with their $.02?  If we gave you
> rsync instead of CVS, would that be sufficient?  Or do you need the
> revision history, etc. of CVS?
> 
> And, any objections to a ~30 minute delay between CVS<->this solution?
> 
> --kurt

I personally do not need Revision histories, I can't speak for other
ATs.  Rsync with 30min delay is a noted improvement over the standard
rsync policy.  Does this also allow us to sync to  main rotation mirroes
is that already overstressed? I ask because IIRC it may take ~30min for
all the mirrors to sync up to the 'Latest' revision, therefore the sync
that I do _may_ be up to 60min old (worstcase). so main rotation mirror
access would be nice.

Feasable? I know not.

Later Days
-- 
Lares Moreau <lares.moreau@gmail.com>  | LRU: 400755 http://counter.li.org
Gentoo x86 Arch Tester                 |               ::0 Alberta, Canada
Public Key: 0D46BB6E @ subkeys.pgp.net |           Encrypted Mail Prefered
Key fingerprint = 0CA3 E40D F897 7709 3628  C5D4 7D94 483E 0D46 BB6E

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 1826 bytes --]

Lares Moreau wrote:

> I personally do not need Revision histories, I can't speak for other
> ATs.  Rsync with 30min delay is a noted improvement over the standard
> rsync policy.  Does this also allow us to sync to  main rotation mirroes
> is that already overstressed? I ask because IIRC it may take ~30min for
> all the mirrors to sync up to the 'Latest' revision, therefore the sync
> that I do _may_ be up to 60min old (worstcase). so main rotation mirror
> access would be nice.

They aren't overstressed. The delay is just a function of the
implementation we have. Now, to make this work better, we could do a
direct sync again cvs with the --exclude-cvs option. The down side of
this is that it doesn't include any metadata or glsa files. Basically,
any of the regen process we run on the public tree wouldn't be done if
we did the direct rsync. If files are all you need, then this approach
would probably be the best. I can see problems using the current rsync
mirrors because of the lag time.

Reason being, there's going to be at least a 25-55 minute delay from
when a developer commits something and the time it hits a mirror on
rsync.g.o. The flow works as this:

1) CVS is updated/regen'd at :05/:35 on the master box
2) rsync.g.o syncs at :00/:30

So, if I commit something at say :07, users won't see that file until
the next :00 rsync from the mirrors. But, if I committed something at
:01, the user will see it in 29 minutes. If we have a dedicated box that
does a direct sync, we can reduce that time to 30min. Is that needed, or
is the 25-55 minute lag good enough?

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Retiring devs [was Re: [gentoo-dev] implementation details for GLEP 41]

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]

On Sat, Nov 19, 2005 at 11:04:44PM +0000, Kurt Lieber wrote:
> > The problem is in detection- an infra issue that could be solved by 
> > either allowing normal devrel people to run the detection scripts 
> > themselves (rather then asking infra to do so)
> 
> First I've heard of this request.  Has a bug been submitted for it?  It's
> easy enough to set up some cron jobs to run scripts and email output to an
> alias or mailing list.  

Only the usual irc infra requests (will take that comment as 
indication it's time to open a bug).

Would need the ability to maintain a blacklist of users to 
auto-ignore (releng), and would need to pull from svn also (something 
the current script doesn't handle afaik).

Binding pulling buzilla stats in would be needed also (poke kloeri 
about that one).

Ultimately, tracking actual pulls (ssh access on lark) rather then 
just pushes would be needed to- otherwise new doc devs, AT's, and new 
alt devs would be flagged due to their lack of the write bit.

Forums people, any thoughts/requirements?
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Lares Moreau]

[-- Attachment #1: Type: text/plain, Size: 692 bytes --]

On Sat, 2005-11-19 at 18:13 -0600, Lance Albertson wrote:
> is the 25-55 minute lag good enough?

It may need to be good enough.  Personally I would like to have < 5-7
min. That way when I'm working with a dev, I can keep up to speed with
her/him without having to resort to an overlay and emailing
ebuilds.tar's. A dedicated 'AT/HT' sync box sounds like an acceptable
solution.

-- 
Lares Moreau <lares.moreau@gmail.com>  | LRU: 400755 http://counter.li.org
Gentoo x86 Arch Tester                 |               ::0 Alberta, Canada
Public Key: 0D46BB6E @ subkeys.pgp.net |           Encrypted Mail Prefered
Key fingerprint = 0CA3 E40D F897 7709 3628  C5D4 7D94 483E 0D46 BB6E

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 1532 bytes --]

Lares Moreau wrote:
> On Sat, 2005-11-19 at 18:13 -0600, Lance Albertson wrote:
> 
>>is the 25-55 minute lag good enough?

> It may need to be good enough.  Personally I would like to have < 5-7
> min. That way when I'm working with a dev, I can keep up to speed with
> her/him without having to resort to an overlay and emailing
> ebuilds.tar's. A dedicated 'AT/HT' sync box sounds like an acceptable
> solution.

For now, I don't want to rsync more than every 30 minutes (concerns of
overloading the main cvs server). Pylon has mentioned that the newer
version of cvs has better commit hooks that may allow for more of a live
replication effect, but I don't expect that to happen any time soon. I
will try and come up with a revised version of GLEP 41 and see if
hparker and folks will agree with this new solution.

We will probably still have the blocking script on this server, but will
be at a much higher level. This is just to prevent folks from abusing
the service or giving out their access for other people to use. I really
don't see that happening, but I would prefer to have some kind of
prevention in place for infra's sake. I'll have to think out details on
the authentication scheme for access, but I would assume it would be per
AT and not a shared access account.

Thoughts?

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Lares Moreau]

[-- Attachment #1: Type: text/plain, Size: 2372 bytes --]

On Sat, 2005-11-19 at 19:02 -0600, Lance Albertson wrote:

> For now, I don't want to rsync more than every 30 minutes (concerns of
> overloading the main cvs server). Pylon has mentioned that the newer
> version of cvs has better commit hooks that may allow for more of a live
> replication effect, but I don't expect that to happen any time soon. I
> will try and come up with a revised version of GLEP 41 and see if
> hparker and folks will agree with this new solution.
> 
> We will probably still have the blocking script on this server, but will
> be at a much higher level. This is just to prevent folks from abusing
> the service or giving out their access for other people to use. I really
> don't see that happening, but I would prefer to have some kind of
> prevention in place for infra's sake. I'll have to think out details on
> the authentication scheme for access, but I would assume it would be per
> AT and not a shared access account.
> 
> Thoughts?

If any user really wanted to get the access that AT/HT's get, and the
AT/HT was so to give them it, there would be different IP addresses from
the same auth 'similaneously'. ie. logs state, IP A, IPB IPA, IPb. this
would indicate a security violation and revocation of privilege for the
AT/HT. Accomplished Via script?
Personally, If I wanted a user to have access to the same tree I had, I
would say A) chill for 12hrs, B) sync to my local mirror, C) post
ebuild.tar for them.  I don't believe there is an issue with AT/HT's
disseminating access to users. However I understand the need to be
prepared in case it happens. 

25-55min delay may need to be acceptable.

<brainstorming out loud>
Allow (x) access to the dedicated rsync server, not limited by time.
	- Allow Devs to change this number if they feel it is necessary
		- <5min access when working directly with Dev.
	- number reset every (y) days.
	(this means new infra, so prolly not)

Per AT Access:
	Each AT upload their ssh_pub to the existing infra - use that
for ?secure? rsync auth.
</>

-- 
Lares Moreau <lares.moreau@gmail.com>  | LRU: 400755 http://counter.li.org
Gentoo x86 Arch Tester                 |               ::0 Alberta, Canada
Public Key: 0D46BB6E @ subkeys.pgp.net |           Encrypted Mail Prefered
Key fingerprint = 0CA3 E40D F897 7709 3628  C5D4 7D94 483E 0D46 BB6E

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Grant Goodyear]

[-- Attachment #1: Type: text/plain, Size: 1030 bytes --]

Kurt Lieber wrote: [Sat Nov 19 2005, 04:42:41PM CST]
> On Sat, Nov 19, 2005 at 05:06:15PM +0000 or thereabouts, Kurt Lieber wrote:
> If the requirement is for r/o CVS access to the same CVS server that the
> pure-blooded developers use (sorry, couldn't resist) then it may require
> upgrades to our existing server and/or purchasing a new server.  

What about authenticated viewcvs on the live CVS server instead?  Back
when we had a live viewcvs I used to use it all the time for doing
exactly what the ATs want to do now, and I assume that viewcvs puts much
less load on the server than a CVS pull does.  

In any event, do we need a new server anyway?  We actually do have some
money that could be spent on such things, and the CVS server is really
high on the list of for which I, personally, would be more than willing
to spend it.

-g2boojum-
-- 
Grant Goodyear	
Gentoo Developer
g2boojum@gentoo.org
http://www.gentoo.org/~g2boojum
GPG Fingerprint: D706 9802 1663 DEF5 81B0  9573 A6DC 7152 E0F6 5B76

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Grant Goodyear]

[-- Attachment #1: Type: text/plain, Size: 498 bytes --]

Kurt Lieber wrote: [Sat Nov 19 2005, 04:47:37PM CST]
> OK, fine.  Devrel does not have an established track record of retiring
> devs who are otherwise inactive.  

Just as an aside, I've seen scores (if not more) of devs retired within
the last couple of months, so I think that problem is currently being
fixed.

-g2boojum-
-- 
Grant Goodyear	
Gentoo Developer
g2boojum@gentoo.org
http://www.gentoo.org/~g2boojum
GPG Fingerprint: D706 9802 1663 DEF5 81B0  9573 A6DC 7152 E0F6 5B76

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Ned Ludd]

On Sat, 2005-11-19 at 22:25 -0600, Grant Goodyear wrote:
> Kurt Lieber wrote: [Sat Nov 19 2005, 04:42:41PM CST]
> > On Sat, Nov 19, 2005 at 05:06:15PM +0000 or thereabouts, Kurt Lieber wrote:
> > If the requirement is for r/o CVS access to the same CVS server that the
> > pure-blooded developers use (sorry, couldn't resist) then it may require
> > upgrades to our existing server and/or purchasing a new server.  
> 
> What about authenticated viewcvs on the live CVS server instead?  Back
> when we had a live viewcvs I used to use it all the time for doing
> exactly what the ATs want to do now, and I assume that viewcvs puts much
> less load on the server than a CVS pull does.  
> 
> In any event, do we need a new server anyway?  We actually do have some
> money that could be spent on such things, and the CVS server is really
> high on the list of for which I, personally, would be more than willing
> to spend it.
> 
> -g2boojum-

If we were able to purchase hardware then we might as well make it the
anon cvs/svn server, no keys/auth are needed then and simple aliases
would suffice on toucan maintained by the AT leads.


-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] implementation details for GLEP 41

[Corey Shields]

On Saturday 19 November 2005 08:25 pm, Grant Goodyear wrote:
> In any event, do we need a new server anyway?  We actually do have some
> money that could be spent on such things, and the CVS server is really
> high on the list of for which I, personally, would be more than willing
> to spend it.
>
> -g2boojum-

This is a good possibility..  I personally don't use cvs that much but have 
heard from quite a few people that it could be faster.  And the box that it 
is currently on does not have any OOB management.

We've talked about OSL getting a couple of new boxes for Gentoo, one for a new 
dev.gentoo.org and one for cvs, but it's looking like just the one for dev 
will be a reality.  I will be ordering it next week.  That said, we can roll 
in an order for an additional box and get a good price on it.

The problem in all of this is the money transfer.  I'll check on Monday if 
there is a possible way to go from Gentoo Paypal to OSL Foundation.  We can 
whip out a proposal if that works out.  (there are other ways of doing it, 
yes, but this would be cheapest I know of right now)

-C

-- 
Corey Shields
Gentoo Linux Infrastructure Team
Gentoo Foundation Board of Trustees
http://www.gentoo.org/~cshields
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] implementation details for GLEP 41

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 2290 bytes --]

Ned Ludd wrote:
> On Sat, 2005-11-19 at 22:25 -0600, Grant Goodyear wrote:
> 
>>Kurt Lieber wrote: [Sat Nov 19 2005, 04:42:41PM CST]
>>
>>>On Sat, Nov 19, 2005 at 05:06:15PM +0000 or thereabouts, Kurt Lieber wrote:
>>>If the requirement is for r/o CVS access to the same CVS server that the
>>>pure-blooded developers use (sorry, couldn't resist) then it may require
>>>upgrades to our existing server and/or purchasing a new server.  
>>
>>What about authenticated viewcvs on the live CVS server instead?  Back
>>when we had a live viewcvs I used to use it all the time for doing
>>exactly what the ATs want to do now, and I assume that viewcvs puts much
>>less load on the server than a CVS pull does.  
>>
>>In any event, do we need a new server anyway?  We actually do have some
>>money that could be spent on such things, and the CVS server is really
>>high on the list of for which I, personally, would be more than willing
>>to spend it.
>>
>>-g2boojum-
> 
> 
> If we were able to purchase hardware then we might as well make it the
> anon cvs/svn server, no keys/auth are needed then and simple aliases
> would suffice on toucan maintained by the AT leads.

I currently have hardware for an anon cvs/svn server, just no time to
implement currently. I was hoping to get to this in the next few months
time permitting of course. I do not want anon cvs/svn on our live server
at all. Any implementation of such service will need to be on separate
dedicated boxes. I haven't worked out the details for 'live' data, but
the possibilities are there I believe for part of it.

We already have a viewcvs site that updates every 30 minutes (or maybe
an hour, I don't quite remember) that's in testing currently. Check out
viewcvstest.g.o if you're curious. To me the cvs/svn server should only
be doing that, nothing more. For now, i think the dedicated rsync server
is the best route that will accomplish the needs of the ATs. (which I
have a server for also). Until we get an official anon cvs/svn service
implemented, this is the best infra can do.

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 1592 bytes --]

Corey Shields wrote:
> On Saturday 19 November 2005 08:25 pm, Grant Goodyear wrote:
> 
>>In any event, do we need a new server anyway?  We actually do have some
>>money that could be spent on such things, and the CVS server is really
>>high on the list of for which I, personally, would be more than willing
>>to spend it.
>>
>>-g2boojum-
> 
> 
> This is a good possibility..  I personally don't use cvs that much but have 
> heard from quite a few people that it could be faster.  And the box that it 
> is currently on does not have any OOB management.
> 
> We've talked about OSL getting a couple of new boxes for Gentoo, one for a new 
> dev.gentoo.org and one for cvs, but it's looking like just the one for dev 
> will be a reality.  I will be ordering it next week.  That said, we can roll 
> in an order for an additional box and get a good price on it.
> 
> The problem in all of this is the money transfer.  I'll check on Monday if 
> there is a possible way to go from Gentoo Paypal to OSL Foundation.  We can 
> whip out a proposal if that works out.  (there are other ways of doing it, 
> yes, but this would be cheapest I know of right now)

Yeah, we defiantly could use a beefy new server for CVS/SVN. Just make
sure you chat with robbat2/Pylon on the specifics for the requirements.
I believe the main thing they wanted was lots of ram.

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Corey Shields]

On Saturday 19 November 2005 08:50 pm, Lance Albertson wrote:
> Yeah, we defiantly could use a beefy new server for CVS/SVN. Just make
> sure you chat with robbat2/Pylon on the specifics for the requirements.
> I believe the main thing they wanted was lots of ram.

As discussed before, the new dev will be a dual xeon 3.0/1M, 2GB ram, 6x146GB 
U320 scsi.   adding more ram to this setup wouldn't be a problem. I'll cc 
them and ask how much ram hits the sweet spot and get a new quote this week.

-C

-- 
Corey Shields
Gentoo Linux Infrastructure Team
Gentoo Foundation Board of Trustees
http://www.gentoo.org/~cshields
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] CVS-Server requirements (was: implementation details for GLEP 41)

[Lars Weiler]

[-- Attachment #1: Type: text/plain, Size: 873 bytes --]

* Lance Albertson <ramereth@gentoo.org> [05/11/19 22:50 -0600]:
> Yeah, we defiantly could use a beefy new server for CVS/SVN. Just make
> sure you chat with robbat2/Pylon on the specifics for the requirements.
> I believe the main thing they wanted was lots of ram.

CVS/SVN doesn't need much CPU load or even several CPUs and
also we don't need a lot of disk-space.  But our setup could
make use of a lot of fast RAM and a nice RAID (which we
don't have at the moment).

So specs are:
- ~3GHz Xeon
- 4-6GB of RAM
- RAID-5 or -10 with u320 disks (for the actual data, 20GB
  would be enough for the next years)
- a very good network-connection

Regards, Lars

-- 
Lars Weiler  <pylon@gentoo.org>  +49-171-1963258
Gentoo Linux PowerPC    : Developer and Release Engineer
Gentoo Infrastructure   : CVS Administrator
Gentoo Foundation       : Trustee

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] implementation details for GLEP 41

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1237 bytes --]

On Sat, Nov 19, 2005 at 09:04:13PM -0800, Corey Shields wrote:
> On Saturday 19 November 2005 08:50 pm, Lance Albertson wrote:
> > Yeah, we defiantly could use a beefy new server for CVS/SVN. Just make
> > sure you chat with robbat2/Pylon on the specifics for the requirements.
> > I believe the main thing they wanted was lots of ram.
> As discussed before, the new dev will be a dual xeon 3.0/1M, 2GB ram, 6x146GB 
> U320 scsi.   adding more ram to this setup wouldn't be a problem. I'll cc 
> them and ask how much ram hits the sweet spot and get a new quote this week.
4Gb of RAM would enable the current CVS speedups to continue, and also
allow for keeping all of the CVS/SVN trees (2Gb of data presently) in
the memory-cached files (important for speed).

The Xeon's need to be be HT-capable (the present ones are), as that
helps spread available CPU around the tasks that do use it better.

The 6x146GB is overkill for storage, unless you have some other plans
that I'm not aware of (I'm assuming RAID5 with a hot-spare, so 4x146GB
usable). 6x72GB might be more suitable for the budget.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 241 bytes --]

Re: Retiring devs [was Re: [gentoo-dev] implementation details for GLEP 41]

[Sune Kloppenborg Jeppesen]

[-- Attachment #1: Type: text/plain, Size: 445 bytes --]

On Sunday 20 November 2005 01:26, Brian Harring wrote:
> Would need the ability to maintain a blacklist of users to 
> auto-ignore (releng), and would need to pull from svn also (something 
> the current script doesn't handle afaik).
AFAIR solar? had a small script for this.

> Forums people, any thoughts/requirements?
Some of the security devs also don't do many commits.

-- 
Sune Kloppenborg Jeppesen
Gentoo Linux Security Team

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] Re: implementation details for GLEP 41

[Duncan]

Robin H. Johnson posted
<20051120054441.GA17389@curie-int.vc.shawcable.net>, excerpted below,  on
Sat, 19 Nov 2005 21:44:41 -0800:

> The 6x146GB is overkill for storage, unless you have some other plans
> that I'm not aware of (I'm assuming RAID5 with a hot-spare, so 4x146GB
> usable). 6x72GB might be more suitable for the budget.

As I just RAID-ed my main system, and have the info fresh... 

If the capacity is there, go RAID6 (dual parity RAID5, so two drives can
drop out without the thing dieing) with a hot-spare as well, so
threex146GB usable.

In any case, I'd go RAID6 with no hot-spare over RAID5 with a hot-spare,
as it's effectively the same thing, only with RAID6, you can lose two at
once without dieing, instead of only one -- and you hope the second waits
to die at least until the hot-spare gets synced.

This of course assumes software RAID, as RAID6 is certainly a kernel
option. If it's hardware RAID, you of course go with the capacities the
hardware supplies, and I'd guess RAID6 is a less common option, certainly
less commonly known.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list

Re: Retiring devs [was Re: [gentoo-dev] implementation details for GLEP 41]

[Wernfried Haas]

On Sat, Nov 19, 2005 at 06:26:28PM -0600, Brian Harring wrote:
> Forums people, any thoughts/requirements?

Currently there are approximately 10 mods/admins. In general it's
possible for us to keep track of who of us is active or not. 
Those folks also have toucan access and _should_ update their 
away when gone. 

If devrel thinks it is necessary it may be possible to set up 
automatic notifications if someone has not logged into the 
forums in a while (not sure if required and how much effort
to implement).

cheers,
	Wernfried

-- 
Wernfried Haas (amne) - amne at gentoo dot org
Gentoo Forums: http://forums.gentoo.org
IRC: #gentoo-forums on freenode - email: forum-mods at gentoo dot org
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] CVS-Server requirements (was: implementation details for GLEP 41)

[Lares Moreau]

[-- Attachment #1: Type: text/plain, Size: 1215 bytes --]

The Specs giving were for the new dev box, not the projected CVS/SVN
box.  I think the wire got crossed somewhere along the way.
-Lares
On Sun, 2005-11-20 at 06:31 +0100, Lars Weiler wrote:
> * Lance Albertson <ramereth@gentoo.org> [05/11/19 22:50 -0600]:
> > Yeah, we defiantly could use a beefy new server for CVS/SVN. Just make
> > sure you chat with robbat2/Pylon on the specifics for the requirements.
> > I believe the main thing they wanted was lots of ram.
> 
> CVS/SVN doesn't need much CPU load or even several CPUs and
> also we don't need a lot of disk-space.  But our setup could
> make use of a lot of fast RAM and a nice RAID (which we
> don't have at the moment).
> 
> So specs are:
> - ~3GHz Xeon
> - 4-6GB of RAM
> - RAID-5 or -10 with u320 disks (for the actual data, 20GB
>   would be enough for the next years)
> - a very good network-connection
> 
> Regards, Lars
> 
-- 
Lares Moreau <lares.moreau@gmail.com>  | LRU: 400755 http://counter.li.org
Gentoo x86 Arch Tester                 |               ::0 Alberta, Canada
Public Key: 0D46BB6E @ subkeys.pgp.net |           Encrypted Mail Prefered
Key fingerprint = 0CA3 E40D F897 7709 3628  C5D4 7D94 483E 0D46 BB6E

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Re: implementation details for GLEP 41

[Lares Moreau]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

On Sun, 2005-11-20 at 04:29 -0700, Duncan wrote:
> If the capacity is there, go RAID6 (dual parity RAID5, so two drives can
> drop out without the thing dieing) with a hot-spare as well, so
> threex146GB usable.

Is RAID6 production ready?
-- 
Lares Moreau <lares.moreau@gmail.com>  | LRU: 400755 http://counter.li.org
Gentoo x86 Arch Tester                 |               ::0 Alberta, Canada
Public Key: 0D46BB6E @ subkeys.pgp.net |           Encrypted Mail Prefered
Key fingerprint = 0CA3 E40D F897 7709 3628  C5D4 7D94 483E 0D46 BB6E

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Re: implementation details for GLEP 41

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

On Sun, 20 Nov 2005 07:49:11 -0700 Lares Moreau
<lares.moreau@gmail.com> wrote:
| On Sun, 2005-11-20 at 04:29 -0700, Duncan wrote:
| > If the capacity is there, go RAID6 (dual parity RAID5, so two
| > drives can drop out without the thing dieing) with a hot-spare as
| > well, so threex146GB usable.
| 
| Is RAID6 production ready?

RAID6 was only invented because a certain large hardware manufacturer
shipped a bunch of duff disks in one of its drive arrays. In practice
it's not necessary, because if you're taking the kind of damage that
kills multiple drives over a short period then you're going to lose
more than two drives anyway.

-- 
Ciaran McCreesh : Gentoo Developer (Look! Shiny things!)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Retiring devs [was Re: [gentoo-dev] implementation details for GLEP 41]

[Bryan Ãstergaard]

On Sat, Nov 19, 2005 at 06:26:28PM -0600, Brian Harring wrote:
> On Sat, Nov 19, 2005 at 11:04:44PM +0000, Kurt Lieber wrote:
> > > The problem is in detection- an infra issue that could be solved by 
> > > either allowing normal devrel people to run the detection scripts 
> > > themselves (rather then asking infra to do so)
> > 
> > First I've heard of this request.  Has a bug been submitted for it?  It's
> > easy enough to set up some cron jobs to run scripts and email output to an
> > alias or mailing list.  
> 
> Only the usual irc infra requests (will take that comment as 
> indication it's time to open a bug).
I'm currently retiring a somewhat large portion of inactive devs. I
don't use solars script as it doesn't work correctly in some cases. This
has been discussed several times on #-infra together with the need for
checking bugzilla activity. I've fixed the bugzilla activity using a
slightly hackish php script that pulls activity directly from the
database for now but it should probably be cleaned up a bit. My biggest
problem really is checking cvs/svn activity right now because of the
aforementioned bug that more or less forces me to use 'cvs history'.

I'll file a bug or two with my requirements later today so it can be
handled in a more orderly fashion.
> 
> Would need the ability to maintain a blacklist of users to 
> auto-ignore (releng), and would need to pull from svn also (something 
> the current script doesn't handle afaik).
> 
> Binding pulling buzilla stats in would be needed also (poke kloeri 
> about that one).
> 
> Ultimately, tracking actual pulls (ssh access on lark) rather then 
> just pushes would be needed to- otherwise new doc devs, AT's, and new 
> alt devs would be flagged due to their lack of the write bit.
I don't really care about anything besides commits right now. Even for
ATs having only ro cvs access I should still be able to determine their
activity from bugs.g.o activity. Failing that I do talk to leads when in
doubt and always try to cc people on any retirement bugs I file so they
can object if needed.
> 
> Forums people, any thoughts/requirements?
> ~harring
We should be able to handle forums staff the same way I currently check
bugs activity. Only requires ro access to the database and a small
script but this would obviously have to be discussed with infra and
forum leads.

Regards,
Bryan Østergaard
-- 
gentoo-dev@gentoo.org mailing list

Re: Retiring devs [was Re: [gentoo-dev] implementation details for GLEP 41]

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 1856 bytes --]

Bryan Ãstergaard wrote:
> On Sat, Nov 19, 2005 at 06:26:28PM -0600, Brian Harring wrote:
> 
>>On Sat, Nov 19, 2005 at 11:04:44PM +0000, Kurt Lieber wrote:
>>
>>>>The problem is in detection- an infra issue that could be solved by 
>>>>either allowing normal devrel people to run the detection scripts 
>>>>themselves (rather then asking infra to do so)
>>>
>>>First I've heard of this request.  Has a bug been submitted for it?  It's
>>>easy enough to set up some cron jobs to run scripts and email output to an
>>>alias or mailing list.  
>>
>>Only the usual irc infra requests (will take that comment as 
>>indication it's time to open a bug).
> 
> I'm currently retiring a somewhat large portion of inactive devs. I
> don't use solars script as it doesn't work correctly in some cases. This
> has been discussed several times on #-infra together with the need for
> checking bugzilla activity. I've fixed the bugzilla activity using a
> slightly hackish php script that pulls activity directly from the
> database for now but it should probably be cleaned up a bit. My biggest
> problem really is checking cvs/svn activity right now because of the
> aforementioned bug that more or less forces me to use 'cvs history'.
> 
> I'll file a bug or two with my requirements later today so it can be
> handled in a more orderly fashion.

I think we've fixed some of those issues with solar's script. You just
need to look at the list and make your assumptions. The script is great
to spitting out a list that you can look it. Its not 100%, but its good
enough to at least use.

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Retiring devs [was Re: [gentoo-dev] implementation details for GLEP 41]

[Bryan Ãstergaard]

On Sun, Nov 20, 2005 at 09:34:20AM -0600, Lance Albertson wrote:
> 
> I think we've fixed some of those issues with solar's script. You just
> need to look at the list and make your assumptions. The script is great
> to spitting out a list that you can look it. Its not 100%, but its good
> enough to at least use.
> 
Actually, solars script is completely broken after the move to ldap but
he should be working on it now :) Still leaves the other bits of the
puzzle though (bugs and forums activity).

Regards,
Bryan Østergaard
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: implementation details for GLEP 41

[Corey Shields]

On Sunday 20 November 2005 06:49 am, Lares Moreau wrote:
> On Sun, 2005-11-20 at 04:29 -0700, Duncan wrote:
> > If the capacity is there, go RAID6 (dual parity RAID5, so two drives can
> > drop out without the thing dieing) with a hot-spare as well, so
> > threex146GB usable.
>
> Is RAID6 production ready?

If you are running HP equipment they have been doing it for years, calling it 
RAID ADG (advanced data guarding iirc).  I've setup all of my servers as RAID 
ADG plus a hot spare to compensate for their disk failure rate.

-C

-- 
Corey Shields
Gentoo Linux Infrastructure Team
Gentoo Foundation Board of Trustees
http://www.gentoo.org/~cshields
-- 
gentoo-dev@gentoo.org mailing list

Re: Retiring devs [was Re: [gentoo-dev] implementation details for GLEP 41]

[Ned Ludd]

On Sun, 2005-11-20 at 16:43 +0100, Bryan Ãstergaard wrote:
> On Sun, Nov 20, 2005 at 09:34:20AM -0600, Lance Albertson wrote:
> > 
> > I think we've fixed some of those issues with solar's script. You just
> > need to look at the list and make your assumptions. The script is great
> > to spitting out a list that you can look it. Its not 100%, but its good
> > enough to at least use.
> > 
> Actually, solars script is completely broken after the move to ldap but
> he should be working on it now :) Still leaves the other bits of the
> puzzle though (bugs and forums activity).


How but the new one I just wrote 30 mins ago? Is that any better?
Speak up now if not cuz I've already stuck it in /etc/cron.monthly to
mail us.


-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: Retiring devs [was Re: [gentoo-dev] implementation details for GLEP 41]

[Wernfried Haas]

On Sun, Nov 20, 2005 at 04:10:34PM +0100, Bryan Ãstergaard wrote:
> We should be able to handle forums staff the same way I currently check
> bugs activity. Only requires ro access to the database and a small
> script but this would obviously have to be discussed with infra and
> forum leads.
As stated in my other mail (where i wrote 10 people, it's actually 18,
sorry) i guess that can be implemented with not too much effort. It's
probably easiest to discuss the details somewhere on irc
(#gentoo-forums, #gentoo-devrel or wherever else).

cheers,
	Wernfried

-- 
Wernfried Haas (amne) - amne at gentoo dot org
Gentoo Forums: http://forums.gentoo.org
IRC: #gentoo-forums on freenode - email: forum-mods at gentoo dot org
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Re: implementation details for GLEP 41

[Duncan]

Ciaran McCreesh posted <20051120145749.0027af0d@snowdrop.home>, excerpted
below,  on Sun, 20 Nov 2005 14:57:49 +0000:

> On Sun, 20 Nov 2005 07:49:11 -0700 Lares Moreau
> <lares.moreau@gmail.com> wrote:
> | On Sun, 2005-11-20 at 04:29 -0700, Duncan wrote:
> | > If the capacity is there, go RAID6 (dual parity RAID5, so two
> | > drives can drop out without the thing dieing) with a hot-spare as
> | > well, so threex146GB usable.
> | 
> | Is RAID6 production ready?
> 
> RAID6 was only invented because a certain large hardware manufacturer
> shipped a bunch of duff disks in one of its drive arrays. In practice
> it's not necessary, because if you're taking the kind of damage that
> kills multiple drives over a short period then you're going to lose
> more than two drives anyway.

There's another advantage as well.  Single disk failure is common enough
to be worrying about or raid5 wouldn't be in consideration.  I'm
certainly no expert, but from from my research previous to installing
here, it is said that raid6 in single failure mode maintains speed,
while a raid5 with hot-spare would be responding far slower during the
same time, as it brought the hot-spare online and did the rebuild.

Thus, if one is going to bother with the hot-spare in the first place,
rather than just run the raid5 in degraded mode until a spare can be
procured and installed, one might as well put that hot-spare to use making
the raid5 a raid6, both protecting against the corner-case of a
short-period compound failure, AND maintaining speed during a simple
failure.

Of course, if that speed maintenance is a a critical factor, then one
would hot-spare the raid6 as well, so non-degraded operation could be
resumed ASAP, thus again allowing a single failure without degrading speed.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list