[gentoo-dev] Making the developer community more open

[gentoo-dev] Making the developer community more open

[Daniel Drake]

"more open"? I can't think of a decent way to phrase the subject line 
which might make it sound it was coming from a native English 
speaker..ahem..anyway:

I read a complimentary comment from a Gentoo user recently (can't 
remember exactly where, so this is from memory). It was something along 
the lines of "Gentoo is great and will continue to be great for the 
foreseeable future. You have built the required structure to keep up 
with the rate of change in your environment (i.e. the increasingly rapid 
rate of development of open-source sofware)."
(if anyone can point me to where I read that, I'd appreciated it).

I think there's a lot of truth in that, especially the way that he/she 
highlights the fact that simply keeping up with what goes on around us 
is key to our "survival".

I won't go as far to say that I *don't* think we can keep up with our 
current "system", but I think there is plenty of room for improvement.

One of the bigger problems is that we have a huge user community who are 
keen on contributing, but we have such a high barrier for entry to the 
developer community. Quite rightly so - we're dealing with a live tree, 
so we can't give out commit access on the street.

At the same time, I feel that we're missing out. Comparing Gentoo with 
some other large open-source communities that I am personally involved 
in, I feel that we're too closed.

A developer recently compared Gentoo dev-ship to a marriage. In an ideal 
world, sure, we'd love for every single person who makes any kind of 
contribution to the project to become a full-time contributor who never 
goes AWOL or sleeps with another project. But more realistically, I 
think we need to become more open and flexible - as volunteers, people's 
interests change, some people will stop contributing after they have 
fixed whatever problem motivated them to contribute, etc. How can we 
handle this better?

We have a large expense on both sides when adding a developer to the 
project. I personally have lost developer candidates, undoubtedly more 
technically experienced than myself, who simply did not have the time to 
go through a month-long recruitment process which involved studying 
various documents not even relevant to the small area they would be 
contributing to. On the other side, it's a fair expense to add a 
developer to the project due to all of the 
quizzing/assessing/account-creating/access-elevation/...

Additionally, a significant percentage of developers who have joined 
recently have gone AWOL after a few months. That hurts us, given the 
expense we went through recruiting and adding them, and the time needed 
to reverse that and retire them.

I am not claiming this is an easy problem to solve - we do need to be 
especially careful that any changes made do not decrease the quality of 
commits to the live portage tree. This is why I am asking for help.

I'm looking for ideas - preferably big, drastic, shiny ones. Ignore any 
issues relating to migration away from our current system. What would be 
the _ideal_ way for Gentoo to handle contributions from anyone? (note 
that I'm dropping the user/developer community separation in that 
question, as the boundary between those could change in these ideas)
How would an ideal recruitment process work, if there would be one at all?

Please try and keep replies on-topic - I'm not trying to start a 
discussion/flamewar on the current recruitment system or anything like that.

To get you thinking, I suggest reading the section titled "Open 
Development Team" at 
http://www.samspublishing.com/articles/article.asp?p=23200&seqNum=3
which is part of a (very good) larger article detailing why Linux kernel 
development works so well.

Any ideas?

Daniel
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 997 bytes --]

On Mon, 20 Mar 2006 23:07:37 +0000 Daniel Drake <dsd@gentoo.org> wrote:
| One of the bigger problems is that we have a huge user community who
| are keen on contributing, but we have such a high barrier for entry
| to the developer community. Quite rightly so - we're dealing with a
| live tree, so we can't give out commit access on the street.

A relatively easy way of lowering that barrier would be to provide
good, up to date, example-oriented ebuild writing documentation.
There's too much stuff that people need to know to write ebuilds that's
not written down anywhere -- this not only makes it harder for users to
write good ebuilds, but also leads to some of them being dissuaded when
they're told that the only way to know what's policy is by having paid
attention on the mailing lists for the past five years.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Making the developer community more open

[George Prowse]

[-- Attachment #1: Type: text/plain, Size: 1564 bytes --]

On 20/03/06, Ciaran McCreesh <ciaranm@gentoo.org> wrote:
>
> On Mon, 20 Mar 2006 23:07:37 +0000 Daniel Drake <dsd@gentoo.org> wrote:
> | One of the bigger problems is that we have a huge user community who
> | are keen on contributing, but we have such a high barrier for entry
> | to the developer community. Quite rightly so - we're dealing with a
> | live tree, so we can't give out commit access on the street.
>
> A relatively easy way of lowering that barrier would be to provide
> good, up to date, example-oriented ebuild writing documentation.
> There's too much stuff that people need to know to write ebuilds that's
> not written down anywhere -- this not only makes it harder for users to
> write good ebuilds, but also leads to some of them being dissuaded when
> they're told that the only way to know what's policy is by having paid
> attention on the mailing lists for the past five years.
>
> Bridge the gap between the developers and the users and you have a user
community that has a better knowlege of the distro and it's working and you
have a developeer community more in touch with it's users. Most people would
love to help but knowing where to start is the greatest challenge, make it
easy for them to know how to be a developer and then they can decide if they
have something to contribute. As Ciaran said, up-to-date ebuild
documentation is a must but it needs a bit of "the idiots guide to ebuilds"
style to it. Also a big assumption is made that people know how to use CVS
and other systems.

"Book them and they will come..."

George

[-- Attachment #2: Type: text/html, Size: 1922 bytes --]

Re: [gentoo-dev] Making the developer community more open

[Bret Towe]

On 3/20/06, Daniel Drake <dsd@gentoo.org> wrote:
> "more open"? I can't think of a decent way to phrase the subject line
> which might make it sound it was coming from a native English
> speaker..ahem..anyway:
>
> I read a complimentary comment from a Gentoo user recently (can't
> remember exactly where, so this is from memory). It was something along
> the lines of "Gentoo is great and will continue to be great for the
> foreseeable future. You have built the required structure to keep up
> with the rate of change in your environment (i.e. the increasingly rapid
> rate of development of open-source sofware)."
> (if anyone can point me to where I read that, I'd appreciated it).
>
> I think there's a lot of truth in that, especially the way that he/she
> highlights the fact that simply keeping up with what goes on around us
> is key to our "survival".
>
> I won't go as far to say that I *don't* think we can keep up with our
> current "system", but I think there is plenty of room for improvement.
>
> One of the bigger problems is that we have a huge user community who are
> keen on contributing, but we have such a high barrier for entry to the
> developer community. Quite rightly so - we're dealing with a live tree,
> so we can't give out commit access on the street.
>
> At the same time, I feel that we're missing out. Comparing Gentoo with
> some other large open-source communities that I am personally involved
> in, I feel that we're too closed.
>
> A developer recently compared Gentoo dev-ship to a marriage. In an ideal
> world, sure, we'd love for every single person who makes any kind of
> contribution to the project to become a full-time contributor who never
> goes AWOL or sleeps with another project. But more realistically, I
> think we need to become more open and flexible - as volunteers, people's
> interests change, some people will stop contributing after they have
> fixed whatever problem motivated them to contribute, etc. How can we
> handle this better?
>
> We have a large expense on both sides when adding a developer to the
> project. I personally have lost developer candidates, undoubtedly more
> technically experienced than myself, who simply did not have the time to
> go through a month-long recruitment process which involved studying
> various documents not even relevant to the small area they would be
> contributing to. On the other side, it's a fair expense to add a
> developer to the project due to all of the
> quizzing/assessing/account-creating/access-elevation/...
>
> Additionally, a significant percentage of developers who have joined
> recently have gone AWOL after a few months. That hurts us, given the
> expense we went through recruiting and adding them, and the time needed
> to reverse that and retire them.
>
> I am not claiming this is an easy problem to solve - we do need to be
> especially careful that any changes made do not decrease the quality of
> commits to the live portage tree. This is why I am asking for help.
>
> I'm looking for ideas - preferably big, drastic, shiny ones. Ignore any
> issues relating to migration away from our current system. What would be
> the _ideal_ way for Gentoo to handle contributions from anyone? (note
> that I'm dropping the user/developer community separation in that
> question, as the boundary between those could change in these ideas)
> How would an ideal recruitment process work, if there would be one at all?
>
> Please try and keep replies on-topic - I'm not trying to start a
> discussion/flamewar on the current recruitment system or anything like that.
>
> To get you thinking, I suggest reading the section titled "Open
> Development Team" at
> http://www.samspublishing.com/articles/article.asp?p=23200&seqNum=3
> which is part of a (very good) larger article detailing why Linux kernel
> development works so well.
>
> Any ideas?

perhaps having some proxys of a sort that accept patchs and such
from trusted users that would commit fixes to portage would help.
similiar to the kernel format that way users can 'commit'/help out quickly
without having to go thru the long process of becoming a dev


> Daniel
> --
> gentoo-dev@gentoo.org mailing list
>
>

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[Stefan Schweizer]

On 3/21/06, Bret Towe <magnade@gmail.com> wrote:
> perhaps having some proxys of a sort that accept patchs and such
> from trusted users that would commit fixes to portage would help.
> similiar to the kernel format that way users can 'commit'/help out quickly
> without having to go thru the long process of becoming a dev

That is imo a key feature: Get contributions back to the users easily

It doe snot need to be the portage-tree .. but an official
user-overlay or contrib-overlay would definitely help to get a lot of
people involved.

Other projects are providing similar infrastructure with very good
results, see the Arch User Repository for example:
http://aur.archlinux.org

It would be great to have something like that available for gentoo-users.

Regards,
Stefan

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[m h]

I'm not a gentoo dev (just a satisfied user), but I lurk on this list.

I was at PyCon last month.  I would estimate that about 40% of the
people there ran linux on their laptops.  The most popular distros
were gentoo and ubuntu.  (Not this is not a scientific study, just my
observations from talking to people there).  While I was there the
person next to me starting hacking the ebuild classes to handles eggs
(so he could emerge turbogears).  I talked to at least 3 others who
were running gentoo.   I asked all of them if they had worked on
portage.  Most said "No, the code is a little scary".  (I'll concur
with that sentiment, as the code doesn't feel very pythonic).

If you want to attract more developers (python people), a few things are needed:

 * Portage documentation.  How the innards work.  There is very little
docs/comments in the portage code
 * Unittests - without this how do I know that my change to portage
didn't break someone else's corner case
 * Refactoring into a more pythonic style.  Note that this is pretty
hard without unittests.

Take this as a grain of salt, from an observer, who believes that
there are a lot of potential users (who know python), and who could
easily contribute, if the bar was lowered a bit.  (Or steps were
provided to reach a little higher ;))

-matt

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 648 bytes --]

On Tue, 21 Mar 2006 00:58:07 +0100 "Stefan Schweizer"
<genstef@gentoo.org> wrote:
| It doe snot need to be the portage-tree .. but an official
| user-overlay or contrib-overlay would definitely help to get a lot of
| people involved.

The problem is security. It's extremely easy to sneak some very devious
code (e.g. that'd grant remote root access and post an IP somewhere)
into an ebuild that'd be very hard to spot by anyone who doesn't
reallllly know what they're doing.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Making the developer community more open

[m h]

George-
Not sure if you have seen this or not.  Check out Conary [1] from
rPath.  Think of it as Rpm+Ebuild+Distributed.  It's done by some
people who used to be at Redhat and in one of the whitepapers, they
specifically mention portage/ebuild.

-matt

1 - http://wiki.conary.com/FrontPage

On 3/20/06, George Shapovalov <george@gentoo.org> wrote:
> A quick update.
>
> Please use this link for the proposal instead of the one listed in original
> post in the bug:
> http://dev.gentoo.org/~george/epsp/proposal.html
>
> The files have been migrated to my gentoo space, as proper. I just added
> comment to the bug and I'll put up some remonder at the place the original
> link points to (but I am no longer at that place, so I cannot predict for how
> long that will work..)
>
> > Please take a look at #1523 (note the number ;)), it addresses essentially
> George
> --
> gentoo-dev@gentoo.org mailing list
>
>

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[Mike Auty]

Well,
	I think a lot of what I've been thinking recently has already been said 
by Daniel.  I'm actually in the middle of being inducted and I'm just 
concerned that I'm going to get extra responsibility without any real 
positive aspects for me.  I don't really *want* access to check into 
portage, I don't know what I'm doing (yet) and I'm not certain I can 
invest the time to learn all the precise policy to ensure I don't make a 
mistake, or worse put up with the stress of worrying I'll do it wrong 
and affect the entire gentoo vmware-using userbase.
	What I tend to do is make ebuilds for things that I want to try out or 
things that are broken, and I'd really like to just keep on doing that, 
but have it more accessible to the rest of the userbase.
	I think the idea of a proxy is a good one.  I don't know about a whole 
user repository though, because as Ciaran pointed out, one bad apple and 
everybody gets rooted.  No one would trust it, so it wouldn't be worth 
it anyway.

* It'd be a good idea to have a larger group of devs not dedicated to a 
particular project, but who can take user submitted ebuilds, vet them 
for nastiness, and submit them.  They'll be officially contributed 
ebuilds, they won't get updated until either a dev offers to take them 
on, or the community offers to fix them.

* I think also a larger number of bugzilla scouring devs could push 
through well tested (lots of positive feedback, no negative feedback) 
patches that the maintainers for whatever reason (aren't there, forgot 
about it, don't have the time) just aren't putting through.  That would 
require bending the maintainer etiquette a bit though.

* I guess a *quick* concise policy guide would help.  Separate from the 
ebuild guide which is trying to teach you *how* to write ebuilds, this 
policy guide would tell you what MUST and MUST NOT happen in a good 
Quality Assured ebuild.  For instance, if the various and sundry checks 
in repoman could be extracted for people to run over their own overlays 
without all the main portage cvs machinery in there, it would help them 
get *much* better trained in what makes a good ebuild and what doesn't. 
    If it can already do that, then it needs better documentation.

* Finally the mentoring scheme could perhaps be more streamlined.  So 
rather than having an all-or-nothing gentoo developer membership.  Those 
developers being mentored have a repoman-like interface that works 
almost exactly the same way, but instead of putting directly into0 the 
tree, it would submit to a holding area where an experienced ebuild 
writer can either send it back to them with comments, or put a tick next 
to it and pass it into the main overlay.  This then would allow people 
to work up to becoming a full dev, and take their time over it.  It 
would also offer a kind of buffer area for people who just want to help 
for a few months, their privilege levels don't have to rise too high.

	So these are some ideas.  Sorry, I was trying to get these out 
concisely but tripped on my tongue somewhere along the way, hope they 
help generate some ideas...
	Mike  5:)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[George Shapovalov]

Monday, 20. March 2006 23:07, Daniel Drake Ви написали:
> I'm looking for ideas - preferably big, drastic, shiny ones. Ignore any
> issues relating to migration away from our current system. What would be
> the _ideal_ way for Gentoo to handle contributions from anyone? 
> Any ideas?
Heh, and that bug almost got closed for the 2nd time recently :).

Please take a look at #1523 (note the number ;)), it addresses essentially 
this issue, or pretty similar. Half of the ideas from there we got done 
already, but another half is nowhere near. I'd even say they need some 
drastic redesign first, before they should go anywhere near glep or "nearing 
implementation". For one infra will not be happy at all about more major 
stuff, but you said "don't bother, just give me ideas", so here you go ;). 
Then some subprojects that are necessary to get that done in full were talked 
about and restarted a few times, but eventually died every time. I am talking 
about gentoo-stats and similar..

Frankly, I haven't thought much about this for the last two years or so, being 
busy with "issues of the day" mostly (and undergoing big real life 
transitions :)), but nonetheless kept the bug open, as from my experience 
this issue resurfaces every year or so. So, please take a look. Hopefully you 
will find something usefull..

George

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[George Shapovalov]

A quick update.

Please use this link for the proposal instead of the one listed in original 
post in the bug:
http://dev.gentoo.org/~george/epsp/proposal.html

The files have been migrated to my gentoo space, as proper. I just added 
comment to the bug and I'll put up some remonder at the place the original 
link points to (but I am no longer at that place, so I cannot predict for how 
long that will work..)

> Please take a look at #1523 (note the number ;)), it addresses essentially
George
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 2360 bytes --]

m h wrote:
> I'm not a gentoo dev (just a satisfied user), but I lurk on this list.
> 
> I was at PyCon last month.  I would estimate that about 40% of the
> people there ran linux on their laptops.  The most popular distros
> were gentoo and ubuntu.  (Not this is not a scientific study, just my
> observations from talking to people there).  While I was there the
> person next to me starting hacking the ebuild classes to handles eggs
> (so he could emerge turbogears).  I talked to at least 3 others who
> were running gentoo.   I asked all of them if they had worked on
> portage.  Most said "No, the code is a little scary".  (I'll concur
> with that sentiment, as the code doesn't feel very pythonic).
> 
> If you want to attract more developers (python people), a few things are needed:

That depends on how they contribute, I personally don't want random
python master bob contributing pieces to portage itself.  Portage things
are not necessarily as simple as people make them out to be.  Even
developers who know the code well make mistakes in adding and removing
code.  As solar once pointed out "the only man I trust to touch the
resolver is Jstubbs."  I realize thats a bit elitest...but at the same
time...I am overly cautious ;)

However we always accept patches and I think we get most of them
critiqued, sometimes it may take an extra prodding mail or two.  We
usually don't implement your features for you though ;)

> 
>  * Portage documentation.  How the innards work.  There is very little
> docs/comments in the portage code

Someone has to write them; I have some of it done, it's been a longtime
project that I've worked on off and on; I actually had more done last
year and I know kutsuya did some as well.  However these are not
particularly interesting..and no one wants to document the 2.X branch.

>  * Unittests - without this how do I know that my change to portage
> didn't break someone else's corner case
No one is writing unittests for the 2.X branch
>  * Refactoring into a more pythonic style.  Note that this is pretty
> hard without unittests.
See above :)

> 
> Take this as a grain of salt, from an observer, who believes that
> there are a lot of potential users (who know python), and who could
> easily contribute, if the bar was lowered a bit.  (Or steps were
> provided to reach a little higher ;))
> 
> -matt
> 

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 894 bytes --]

Re: [gentoo-dev] Making the developer community more open

[Jason Stubbs]

On Tuesday 21 March 2006 12:32, Alec Warner wrote:
> m h wrote:
> > I'm not a gentoo dev (just a satisfied user), but I lurk on this list.
> > 
> > I was at PyCon last month.  I would estimate that about 40% of the
> > people there ran linux on their laptops.  The most popular distros
> > were gentoo and ubuntu.  (Not this is not a scientific study, just my
> > observations from talking to people there).  While I was there the
> > person next to me starting hacking the ebuild classes to handles eggs
> > (so he could emerge turbogears).  I talked to at least 3 others who
> > were running gentoo.   I asked all of them if they had worked on
> > portage.  Most said "No, the code is a little scary".  (I'll concur
> > with that sentiment, as the code doesn't feel very pythonic).
> > 
> > If you want to attract more developers (python people), a few things are needed:
> 
> That depends on how they contribute, I personally don't want random
> python master bob contributing pieces to portage itself.  Portage things
> are not necessarily as simple as people make them out to be.  Even
> developers who know the code well make mistakes in adding and removing
> code.  As solar once pointed out "the only man I trust to touch the
> resolver is Jstubbs."  I realize thats a bit elitest...but at the same
> time...I am overly cautious ;)

The resolver as it stands now is not overly difficult. One does really need
to know it back to front though. I should really make splitting it out and
documenting it my big project for 2.2.

--
Jason Stubbs
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[Alin Nastac]

[-- Attachment #1: Type: text/plain, Size: 1430 bytes --]

Daniel Drake wrote:

> We have a large expense on both sides when adding a developer to the
> project. I personally have lost developer candidates, undoubtedly more
> technically experienced than myself, who simply did not have the time
> to go through a month-long recruitment process which involved studying
> various documents not even relevant to the small area they would be
> contributing to. On the other side, it's a fair expense to add a
> developer to the project due to all of the
> quizzing/assessing/account-creating/access-elevation/...

Technical ability isn't the only requirement for gentoo devs. They also
must be motivated individuals and these high barriers you are talking
about test this quality of the candidates.
If they quit just because recruitment process is long, what makes you
think they will stay active long enough to actually worth adding them to
dev corpus?

>
> Additionally, a significant percentage of developers who have joined
> recently have gone AWOL after a few months. That hurts us, given the
> expense we went through recruiting and adding them, and the time
> needed to reverse that and retire them.

Yes, it is hard to find the right people. Yes, a big percentage of
recruiting team's time will be lost on useless additions/removals. But
the only solution is scaling the recruiting team to gentoo needs.
IMO recruiting team is too small to cope with the current size of the
project.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 258 bytes --]

Re: [gentoo-dev] Making the developer community more open

[Simon Stelling]

Bret Towe wrote:
> perhaps having some proxys of a sort that accept patchs and such
> from trusted users that would commit fixes to portage would help.
> similiar to the kernel format that way users can 'commit'/help out quickly
> without having to go thru the long process of becoming a dev

Users can (and do) attach patches to any bug in bugzilla. When applying 
such patches, the committing dev hopefully checks the patch and makes 
sure it's clean, so he already is the kind of proxy you are asking for.

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 Developer
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[Thomas Cort]

> One of the bigger problems is that we have a huge user
> community who are keen on contributing, but we have such
> a high barrier for entry to the developer community.

There are the arch tester[1] projects (x86, amd64, ppc, alpha (soon),
and maybe others). Those lower the barrier a lot while still requiring
some level of knowledge (passing the ebuild quiz). A lot of ATs
eventually become devs. Maybe the various AT projects could be
advertised more, like the x86 AT team was this week in GWN[2].

[1] http://www.gentoo.org/proj/en/base/x86/arch-testers-faq.xml#whoat
[2] http://www.gentoo.org/news/en/gwn/20060320-newsletter.xml

-Thomas

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 835 bytes --]

On Tuesday 21 March 2006 07:05, Alin Nastac wrote:
> Yes, it is hard to find the right people. Yes, a big percentage of
> recruiting team's time will be lost on useless additions/removals. But
> the only solution is scaling the recruiting team to gentoo needs.
> IMO recruiting team is too small to cope with the current size of the
> project.

Probably the biggest reason for the closedness of the project is probably 
that there are no "official" ways to become a dev that originate from the 
candidate. Most if not all new developers are asked whether they would 
like to become a developer. This method has the advantage of weeding out 
goldseekers and other people who should not be given access to the tree.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Making the developer community more open

[Brandon Edens]

[-- Attachment #1: Type: text/plain, Size: 4468 bytes --]

On Mon, Mar 20, 2006 at 11:07:37PM +0000, Daniel Drake wrote:
> I'm looking for ideas - preferably big, drastic, shiny ones. Ignore any 
> issues relating to migration away from our current system. What would be 
> the _ideal_ way for Gentoo to handle contributions from anyone? (note 
> that I'm dropping the user/developer community separation in that 
> question, as the boundary between those could change in these ideas)
> How would an ideal recruitment process work, if there would be one at all?


When I was a system administrator working with Gentoo I would've appreciated a
way to interact with the other Gentoo system administrators. (ie gentoo-server).
I would've liked it even more if I could've communicated with the Gentoo
University Department System Administrators. When I say communicated I mean
interacted with.

Now that I'm an AMD64 laptop Gentoo user I would like a concise way of
communicating back to my community the AMD64 users and specifically the laptop
users. In fact I'd like to know what other people are using the Compaq Presario
V2000Z AMD Turion64. I'd also like to know what software they're running on
their laptops and if they consider it stable. What kernel configurations they're
using. What functionality is broken and what needs to be fixed.

I'd like an easy way to communicate with them, pass them notes about problems
with packages. If I trust them a-lot then I'd like to use their binary built
packages as well as allow them to use the ones built by me.  I guess we'd create
a sort of p2p mini-pocket of gentoo users with our relationship built upon
trust.

I imagined Gentoo on Win32, something like Cygwin. I maintained a computer lab
filled with Windows machines. I'd like to install Gentoo Win32 on one machine.
Install it on the next machine then tell that machine about the first. I'd like
to install it on a third machine then tell it about the first or the second but
have that third machine then know about both. I'd like to compile a piece of
software on one of the machines then know that any of the other two will
automatically get the binary version of the package from the one that compiled
it because they trust that machine. The damage to unnamed others from the
existence of a system like this would be quite excessive IMO.

So I've structured this email as a want-list. However, I'm not oblivious to how
this is implemented. I suppose the idea is to restructure Gentoo into a tiered
community (as mentioned by other posters). Make it easy for tiers to birth and
die (we might like a rhode island gentoo users group for instance, might not
last for more than a year.) Maybe one tier is just me and my friends sharing our
hacks, customizations, letting each other know about some exciting package,
etc... I need my portage/emerge to act as a meta system that pulls from the
various communities based upon how much I trust those communities.

How do we get there from here? I suppose just start adding functionality to
portage to support this. One part could be just expanding upon the portage
overlay. It might be nice if portage became better defined so that we could
implement it in a variety of programming languages (I'm into LISP programming
for fun).
Portage as a daemon?

Another concrete feature would be to allow by convention a specific directory
that could be created and used for applying patches during the build process
(without modifying ebuilds). A place where portage will automatically apply that
patch during the build of some piece of software. So lets say this feature of
portage existed, perhaps I could further put the patch in various community
portage overlays and others in that community could learn about that patch. I
suppose its those sorts of massive optimizations/conventions/intelligence that I
always appreciated about Gentoo and its packaging system.

I appreciate that Gentoo is more hacker friendly and less Debian ivory tower.  I
think some of that is source-built distro versus a binary distro.

Some of this is the gentoo-stats project that died. It'd be nice to know about
the other people in my version of the Gentoo community. If I'd had statistics
that MIT was using Gentoo on 5000 machines with each having an average up-time
of 100 days (information gleamed from portage's community functions); then I
could've marketed Gentoo better to my bosses.

I'm rambling.
Enjoy,
Brandon Edens


[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Making the developer community more open

[Daniel Drake]

Hi Brandon,

Brandon Edens wrote:
> When I was a system administrator working with Gentoo I would've
> appreciated a way to interact with the other Gentoo system
> administrators. <snip>

You seem to be purely describing interactions with the user community 
from a user perspective.

My post was about the gap between the user community and the developer 
community, and how the developer community can open up.

Thanks for the feedback - it's always interesting to read this kind of 
thing, but it's not what I'm looking for at this point.

Daniel
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[Daniel Goller]

[-- Attachment #1: Type: text/plain, Size: 2113 bytes --]

> Now that I'm an AMD64 laptop Gentoo user I would like a concise way of
> communicating back to my community the AMD64 users and specifically the laptop
> users. In fact I'd like to know what other people are using the Compaq Presario
> V2000Z AMD Turion64. I'd also like to know what software they're running on
> their laptops and if they consider it stable. What kernel configurations they're
> using. What functionality is broken and what needs to be fixed.

I enjoy using a V2000Z ( V2311US ) as my development machine, it runs
kde 3.5 well (even so, i am back on my beloved icewm), kmail with it's
customizable panes had been most enjoyable for email on a 14" widescreen
thus far, but the new masked evolution with USE=widescreen seems to be
replacing it rather quickly.
I prefer modular Xorg (7.0) with EXA acceleration in 2D by far over the
binary drivers with either 6.8.2 or xorg 7. (3D will have to wait until
someone gets dri working on shared memory laptops)
I patch my kernel so i can "undervolt" it, i had been compiling at 90C
at 1600mhz and a -e world (Gotta test gcc-4.1 some day) for days seemed
out of question, now i compile at ~60C at 1600mhz, which i guess makes
me a "preservist". (i mean preserving hardware, not battery capacity, as
the savings at 800MHz are less impressive thus battery life while
working in a text editor/web/irc/email is not affected by it much) (if
you love to compile while being unplugged the difference should be quite
nice :)
For wireless i still prefer ndiswrapper over bcm43xx.
I did keep windows around although the one thing i do most often from
there is flash my bios, so it does not see too much use.
I still can't use the memory card reader.
But there is so much that works reliably on here i do not miss that thus
far.
I do not have suspend to disk set up right now, swsusp worked fine,
suspend2 i always had a harder time getting working, what i typically
use is suspend to ram which now works flawless.

If you have any particular questions, let me know, this is what came to
mind on a more general basis.

Daniel

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Making the developer community more open

[Daniel Goller]

[-- Attachment #1: Type: text/plain, Size: 1385 bytes --]

On Tue, 2006-03-21 at 13:09 +0100, Simon Stelling wrote:
> Bret Towe wrote:
> > perhaps having some proxys of a sort that accept patchs and such
> > from trusted users that would commit fixes to portage would help.
> > similiar to the kernel format that way users can 'commit'/help out quickly
> > without having to go thru the long process of becoming a dev
> 
> Users can (and do) attach patches to any bug in bugzilla. When applying 
> such patches, the committing dev hopefully checks the patch and makes 
> sure it's clean, so he already is the kind of proxy you are asking for.
> 
> -- 

maybe he more means having a "working relationship" with people rather
than sending them to attach patches to bugzilla. make it more personal
than clinical
circumventing the requirement for an AT position, a casual "i give you
patches as i come across problems on my box, can you take care of them"
relationship for people who can and will contribute occasionally, w/o
the time to take on a dev position w/o commit access (aka Arch Tester)
like the people you hang out with on irc even, the ones that help other
users, or the kind you see active on forums, take their occational
patch/ebuild
like less red tape, more acceptance of the occasional contribution

how about that as "proxy"?

Daniel
 
> Kind Regards,
> 
> Simon Stelling
> Gentoo/AMD64 Developer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Making the developer community more open

[Jonathan Coome]

On Mon, 2006-03-20 at 15:45 -0800, Bret Towe wrote:
> perhaps having some proxys of a sort that accept patchs and such
> from trusted users that would commit fixes to portage would help.
> similiar to the kernel format that way users can 'commit'/help out quickly
> without having to go thru the long process of becoming a dev

Taking this idea a bit further, what about proxy maintainers? There seem
to be quite a few packages that are being effectively maintained by
users on bugzilla, but are not in portage because they don't have a
maintainer. A developer could then take these ebuilds, make sure they
don't do anything malicious, or break QA, or whatever, and act as the
bridge between the portage tree and the users actually working on the
ebuild and keeping things up to date and working.

There could then be a bug for each such package where all the patches,
ebuilds and version bumps could be posted. The developer who accepts the
package could just take those ebuilds, maybe corrected if necessary, and
commit them. If the ebuild breaks, it's up to the developer to fix it.
If the package breaks, however, it would be up to the users on the bug
to fix it, although of course the developer would be able to fix it if
he or she could.

If there doesn't seem to be anyone interested in keeping the package
working anymore then it could be masked and subsequently removed as
packages are now. If there are security problems and the fix is not
trivial, it might be sensible to mask the package until someone can fix
it rather than waiting for a fix to become available.

If the developer working as the proxy disappeared, or retired, then the
packages could be assigned back to maintainer-wanted (not
maintainer-needed) but left in the tree until they broke, at which point
they could be removed again unless anyone wants to pick them up.
Similarly, if the users maintaining the package disappeared and the
package broke, it could be masked and removed.

This would seem to me to add more flexibility, and allow more ebuilds to
get into the tree without breaking the tree or causing security
problems. The only difference would be that the developer who took the
package would not be responsible for making sure the program worked -
that would be the responsibility of the users maintaining it in
bugzilla. There should probably be some large, friendly warnings to
inform anyone installing it that is the case, as well.

What do you think?

--
Jonathan Coome  <maedhros@gentoo.org>
Gentoo Forums Moderator

-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Making the developer community more open

[Duncan]

Jonathan Coome posted
<1143024569.27445.23.camel@getafix.chiltonfoliat.org>, excerpted below, 
on Wed, 22 Mar 2006 10:49:29 +0000:

> Taking this idea a bit further, what about proxy maintainers? There seem
> to be quite a few packages that are being effectively maintained by
> users on bugzilla, but are not in portage because they don't have a
> maintainer. A developer could then take these ebuilds, make sure they
> don't do anything malicious, or break QA, or whatever, and act as the
> bridge between the portage tree and the users actually working on the
> ebuild and keeping things up to date and working.

[snip the juicy details]

I think this sounds very much like the Mandrake contrib packages, back
when I was there.  It's a decent idea and it seemed to work reasonably
well, from a user and active cooker/testing participant.

The easiest way to handle "contrib" as far as that "big warning" is to
make  it a separate tree.  That way, folks who want the flexibility get
it, but those who prefer not to "risk it", don't  have to worry about it. 
As well, contribs becomes another fertile developer recruitment ground. 

Unfortunately, for that, portage needs full multi-repository support. 
Overlays function much that way already, but to do it right, we need
multi-repository-update support, and Portage just doesn't have it yet.

...

A possible alternative that could be rolled out sooner would be some form
of "contrib" eclass.  Make it a simple matter to inherit contrib and get
the standard contrib warnings and handling.  One thing the eclass could
handle would be a USE=contrib flag.  With it off, the build wouldn't
merge.  That would take care of user choice.  No non-contrib package could
dep on a contrib package so if such a dependency came about, either the
non-contrib package would have to be dropped (or at least dropped to
contrib status even if it was "contributed" by a dev), or the dep raised 
to full support (non-contrib) status.

The dependency rule above would by definition mean that nobody could get
contrib accidentally, since the only way to get a contrib package would be
merging it or another contrib package that depended on it from the command
line.  This would also solve the interactivity aka broken emerge session
issue, since the portage protest and failure would be right up front,
before merging started.

Making it a use flag would allow control of specific packages thru
package.use, just as now, so a user could decide that he trusted one
contributor as the author of a specific package (and his opinion of the
dep chain) without forcing it to apply to the entire contrib tree.

There remains the question of naming.  A contrib-cat/package tree
paralleling the main category structure would potentially double the
categories right there.  Not really practical.  cat/package-contrib-ver
would be more practical, and allow on-sight identification, but would of
course necessitate a package rename if the contrib vs. full-supported
status changed.  This aspect could be debated if the idea in general gains
enough favor to consider a glep or the like.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Making the developer community more open

[Michael Crute]

On 3/22/06, Duncan <1i5t5.duncan@cox.net> wrote:
> A possible alternative that could be rolled out sooner would be some form
> of "contrib" eclass.  Make it a simple matter to inherit contrib and get
> the standard contrib warnings and handling.  One thing the eclass could
> handle would be a USE=contrib flag.  With it off, the build wouldn't
> merge.  That would take care of user choice.  No non-contrib package could
> dep on a contrib package so if such a dependency came about, either the
> non-contrib package would have to be dropped (or at least dropped to
> contrib status even if it was "contributed" by a dev), or the dep raised
> to full support (non-contrib) status.
>
> The dependency rule above would by definition mean that nobody could get
> contrib accidentally, since the only way to get a contrib package would be
> merging it or another contrib package that depended on it from the command
> line.  This would also solve the interactivity aka broken emerge session
> issue, since the portage protest and failure would be right up front,
> before merging started.
>
> Making it a use flag would allow control of specific packages thru
> package.use, just as now, so a user could decide that he trusted one
> contributor as the author of a specific package (and his opinion of the
> dep chain) without forcing it to apply to the entire contrib tree.
>
> There remains the question of naming.  A contrib-cat/package tree
> paralleling the main category structure would potentially double the
> categories right there.  Not really practical.  cat/package-contrib-ver
> would be more practical, and allow on-sight identification, but would of
> course necessitate a package rename if the contrib vs. full-supported
> status changed.  This aspect could be debated if the idea in general gains
> enough favor to consider a glep or the like.

Maybe taking a slightly different approach at this same idea is  what
needs done. Create a new mask level for contrib where anything deemed
"stable" yet unmaintained could go so that users will have access to
it without needing to search bugzilla or some third-party site.  I
would also propose an unstable mask as well where testing ebuilds can
go, things that are in bugzilla but have not yet been vetted. The
process for getting unstable ebuilds from bugzilla to portage could
even be automated to the extent that when an ebuild is put into
bugzilla it gets auto committed to the tree but masked unstable. This
way all the "latest greatest" ebuilds are always available in the tree
but it requires the user to consciously unmask them for use. You could
even add a big red warning for unstable ebuilds to let people know
that they should examine the ebuild before emerging... just so if they
DO get rooted due to a bad ebuild you can say they where warned.

You could further extend the process of emerging unstable ebuilds so
that a successful emerge would create a vote "for" the ebuild in
bugzilla (attached to the original ebuild bug) and an unsuccessful
emerge would allow the user to add a comment/vote against the ebuild
in bugzilla.

Perhaps it is a radical approach but that's just my $0.02 on how to
open the dev community.

-Mike

--
________________________________
Michael E. Crute
http://mike.crute.org

It is a mistake to think you can solve any major problems just with potatoes.
--Douglas Adams

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Making the developer community more open

[Thomas Cort]

> > A developer could then take these ebuilds, make sure they
> > don't do anything malicious, or break QA, or whatever, and act as the
> > bridge between the portage tree and the users actually working on the
> > ebuild and keeping things up to date and working.

> The easiest way to handle "contrib" as far as that "big warning" is to
> make it a separate tree.  That way, folks who want the flexibility get
> it, but those who prefer not to "risk it", don't  have to worry about it.
> As well, contribs becomes another fertile developer recruitment ground.

Why would the packages need a "big warning"/overlay/eclass if they
were checked by a developer to make sure they "don't do anything
malicious, or break QA, or whatever"? There are many user contributed
ebuilds that have made their way into portage after being reviewed by
devs that don't have any such warnings.

I don't think creating a "contrib" overlay as an official part of
Gentoo would be a good idea because making it an official Gentoo
project conveys a certain level of quality. If the quality is there,
then why not add the ebuilds to portage in the first place? If the
quality isn't there, then you will have a lot of unhappy users
complaining that an official Gentoo overlay broke their system.

Having a non-Gentoo sponsored contrib overlay wouldn't be a good idea
either IMO because the contributors wouldn't be contributing to
Gentoo, and they wouldn't be interacting as much with the Gentoo
developer community. Sure they would learn a lot of the skills
required to be a Gentoo developer, but they wouldn't be increasing the
value of anything in portage (unless they got a proxy to commit some
of their work to portage). Also, there are many overlays out there
already. Adding another one won't help with "making the developer
community more open". Additionally, I don't personally know of a lot
of people who actually use third party overlays except to get an
ebuild for a particular package they want or to beta test ebuilds.

-Thomas

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Making the developer community more open

[Thomas Cort]

> The process for getting unstable ebuilds from bugzilla to portage could
> even be automated to the extent that when an ebuild is put into
> bugzilla it gets auto committed to the tree but masked unstable.

I don't think that auto committing user submitted ebuilds is safe,
even if they are masked. For instance, someone could put something
malicious in global scope in the ebuild. Stuff in global scope gets
interpreted whenever the ebuild is sourced. More info on scope:
http://www.gentoolinux.org/proj/en/devrel/handbook/handbook.xml?part=3&chap=1#doc_chap3_sect4

-Thomas

-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Making the developer community more open

[Dan Meltzer]

Asking developers to "proxy" takes almost as much time as it does to
ask them to maintain a package by themselves.  The developer is
directly responsible for anything he commits, so he will have to still
test the ebuild, still test any revisions, and still follow the
package to make sure there are no problems.  The writing the ebuild
part of the process is not that much of the commitment, I don't see
the point.

On 3/22/06, Thomas Cort <linuxgeek@gmail.com> wrote:
> > > A developer could then take these ebuilds, make sure they
> > > don't do anything malicious, or break QA, or whatever, and act as the
> > > bridge between the portage tree and the users actually working on the
> > > ebuild and keeping things up to date and working.
>
> > The easiest way to handle "contrib" as far as that "big warning" is to
> > make it a separate tree.  That way, folks who want the flexibility get
> > it, but those who prefer not to "risk it", don't  have to worry about it.
> > As well, contribs becomes another fertile developer recruitment ground.
>
> Why would the packages need a "big warning"/overlay/eclass if they
> were checked by a developer to make sure they "don't do anything
> malicious, or break QA, or whatever"? There are many user contributed
> ebuilds that have made their way into portage after being reviewed by
> devs that don't have any such warnings.
>
> I don't think creating a "contrib" overlay as an official part of
> Gentoo would be a good idea because making it an official Gentoo
> project conveys a certain level of quality. If the quality is there,
> then why not add the ebuilds to portage in the first place? If the
> quality isn't there, then you will have a lot of unhappy users
> complaining that an official Gentoo overlay broke their system.
>
> Having a non-Gentoo sponsored contrib overlay wouldn't be a good idea
> either IMO because the contributors wouldn't be contributing to
> Gentoo, and they wouldn't be interacting as much with the Gentoo
> developer community. Sure they would learn a lot of the skills
> required to be a Gentoo developer, but they wouldn't be increasing the
> value of anything in portage (unless they got a proxy to commit some
> of their work to portage). Also, there are many overlays out there
> already. Adding another one won't help with "making the developer
> community more open". Additionally, I don't personally know of a lot
> of people who actually use third party overlays except to get an
> ebuild for a particular package they want or to beta test ebuilds.
>
> -Thomas
>
> --
> gentoo-dev@gentoo.org mailing list
>
>

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Making the developer community more open

[Stuart Herbert]

Hi Daniel,

On 3/20/06, Daniel Drake <dsd@gentoo.org> wrote:
> One of the bigger problems is that we have a huge user community who are
> keen on contributing, but we have such a high barrier for entry to the
> developer community. Quite rightly so - we're dealing with a live tree,
> so we can't give out commit access on the street.
>
> At the same time, I feel that we're missing out. Comparing Gentoo with
> some other large open-source communities that I am personally involved
> in, I feel that we're too closed.

The two big problems are that non-devs don't know where to go to get
involved, and if they want to do more than just chat, there isn't
anywhere for them to go.

I've been very happy with using svn+trac overlays to bridge this gap. 
They provide a sandbox for contributions to be shared and evaluated. 
They provide a place where I've been able to give commit access to
non-devs, so that they can learn the ropes w/out threatening the
Portage tree proper.  They provide a place where people who just want
to write docs for a single package can contribute.

Overlays create a sense of participation that's lacking with Bugzilla
patch submissions.  Backed up with regular communication (I recommend
not recruiting anyone who won't spend time in the IRC channels, but
that's a personal preference), they help us get things done quicker.

The downside with overlays at the moment is that they're scattered
around the net, and if you don't know where to look they can be very
hard to find.  I've been talking with infra about providing
overlays.g.o as a central hosting service for herd and individual
developer overlays.  Infra have been very supportive of the idea.  I
just need to free up some time to get the service launched.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Making the developer community more open

[Jonathan Coome]

On Wed, 2006-03-22 at 09:15 -0500, Dan Meltzer wrote:
> Asking developers to "proxy" takes almost as much time as it does to
> ask them to maintain a package by themselves.  The developer is
> directly responsible for anything he commits, so he will have to still
> test the ebuild, still test any revisions, and still follow the
> package to make sure there are no problems.  The writing the ebuild
> part of the process is not that much of the commitment, I don't see
> the point.

Well no, that's not really what I was suggesting. Developers who took on
these ebuilds would only be responsible for checking that they don't
break the tree and that they do actually work. They aren't responsible
for fixing the package when it breaks, or for following its development
at all - that's the responsibility of the _users_ maintaining the
package. 

Yes, writing the ebuild is the least part of the process, but there's
often a lot more involved, and it's that that's being done in bugzilla
at the moment. The way I see it, the developer would only be responsible
for the ebuilds, and not for doing everything else.

--
Jonathan Coome <maedhros@gentoo.org>
Gentoo Forums Moderator

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 1300 bytes --]

Stuart Herbert wrote:
> I've been very happy with using svn+trac overlays to bridge this gap. 
> They provide a sandbox for contributions to be shared and evaluated. 
> They provide a place where I've been able to give commit access to
> non-devs, so that they can learn the ropes w/out threatening the
> Portage tree proper.  They provide a place where people who just want
> to write docs for a single package can contribute.
> 
> Overlays create a sense of participation that's lacking with Bugzilla
> patch submissions.  Backed up with regular communication (I recommend
> not recruiting anyone who won't spend time in the IRC channels, but
> that's a personal preference), they help us get things done quicker.
> 
> The downside with overlays at the moment is that they're scattered
> around the net, and if you don't know where to look they can be very
> hard to find.  I've been talking with infra about providing
> overlays.g.o as a central hosting service for herd and individual
> developer overlays.  Infra have been very supportive of the idea.  I
> just need to free up some time to get the service launched.

This definitely sounds like a fun idea. It would be even cooler if we
were using a distributed SCM on both ends that allowed for easy merging.

Donnie


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Daniel Ostrow]

[-- Attachment #1: Type: text/plain, Size: 1824 bytes --]

On Wednesday 22 March 2006 12:03, Donnie Berkholz wrote:
> Stuart Herbert wrote:
> > I've been very happy with using svn+trac overlays to bridge this gap.
> > They provide a sandbox for contributions to be shared and evaluated.
> > They provide a place where I've been able to give commit access to
> > non-devs, so that they can learn the ropes w/out threatening the
> > Portage tree proper.  They provide a place where people who just want
> > to write docs for a single package can contribute.
> >
> > Overlays create a sense of participation that's lacking with Bugzilla
> > patch submissions.  Backed up with regular communication (I recommend
> > not recruiting anyone who won't spend time in the IRC channels, but
> > that's a personal preference), they help us get things done quicker.
> >
> > The downside with overlays at the moment is that they're scattered
> > around the net, and if you don't know where to look they can be very
> > hard to find.  I've been talking with infra about providing
> > overlays.g.o as a central hosting service for herd and individual
> > developer overlays.  Infra have been very supportive of the idea.  I
> > just need to free up some time to get the service launched.
>
> This definitely sounds like a fun idea. It would be even cooler if we
> were using a distributed SCM on both ends that allowed for easy merging.

I agree, I'd love to see something like this, that way I could have my xfce 
stuff someplace more public then my devspace....the only thing that would 
have to be clear is how official the overlays actually were, e.g. how prone 
the team looking after the overlay would be to accepting bugs via the usual 
b.g.o channels etc.

-- 
Daniel Ostrow
Gentoo Foundation Board of Trustees
Gentoo/{PPC,PPC64,DevRel}
dostrow@gentoo.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 584 bytes --]

On Wed, 22 Mar 2006 09:03:38 -0800 Donnie Berkholz
<spyderous@gentoo.org> wrote:
| This definitely sounds like a fun idea. It would be even cooler if we
| were using a distributed SCM on both ends that allowed for easy
| merging.

Word of warning to anyone planning to implement one of these that
includes rsync with cache as a frontend: you will be giving root access
to your box to any user with commit access.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Duncan Coutts]

On Wed, 2006-03-22 at 09:03 -0800, Donnie Berkholz wrote:
> Stuart Herbert wrote:
> > I've been very happy with using svn+trac overlays to bridge this gap. 
> > They provide a sandbox for contributions to be shared and evaluated. 
> > They provide a place where I've been able to give commit access to
> > non-devs, so that they can learn the ropes w/out threatening the
> > Portage tree proper.  They provide a place where people who just want
> > to write docs for a single package can contribute.
> > 
> > Overlays create a sense of participation that's lacking with Bugzilla
> > patch submissions.  Backed up with regular communication (I recommend
> > not recruiting anyone who won't spend time in the IRC channels, but
> > that's a personal preference), they help us get things done quicker.
> > 
> > The downside with overlays at the moment is that they're scattered
> > around the net, and if you don't know where to look they can be very
> > hard to find.  I've been talking with infra about providing
> > overlays.g.o as a central hosting service for herd and individual
> > developer overlays.  Infra have been very supportive of the idea.  I
> > just need to free up some time to get the service launched.
> 
> This definitely sounds like a fun idea. It would be even cooler if we
> were using a distributed SCM on both ends that allowed for easy merging.

We do this within the Haskell herd and with a small handful of outside
contributers. We use darcs for our distributed SCM which makes the
merging trivial.

If works like so:
We keep our testing ebuilds in a shared overlay managed with darcs.
The Haskell herd members have write access. Trusted external
contributers have write access to the overlay too (mostly people who are
in the middle of the recruitment process). The existing devs are of
course responsible for actually committing anything to portage cvs.
Other contributers have read only access but they can "darcs send" us
patches. These do not automatically get applied (as with our trusted
contributers) but get emailed to us and any Haskell herd dev can review
and apply patches sent in this manner quite easily.

We have found that this system works rather well. It allows our testers
and helpers to participate in writing ebuilds which has made the
recruitment process smoother. It provides an intermediate phase in the
recruitment process where they can participate in the herd's work
without any danger of messing up portage cvs. Bugzilla is ok for
submitting whole new ebuilds but it's got far too large an overhead for
one line patches. Using darcs gives us a very low overhead.

We also use the shared overlay as a testing zone for our herd's ebuilds.
Our modus-operandi is to commit changes in ebuilds to the overlay, get
peer review from other herd members and then commit to portage when we
are satisfied. Of course this also makes it easy for our testers to keep
up with the latest versions of ebuilds. With the combination of darcs
and irc, we can get very quick turnaround on our testers finding bugs to
fixing them and getting those changes back to our testers.

-- 
Duncan Coutts : Gentoo Developer (Haskell herd team lead)
email         : dcoutts at gentoo dot org

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Stefan Schweizer]

On 3/22/06, Donnie Berkholz <spyderous@gentoo.org> wrote:
> This definitely sounds like a fun idea. It would be even cooler if we
> were using a distributed SCM on both ends that allowed for easy merging.


I think it should be all in a central place possibly saved with
GPG-Keys that need to be signed by trusted persons so that someone can
get access.
Because security seems to be a big problem with a public overlay, but
I think with gpg-key-based-access it could work.

Also see http://aur.archlinux.org for how arch linux is already
successfully doing it.

Regards,
Stefan

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

> This definitely sounds like a fun idea. It would be even cooler if we
> were using a distributed SCM on both ends that allowed for easy merging.
>
> Donnie

It's probably the right time for me to flesh out what I've been
planning for overlays.g.o.

The vision I have for overlays.g.o is one official home for all Gentoo
overlays run by projects and by individual Gentoo devs.  I see the
homepage itself running Planet (just like planet.g.o), using the RSS
feeds from the overlays to display the changelogs from all the
overlays.  The links down the left hand side of the page go to the
homepage for each of the overlays.  The homepages are separate wiki
instances.

  http://overlays.g.o/proj/<project-name>/ for overlays run by herds
listed in herds.xml
  http://overlays.g.o/svn/<project-name>/ for the SVN repos

and

  http://overlays.g.o/dev/<developer>/ for overlays run by individual developers
  http://overlays.g.o/svn/<developer>/ for the SVN repos

There's a practical limit to the amount of software we can support on
overlays.g.o.  The less we install, the less the overhead of
administrating this system for infra and the overlays admin team (I'm
looking for responsible volunteers to join that team, if you're
interested).

I'd like to offer two wiki engines and two version control systems on
overlays.g.o.  I believe that gives us enough choice without us
loading the box with too much software for us to keep on top of.

One thing that was never planned was any form of shell access to this
box, except for the team creating/destroying overlays.  It looks like
this will be necessary to support a distributed vcs.  I'll talk to
infra and see what we could do about providing some form of ssh access
to help us support a distributed vcs.

Trac and SVN would be my first choice.  MoinMoin would be my
recommendation for the second wiki engine.  What should the second
version control system be?  I don't use them, I have no experience
with them, and so I have no preference of what this should be.

To answer Daniel's question about "official" ... the overlays hosted
on overlays.g.o would be "official".  The "overlays" project will be
accountable for overlays.g.o overall.  It would make sense for the
"overlays" project to be a sub-project of infra.

To ensure "officialness" and (what I personally care more about)
accountability, project overlays will be created for projects that
meet the description of a project in the metastructure [1].  The
overlays team will have to be strict on this, to ensure
"officialness".  The overlay must be requested by one of the leads of
the project.  The lead(s) would be jointly accountable for the overlay
and all its contents.  Leads will be able to ask for commit / wiki
edit access for non-devs.

Developer overlays would only be created for active Gentoo developers,
and they would be accountable for its contents.  Non-developers will
not be given write access to developer overlays.

By default - working on the same principle of trust that governs all
developers w.r.t. the Portage tree - all developers will be able to
commit to all overlays.  If we can't trust you to respect other
people's overlays, then we can't trust you with commit access to the
Portage tree, and you're not fit to be a Gentoo dev in the first place
:P  The only "restriction" will be that you'll need to ask the overlay
project team to setup your access the very first time.

Anyone wanting a "secret" overlay needs to make their own hosting arrangements.

To answer Daniel's other question, about bugs.g.o ... trac on
overlays.g.o will have its bug tracking system disabled.  We already
have one bug tracking system - bugs.g.o - and that's sufficient.

[1] http://www.gentoo.org/proj/en/glep/glep-0039.html

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Duncan Coutts]

(re-sending as I sent from the wrong account)
On Wed, 2006-03-22 at 19:42 +0100, Stefan Schweizer wrote:
> On 3/22/06, Donnie Berkholz <spyderous@gentoo.org> wrote:
> > This definitely sounds like a fun idea. It would be even cooler if we
> > were using a distributed SCM on both ends that allowed for easy merging.
> 
> 
> I think it should be all in a central place possibly saved with
> GPG-Keys that need to be signed by trusted persons so that someone can
> get access.
> Because security seems to be a big problem with a public overlay, but
> I think with gpg-key-based-access it could work.

Yes, we use gpg signed patches for our darcs overlay system. We add the
gpg keys of our trusted contributers to a keychain (on the server where
the darcs repo lives). Then they use "darcs send --sign" and their
patches get applied automagically.

Patches from contributers who don't sign their patches (or if the key
check fails) get forwarded to the Haskell herd's email alias so any herd
member can review and apply / reject the patches.

-- 
Duncan Coutts : Gentoo Developer (Haskell herd team lead)
email         : dcoutts at gentoo dot org

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Danny van Dyk]

Hi Stuart,

I'd like to comment on some of your plans for overlays.g.o.

Am Mittwoch, 22. März 2006 23:03 schrieb Stuart Herbert:
> It's probably the right time for me to flesh out what I've been
> planning for overlays.g.o.
>
> The vision I have for overlays.g.o is one official home for all Gentoo
> overlays run by projects and by individual Gentoo devs.  I see the
Also for Arch/Herd Testers?

> homepage itself running Planet (just like planet.g.o), using the RSS
> feeds from the overlays to display the changelogs from all the
> overlays.  The links down the left hand side of the page go to the
> homepage for each of the overlays.  The homepages are separate wiki
> instances.
Well, there is a couple of Gentoo devs who are not particularly comfortable 
with wikis (including myself). Why change things the way they are currently?
I'd suggest to use one repository per project and to add a 'website' or 'site'
directory. In this case we could use the good ol' GuideXML/SCM combination.
>
>   http://overlays.g.o/proj/<project-name>/ for overlays run by herds
> listed in herds.xml
>   http://overlays.g.o/svn/<project-name>/ for the SVN repos
>
> and
Like above: use http://o.g.o/proj/<project-name>/ for the information content
and http://o.g.o/proj/<project-name>/svn/ for the overlay.

>   http://overlays.g.o/dev/<developer>/ for overlays run by individual
> developers http://overlays.g.o/svn/<developer>/ for the SVN repos
same as above :-)

> There's a practical limit to the amount of software we can support on
> overlays.g.o.  The less we install, the less the overhead of
> administrating this system for infra and the overlays admin team (I'm
> looking for responsible volunteers to join that team, if you're
> interested).
Another reason for dropping the wiki
>
> I'd like to offer two wiki engines and two version control systems on
> overlays.g.o.  I believe that gives us enough choice without us
> loading the box with too much software for us to keep on top of.
In case we use wiki, why _two_ wiki engines?

> To answer Daniel's question about "official" ... the overlays hosted
> on overlays.g.o would be "official".  The "overlays" project will be
> accountable for overlays.g.o overall.  It would make sense for the
> "overlays" project to be a sub-project of infra.

> To ensure "officialness" and (what I personally care more about)
> accountability, project overlays will be created for projects that
> meet the description of a project in the metastructure [1].  The
> overlays team will have to be strict on this, to ensure
> "officialness".  The overlay must be requested by one of the leads of
> the project.  The lead(s) would be jointly accountable for the overlay
> and all its contents.  Leads will be able to ask for commit / wiki
> edit access for non-devs.
Please consider also to let QA and Security have commit access to all overlays 
(If they're so inclined).

> Developer overlays would only be created for active Gentoo developers,
> and they would be accountable for its contents.  Non-developers will
> not be given write access to developer overlays.

> By default - working on the same principle of trust that governs all
> developers w.r.t. the Portage tree - all developers will be able to
> commit to all overlays.  If we can't trust you to respect other
> people's overlays, then we can't trust you with commit access to the
> Portage tree, and you're not fit to be a Gentoo dev in the first place
I would say this should be clarified some more. Surely anybody with an access 
to an overlay can commit, but the projects should be the keepers of the keys.
Overlays are not the tree, they are probably very experimental. I could 
imagine that i wouldn't want anyone to modify say an experimental gcc ebuild 
from toolchain's overlay for example.

Stuart: Overall, i like your proposal. I would suggest to turn it into a GLEP 
and I'm willing to help you with this.

Danny
-- 
Danny van Dyk <kugelfang@gentoo.org>
Gentoo/AMD64 Project, Gentoo Scientific Project

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

Hi Danny,

On 3/23/06, Danny van Dyk <kugelfang@gentoo.org> wrote:
> Hi Stuart,
>
> I'd like to comment on some of your plans for overlays.g.o.

:)

> > The vision I have for overlays.g.o is one official home for all Gentoo
> > overlays run by projects and by individual Gentoo devs.  I see the
> Also for Arch/Herd Testers?

Sure.

> Well, there is a couple of Gentoo devs who are not particularly comfortable
> with wikis (including myself). Why change things the way they are currently?

Because previous threads here on -dev asking for a wiki prove that not
everyone is comfortable / happy with how things currently are ...
myself included.

Wikis are a much lower barrier to entry than GuideXML ... and they
have advantages over GuideXML.

There's no reason why you can't create GuideXML and store / publish it
via the overlay, even if there is a wiki.  We do that with the PHP
overlay.

> I'd suggest to use one repository per project and to add a 'website' or 'site'
> directory. In this case we could use the good ol' GuideXML/SCM combination.

That's easy enough.  We can establish a 'known location' in the VCS
tree where GuideXML will be stored, and run a simple cron script to
extract it and put it into the website directory for public viewing.

Or you could just publish it on www.g.o/proj/en/<project>/ instead :)

> Like above: use http://o.g.o/proj/<project-name>/ for the information content
> and http://o.g.o/proj/<project-name>/svn/ for the overlay.

Could do.  I always preferred separate paths to ensure no clash with
any other content under /proj/<project-name>/.

> Another reason for dropping the wiki

No.  We can make the wiki optional, but not offering a wiki at all
makes it impossible for existing successful, externally hosted
overlays to move to overlays.g.o.

> In case we use wiki, why _two_ wiki engines?

Because different people have different preferences, and I don't
believe in 'one size fits all'.  Adding a little bit of flexibility in
the right places will make o.g.o more successful.

> Please consider also to let QA and Security have commit access to all overlays
> (If they're so inclined).

That's already covered under the principle that QA and Security are
staffed by devs.

> I would say this should be clarified some more. Surely anybody with an access
> to an overlay can commit, but the projects should be the keepers of the keys.
> Overlays are not the tree, they are probably very experimental. I could
> imagine that i wouldn't want anyone to modify say an experimental gcc ebuild
> from toolchain's overlay for example.

I want to operate the overlays on the same principles that we use to
manage the Portage tree.  We tell developers that they have to respect
other people's packages, and can't go around changing what they feel
like.  The same should hold true for the overlays.

> Stuart: Overall, i like your proposal. I would suggest to turn it into a GLEP
> and I'm willing to help you with this.

I already have Infra's agreement and active support to deliver this (I
can't thank Lance and Kurt enough for their help to date on this). 
It's a new service - one that no-one is required to use (although I
ferverently hope that it proves very popular).  I don't believe that
it needs a GLEP.

What it does need is a successful overlay project team, to administer
the service and help it evolve over the coming years.

Best regards,
Stu
--

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Luis Medinas]

On Wed, 2006-03-22 at 22:03 +0000, Stuart Herbert wrote:
> I'd like to offer two wiki engines and two version control systems on
> overlays.g.o.  I believe that gives us enough choice without us
> loading the box with too much software for us to keep on top of.
> 
> One thing that was never planned was any form of shell access to this
> box, except for the team creating/destroying overlays.  It looks like
> this will be necessary to support a distributed vcs.  I'll talk to
> infra and see what we could do about providing some form of ssh access
> to help us support a distributed vcs.
> 
> Trac and SVN would be my first choice.  MoinMoin would be my
> recommendation for the second wiki engine.  What should the second
> version control system be?  I don't use them, I have no experience
> with them, and so I have no preference of what this should be.
> 
> To answer Daniel's question about "official" ... the overlays hosted
> on overlays.g.o would be "official".  The "overlays" project will be
> accountable for overlays.g.o overall.  It would make sense for the
> "overlays" project to be a sub-project of infra.
> 
> To ensure "officialness" and (what I personally care more about)
> accountability, project overlays will be created for projects that
> meet the description of a project in the metastructure [1].  The
> overlays team will have to be strict on this, to ensure
> "officialness".  The overlay must be requested by one of the leads of
> the project.  The lead(s) would be jointly accountable for the overlay
> and all its contents.  Leads will be able to ask for commit / wiki
> edit access for non-devs.
> 
> Developer overlays would only be created for active Gentoo developers,
> and they would be accountable for its contents.  Non-developers will
> not be given write access to developer overlays.
> 
> By default - working on the same principle of trust that governs all
> developers w.r.t. the Portage tree - all developers will be able to
> commit to all overlays.  If we can't trust you to respect other
> people's overlays, then we can't trust you with commit access to the
> Portage tree, and you're not fit to be a Gentoo dev in the first place
> :P  The only "restriction" will be that you'll need to ask the overlay
> project team to setup your access the very first time.
> 
> Anyone wanting a "secret" overlay needs to make their own hosting arrangements.
> 
> To answer Daniel's other question, about bugs.g.o ... trac on
> overlays.g.o will have its bug tracking system disabled.  We already
> have one bug tracking system - bugs.g.o - and that's sufficient.
> 
> [1] http://www.gentoo.org/proj/en/glep/glep-0039.html

Hi Stuart

I agree with the wiki because it seems to be an easy way to users and
developers comunicate together and work. Like i said a few months ago
the documentation won't give any problems to GDP since GDP provides high
level docs. The wiki will also help our projects since it can be added
TODO's, roadmaps and all that stuff. About the overlay the best way imo
is provide a unique overlay with external contribs maintained by users
and devs (of course commit rights only for devs.).

-- 
Luis Medinas <metalgod@gentoo.org>
http://dev.gentoo.org/~metalgod
Gentoo Linux Developer: AMD64,Printing,Media-Optical,Sound

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 345 bytes --]

Stuart Herbert wrote:
> Developer overlays would only be created for active Gentoo developers,
> and they would be accountable for its contents.  Non-developers will
> not be given write access to developer overlays.

This removes much of the motivation for merging overlays to o.g.o, at
least some of the ones I am aware of.

Donnie


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

On 3/23/06, Donnie Berkholz <spyderous@gentoo.org> wrote:
> Stuart Herbert wrote:
> > Developer overlays would only be created for active Gentoo developers,
> > and they would be accountable for its contents.  Non-developers will
> > not be given write access to developer overlays.
>
> This removes much of the motivation for merging overlays to o.g.o, at
> least some of the ones I am aware of.

Then I'll re-consider.

I was hoping that developers would setup projects if there are
multiple contributors (project overlays will allow write access to
non-devs).

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Chris Bainbridge]

On 23/03/06, Stuart Herbert <stuart.herbert@gmail.com> wrote:
> > > The vision I have for overlays.g.o is one official home for all Gentoo
> > > overlays run by projects and by individual Gentoo devs.  I see the
> > Also for Arch/Herd Testers?

The discussion seems to have moved from the original "how can we
become more open to enable our users to contribute?" to "how to
provide testing overlays for existing gentoo devs". I think that the
use of overlays is more a symptom of a problem with portage. Overlay
problems:

They remove ebuilds from the tree
Users have to add yet another overlay / retrieve the ebuilds somehow
Conflicts between overlays
Increases time to moving packages into the tree
Overlay becomes a developers "personal space" making it difficult to
contribute if package is neglected (though that is also a problem
now...)
Lose repository metadata moving a package from overlay to tree
Reduced responsibility for package QA  (I expect "we don't care about
overlays" to become a standard response on bugs.g.o)

And what do we gain:

Eases testing - can push in alpha quality ebuilds
Developers feel safer because they can't break the tree

Surely the solution is to provide that safety net within the tree?
Rather than pushing changes into a live tree, push them into a testing
tree, then after they pass repoman/QA checks, and a build check, apply
the changes to the live tree, otherwise email a rejection. And allow
developers to add their own testing ebuilds to the tree (for a start,
they will be more widely tested).

The current system of overlay usage is very annoying for users,
particularly when bugs are hanging around with packages in the tree,
and after filing bug reports the user is told that the bug is already
fixed in the overlay. Or when new packages are added to overlays
instead of the tree. How are users expected to find them?

Another thing that needs fixing is the massive number of packages that
aren't really maintained. Either the maintainer doesn't respond to
bugs, or the package is maintained by a herd and so no one feels it's
actually their responsibility to deal with the boring bugs, and when
some developer outside of the herd comes across it, they feel like
they can't fix the bug without stepping on someone's toes. What's
worse is that in a lot of these cases there will be a user on bugs.g.o
posting fixes and new ebuilds, and yet they never make it into the
tree.

A system where developer ebuilds are kept in the tree, and users have
a way to automatically contribute ebuilds, either human reviewed, or
in some reputation based system, would be very useful. Users also need
feedback - how many times does a user submit an ebuild via bugzilla to
be told that it doesn't meet QA standards? Why isn't there a system in
place to run repoman/QA/build checks on user ebuilds/patches to make
sure they are fixed *before* being submitted for inclusion into the
tree? And if this could be linked to a bug reporting system where
people report/fix individual ebuilds or packages, and I can just type
'gbugs -l pkgname' and get a list of problems and fixes that other
people have proposed, even better.

I'm not sure whether the answer is more openness of the existing
system, some custom submission flow system, or a distributed SCM, but
I do think there's a lot that could be changed to make things better.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

Hi Luis,

On 3/23/06, Luis Medinas <metalgod@gentoo.org> wrote:
> I agree with the wiki because it seems to be an easy way to users and
> developers comunicate together and work. Like i said a few months ago
> the documentation won't give any problems to GDP since GDP provides high
> level docs. The wiki will also help our projects since it can be added
> TODO's, roadmaps and all that stuff. About the overlay the best way imo
> is provide a unique overlay with external contribs maintained by users
> and devs (of course commit rights only for devs.).

Every team will have their own preference on how to get the most out
of an overlay.  No-one's required to use an overlay, and no-one's
required to give out access to non-devs if they don't want to.  It'll
be okay for different teams to have different practices for their
overlays.

I believe the overlay team's job is to provide the capability, clearly
communicate what isn't allowed, provide technical documentation and
some examples of practices already in use ... and then get out of your
way.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 883 bytes --]

Stuart Herbert wrote:
> On 3/23/06, Donnie Berkholz <spyderous@gentoo.org> wrote:
>> Stuart Herbert wrote:
>>> Developer overlays would only be created for active Gentoo developers,
>>> and they would be accountable for its contents.  Non-developers will
>>> not be given write access to developer overlays.
>> This removes much of the motivation for merging overlays to o.g.o, at
>> least some of the ones I am aware of.
> 
> Then I'll re-consider.
> 
> I was hoping that developers would setup projects if there are
> multiple contributors (project overlays will allow write access to
> non-devs).

I don't think I'm understanding your intent here, because I've read
things two different ways. My main goal is to allow easy contribution by
non-devs, via allowing them to commit directly to some overlay. How is
that possible in your vision?

Thanks,
Donnie


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

Hi Chris,

On 3/23/06, Chris Bainbridge <chris.bainbridge@gmail.com> wrote:
> I think that the
> use of overlays is more a symptom of a problem with portage. Overlay
> problems:

[snip]

Developers are already using overlays, and some teams (including ones
I've been involved in) are actively and successfully using them to
help with recruitment and to provide a way to access ebuilds that
would otherwise still be rotting in Bugzilla.

I'm not saying overlays are for everyone.  It's one approach that some
groups like.  You don't have to use an overlay yourself for your work
if it's an approach that doesn't work for you.  Overlays are not about
to become a mandatory way of working around here.

> Surely the solution is to provide that safety net within the tree?

I cannot imagine a day when non-devs are given commit access to the
Portage tree.  As long as that limitation remains in place, if we're
going to reach out beyond our developer community, we have to reach
out beyond the Portage tree too.

> The current system of overlay usage is very annoying for users,
> particularly when bugs are hanging around with packages in the tree,
> and after filing bug reports the user is told that the bug is already
> fixed in the overlay. Or when new packages are added to overlays
> instead of the tree. How are users expected to find them?

Users have pre-conceived ideas about the contents of the Portage tree.
 I've seen first-hand how badly users react when a hard-masked package
in the tree is withdrawn because it was an experimental approach that
ultimately failed.  Users have unrealistic expectations about the
tree.

If developers are telling users in Bugzilla that bugs are fixed in the
overlay, it's the responsibility of the developers to tell the users
where to go to get those fixes.  I'd have thought that was basic
common sense.  Establishing overlays.g.o as the usual place to go will
help with this, as will promoting very helpful tools such as layman.

> Another thing that needs fixing is the massive number of packages that
> aren't really maintained.

That's a very valid concern.  Overlays can help here, as explained by
the Haskell team, because they make it possible for more people to
contribute.  They're more than a technical solution ... they're a
social tool to build a more sustainable team around.

> What's
> worse is that in a lot of these cases there will be a user on bugs.g.o
> posting fixes and new ebuilds, and yet they never make it into the
> tree.

I'm finding that overlays address this specific scenario very
successfully.  I talk to those users and find out if they're willing
to contribute to an overlay.  More often than not, I find that the
fixes and new ebuilds are coming from a personal overlay that the user
is maintaining anyway.  Being able to commit directly to a shared
overlay means that they can get more involved ... and it gives me a
good level of interaction to help decide whether the person is worth
recruiting as a full-blown dev or not.

> A system where developer ebuilds are kept in the tree, and users have
> a way to automatically contribute ebuilds, either human reviewed, or
> in some reputation based system, would be very useful.

The overlays project doesn't prevent this system being developed or
introduced at all.  We're not looking to corner the market at all. 
We're only providing a resource which will be useful to some teams and
developers.

> Users also need feedback

[snip]  You have good ideas.  What are you doing to make them happen?

> I'm not sure whether the answer is more openness of the existing
> system, some custom submission flow system, or a distributed SCM, but
> I do think there's a lot that could be changed to make things better.

I don't think that there is any one approach that will work for all
devs, all non-devs, all the time.  What I'm doing here is resourcing
one specific approach, and working with Infra and User Relations to
deliver something that provides one bridge between the developer and
non-dev community.  We will need other bridges to be built too.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

Hi Donnie,

On 3/23/06, Donnie Berkholz <spyderous@gentoo.org> wrote:
> I don't think I'm understanding your intent here, because I've read
> things two different ways. My main goal is to allow easy contribution by
> non-devs, via allowing them to commit directly to some overlay. How is
> that possible in your vision?

That's possible because

a) non-devs can be given commit access to the overlay's svn/a.n.other repository
b) non-devs can be given create/edit access to the overlay's wiki / GuideXSL

I do this already for the overlays I host on svn.gnqs.org.  I can't
migrate those overlays to overlays.g.o if we don't grant non-dev
commit access!

The confusion is probably because, in the original vision statement, I
said that these things would only happen for overlays setup by, and
for, official projects.  I wanted a disctinction between who could
commit to overlays run by projects, and who could commit to overlays
run by individual developers.

However, if the concensus is that we want non-devs to be able to
commit to individual developer overlays too, we'll make that possible.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 740 bytes --]

Stuart Herbert wrote:
> The confusion is probably because, in the original vision statement, I
> said that these things would only happen for overlays setup by, and
> for, official projects.  I wanted a disctinction between who could
> commit to overlays run by projects, and who could commit to overlays
> run by individual developers.
> 
> However, if the concensus is that we want non-devs to be able to
> commit to individual developer overlays too, we'll make that possible.

Ah, now it's clear. I didn't understand that distinction earlier, and I
could deal with this proposal.

But it seems rather artificial to me, and I suspect some devs might
enjoy contributions to their non-topical overlays.

Thanks,
Donnie


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

> But it seems rather artificial to me, and I suspect some devs might
> enjoy contributions to their non-topical overlays.

It *is* artificial; that's fair critisism.  I have a personal bias
towards projects.  I'll withdraw the distinction.

So, to be clear: the owners of an overlay (the leads for a project,
the developer for a developer overlay) can request that a non-dev be
given commit rights to the overlay's VCS and / or wiki.  The owners of
an overlay are responsible for everything that happens to an overlay -
including the contributions of non-developers.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Chris Bainbridge]

On 23/03/06, Stuart Herbert <stuart.herbert@gmail.com> wrote:
> Developers are already using overlays, and some teams (including ones
> I've been involved in) are actively and successfully using them to
> help with recruitment and to provide a way to access ebuilds that
> would otherwise still be rotting in Bugzilla.

Developers are using overlays, however, the majority of users aren't.
If the usage of overlays is to increase, then remote overlay support
should be added to emerge. Additionally, in order for users to be able
to contribute to the overlays, it would help if they had anonymous
read access.

> > Surely the solution is to provide that safety net within the tree?
>
> I cannot imagine a day when non-devs are given commit access to the
> Portage tree.  As long as that limitation remains in place, if we're
> going to reach out beyond our developer community, we have to reach
> out beyond the Portage tree too.

The safety net was intended for developers. Packages often get broken
in unexpected ways - something depends on it, a patch is conditional
on some USE flag or arch etc. It would be useful to get an email 5
minutes after a commit saying "you broke something", rather than a bug
report being filed a week later.

> > The current system of overlay usage is very annoying for users,
> > particularly when bugs are hanging around with packages in the tree,
> > and after filing bug reports the user is told that the bug is already
> > fixed in the overlay. Or when new packages are added to overlays
> > instead of the tree. How are users expected to find them?
>
> Users have pre-conceived ideas about the contents of the Portage tree.
>  I've seen first-hand how badly users react when a hard-masked package
> in the tree is withdrawn because it was an experimental approach that
> ultimately failed.  Users have unrealistic expectations about the
> tree.

I don't think it is unrealistic to expect that if a user puts a lot of
work into an ebuild, and it works, then it should somehow end up in
the tree. Unfortunately the options at the moment are to either reject
the ebuild, leave it to rot in bugzilla, or recruit the user as a
developer. Rejecting isn't very nice, the amount of effort and
barriers to become a dev are too great, so we end up with good ebuilds
not being added. Adding the ebuilds to overlays is one option, but
then other users will be expected to find an overlay with their
package in, and then add it to make.conf. As the number of overlays
increases, the amount of effort in synchronising dependencies and
dealing with other problems between them will increase.

Maybe in the real world managing the relationships between overlays
won't be as big a problem as it appears it could be.

> [snip]  You have good ideas.  What are you doing to make them happen?

Not much - time is a great constraint, and writing emails takes much
less time than writing code...

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

Hi Chris,

On 3/23/06, Chris Bainbridge <chris.bainbridge@gmail.com> wrote:
> Developers are using overlays, however, the majority of users aren't.

True.  But does that have to be the audience for overlays?

> If the usage of overlays is to increase, then remote overlay support
> should be added to emerge. Additionally, in order for users to be able
> to contribute to the overlays, it would help if they had anonymous
> read access.

I forgot to mention that anonymous read access would be available,
although we'll have to see how that impacts the hardware.  We may need
to raise some funds / scrounge some extra kit to make o.g.o scale if
it proves wildly popular.

As for remote overlay support ... try using layman to see if that's
enough for today.  If we find we need more support in future, we can
revisit the situation.

We're not trying to get all users to use overlays instead of the
Portage tree.  We're just creating a social space where non-devs who
want to contribute can work more easily with existing devs.

> The safety net was intended for developers.

Then it's a change in topic.  Please start a new thread on here for it.

> Packages often get broken
> in unexpected ways - something depends on it, a patch is conditional
> on some USE flag or arch etc. It would be useful to get an email 5
> minutes after a commit saying "you broke something", rather than a bug
> report being filed a week later.

How does that differ from the service that autorepoman already
provides?  Maybe you need to be talking directly to Mark and the QA
team about this.

> I don't think it is unrealistic to expect that if a user puts a lot of
> work into an ebuild, and it works, then it should somehow end up in
> the tree.

Okay, that's something I'd like to nip in the bud right there.

Something "working" is not the only criteria that Gentoo requires for
a package to go into the tree.  I'm sure you already know that.  If we
don't have a developer willing to maintain the package, it doesn't
belong in the tree.

It's not sustainable to have "fire-and-forget" packages lying around.

One problem I've seen with recruitment is devs who don't "stick", who
stop contributing after a short period of time, and who fade away. 
I'm not saying they're the only answer, but overlays can be used to
see whether someone will make a sustained effort over a decent period
of time, before they are recruited.

> Rejecting isn't very nice,

Agreed.

> the amount of effort and barriers to become a dev are too great

I can't agree with you there.  I believe we can make it easier, and do
so without changing the amount of effort or the deliberate barriers we
have today.

> so we end up with good ebuilds not being added.

Good ebuilds aren't enough.

> Adding the ebuilds to overlays is one option, but
> then other users will be expected to find an overlay with their
> package in, and then add it to make.conf. As the number of overlays
> increases, the amount of effort in synchronising dependencies and
> dealing with other problems between them will increase.

Perhaps.  Perhaps not.

But that's only one aspect of overlays - it's not the whole shebang. 
There are developers and teams that use overlays to maintain packages
that are in the tree, and then they push the packages from the overlay
to the tree when the packages are ready for wider testing.

> Maybe in the real world managing the relationships between overlays
> won't be as big a problem as it appears it could be.

Take layman out for a spin, and let the author know how well it helps with this.

> Not much - time is a great constraint, and writing emails takes much
> less time than writing code...

I appreciate your honesty there :)

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1384 bytes --]

On Wed, 2006-03-22 at 22:03 +0000, Stuart Herbert wrote:
> To answer Daniel's other question, about bugs.g.o ... trac on
> overlays.g.o will have its bug tracking system disabled.  We already
> have one bug tracking system - bugs.g.o - and that's sufficient.

Umm... no?

If some random developer goes out there and creates his own fork of
catalyst in his overlay, I sure don't want to receive a *single* bug on
it.  Ever.

If you're already using Trac, you should keep the bug tracking enabled,
so the bugs stay with the overlay.  Once something moves into the
official tree, then it can use bugs.gentoo.org for its bug tracking.
This means developers that don't wish to participate in the overlays are
not forced to waste their time troubleshooting problems in these
overlays and can focus on our *core* product, the portage tree.

I have no problem with someone, for example, making their own fork of
catalyst for testing new stuff and then, after extensive testing,
submitting it "upstream" to the official repository, but forcing me to
waste my time trying to figure out that some random developer out there
has made a fork that does something different from the official version
when a user has a bug is completely counter-productive.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

Hi Chris,

On 3/23/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> If some random developer goes out there and creates his own fork of
> catalyst in his overlay, I sure don't want to receive a *single* bug on
> it.  Ever.

Your nightmare scenario seems unavoidable.  Enabling per-overlay bug
tracking doesn't stop users posting bugs in bugzilla.  It just causes
confusion for users, because they're not sure where to go.  Normally,
it's not a problem - because the overlay contributors are normally the
owners of the real package.

A hostile fork of Catalyst is very much a special case.

What we could do is say that overlays are for package trees only; ie
they are not general-purpose repositories for holding source trees. 
That would ensure that your nightmare scenario is even less likely to
happen, and that if it does, it's through no fault of the overlays
project :)

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 6902 bytes --]

On Thu, 2006-03-23 at 10:09 +0000, Chris Bainbridge wrote:
> Reduced responsibility for package QA  (I expect "we don't care about
> overlays" to become a standard response on bugs.g.o)

You will *definitely* get this from developers that won't be using the
overlays.

Let's just say you decide to use a toolchain overlay and it exposes some
problem in random app foo because you're using gcc 5.1.99 and we only
have 4.0 in the tree.  You file a bug against package foo without a
patch.  I'm the maintainer.  You've now made me spend my time supporting
something that isn't even in the tree, and could be an artifact of the
overlay itself and something that will *never* end up in the tree.  Why
should I do this?  What we have done here is actually *reduced* the
amount of productive work that I can do by forcing me to deal with these
overlays, even if I choose not to participate.

> Surely the solution is to provide that safety net within the tree?
> Rather than pushing changes into a live tree, push them into a testing
> tree, then after they pass repoman/QA checks, and a build check, apply
> the changes to the live tree, otherwise email a rejection. And allow
> developers to add their own testing ebuilds to the tree (for a start,
> they will be more widely tested).

While this would be great, there is one major obstacle that people miss.
Horsepower.

Let's say I add a new openoffice ebuild to the tree for a security bug.
We now have to wait how many hours for it to hit the tree?  What if the
KDE team is committing a new KDE version at the same time?  I'm sure you
can guess how this could bring all of our development to a complete and
grinding halt.

I wouldn't mind seeing an actual "unstable" designation added to
KEYWORDS.  The basic premise would be like package.mask packages where
things can be done *within the tree* but still has the same air of "this
might be totally busted at some point" as overlays.  Users would be very
unlikely to run unstable on their machines.  Heck, we could even make it
impossible to actually use unstable globally if we wanted to.  The whole
point is that I completely agree with you, a better solution would be to
try to work within the tree to resolve this problem, rather than moving
it outside the tree and into hundreds of individual overlays, which
could be conflicting with each other.

> The current system of overlay usage is very annoying for users,
> particularly when bugs are hanging around with packages in the tree,
> and after filing bug reports the user is told that the bug is already
> fixed in the overlay. Or when new packages are added to overlays
> instead of the tree. How are users expected to find them?

They should not have to.  It's as simple as that.  Bugs in the tree
should be fixed in the tree.  New packages should be added to the tree.
If it isn't a candidate for ~arch, then add it under package.mask
instead.  That is why we have package.mask in the first place.

> Another thing that needs fixing is the massive number of packages that
> aren't really maintained. Either the maintainer doesn't respond to
> bugs, or the package is maintained by a herd and so no one feels it's
> actually their responsibility to deal with the boring bugs, and when
> some developer outside of the herd comes across it, they feel like
> they can't fix the bug without stepping on someone's toes. What's
> worse is that in a lot of these cases there will be a user on bugs.g.o
> posting fixes and new ebuilds, and yet they never make it into the
> tree.

This is really both a political/social problem and one of manpower.
There's a lot more users out there than there are developers.  There are
many developers out there who aren't quite so territorial.  The main
thing is us being civil with each other.  There are many times where a
developer wants to make a fix or change to a game.  If they ask us, we
let them.  It's that simple.  Heck, we usually ask them if they want to
maintain it.  Also, a herd is a group of packages, not a group of
developers.  A group of developers could share responsibility in looking
out for a herd (or many) but they are not a herd themselves.

> A system where developer ebuilds are kept in the tree, and users have
> a way to automatically contribute ebuilds, either human reviewed, or
> in some reputation based system, would be very useful. Users also need
> feedback - how many times does a user submit an ebuild via bugzilla to
> be told that it doesn't meet QA standards? Why isn't there a system in
> place to run repoman/QA/build checks on user ebuilds/patches to make
> sure they are fixed *before* being submitted for inclusion into the
> tree? And if this could be linked to a bug reporting system where
> people report/fix individual ebuilds or packages, and I can just type
> 'gbugs -l pkgname' and get a list of problems and fixes that other
> people have proposed, even better.

Ebuilds that are human reviewed are exactly what we have now.  The
process isn't automated, but it cannot be automated if you expect human
review.  Automated review isn't possible, as it is very easy to work
around automated tests to do something completely malicious in an
ebuild.  There should *always* be the human factor before *anything*
makes it into the tree.  Our reputation and even the stake of Gentoo as
a whole depends on our reliability.

As for ebuilds meeting QA standards, most developers that I am aware of
will inform the user of how to make their ebuild better, and even point
them to relevant documentation to assist them, expecting the user to
make the changes.  This makes for better submission and informs the user
so they don't make the same mistake twice.  All in all, it is good for
all involved.  Any developer that is closing a bug without reason is in
the wrong.  It is as simple as that.  Saying something doesn't meet QA
is bunk *unless* they also point you to reasons *why* it fails QA.

As for bugs being posted, you're more than welcome to use "ALL pkgname"
as a search on bugzilla to get anything related to that package.
There's even a few command-line bugzilla query tools.

> I'm not sure whether the answer is more openness of the existing
> system, some custom submission flow system, or a distributed SCM, but
> I do think there's a lot that could be changed to make things better.

See, I 100% disagree that this is a technical problem.  It cannot have a
technical solution.  Want better developer/user relations?  Start
talking amongst ourselves.  Talk to other users.  Talk to other
developers.

I tend to think our biggest failure is communication.  Improve our
communication and we'll improve Gentoo.  Together.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Eric Edgar]

[-- Attachment #1: Type: text/plain, Size: 2745 bytes --]

I personally think this is a bad idea.  I can understand and support
links to different overlay repositories, however I do not think that
gentoo should host or support overlays on its own infrastructure.  For
one thing supporting overlays on our infrastructure looks like we are
supporting broken ebuilds.  This will also lead to more confusion with
users who find these official overlays and then the overlays conflict
with each other and cause problems that leads to comments like well
gentoo should just know how to fix it and make it all work.  I also
think that this overlay structure will not provide incentives for people
to commit to the main tree.  They will get their ebuild in an overlay
and its hosted on gentoo and distributed to the mirrors.  At that point
its easy for them to continue to use the overlay.  Over time the overlay
will diverge more and more from other overlays and even the main tree.

If this still goes forward I think we should look at the debian layout
where there is the core product and then the security branches etc.

Personally I think this is going to cause more bug reports and less
updates to the main tree.

I also agree that a hostile fork is unlikely, however it is more
possible with the overlay layout as anyone can get an ebuild mirrored on
our infrastructure at this point.

Another thing to concider is how would people be able to contribute to
the overlays?  Is there a review process?  Is there a checkin process?
If no then anyone can post a malicious ebuild that would be a security
nightmare.  I think this security viewpoint alone is enough to
re-evaluate this plan of action.

Eric

On 14:41 Thu 23 Mar     , Stuart Herbert wrote:
> Hi Chris,
> 
> On 3/23/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> > If some random developer goes out there and creates his own fork of
> > catalyst in his overlay, I sure don't want to receive a *single* bug on
> > it.  Ever.
> 
> Your nightmare scenario seems unavoidable.  Enabling per-overlay bug
> tracking doesn't stop users posting bugs in bugzilla.  It just causes
> confusion for users, because they're not sure where to go.  Normally,
> it's not a problem - because the overlay contributors are normally the
> owners of the real package.
> 
> A hostile fork of Catalyst is very much a special case.
> 
> What we could do is say that overlays are for package trees only; ie
> they are not general-purpose repositories for holding source trees. 
> That would ensure that your nightmare scenario is even less likely to
> happen, and that if it does, it's through no fault of the overlays
> project :)
> 
> Best regards,
> Stu
> 
> -- 
> gentoo-dev@gentoo.org mailing list
> 
> 

[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Jakub Moc]

[-- Attachment #1: Type: text/plain, Size: 1961 bytes --]

Chris Gianelloni wrote:
> On Wed, 2006-03-22 at 22:03 +0000, Stuart Herbert wrote:
>> To answer Daniel's other question, about bugs.g.o ... trac on
>> overlays.g.o will have its bug tracking system disabled.  We already
>> have one bug tracking system - bugs.g.o - and that's sufficient.
> 
> Umm... no?
> 
> If some random developer goes out there and creates his own fork of
> catalyst in his overlay, I sure don't want to receive a *single* bug on
> it.  Ever.
> 
> If you're already using Trac, you should keep the bug tracking enabled,
> so the bugs stay with the overlay.  Once something moves into the
> official tree, then it can use bugs.gentoo.org for its bug tracking.
> This means developers that don't wish to participate in the overlays are
> not forced to waste their time troubleshooting problems in these
> overlays and can focus on our *core* product, the portage tree.

Well, I don't care much, as long as:

- there's a separate Overlays product in bugzilla for this

- each such overlay has its own component under Overlay product with
default assignees set up (no, I won't check out all those overlays to
find out the maintainer, also, almost none of them uses metadata.xml)

- users are *vigorously* :P instructed to file the bugs under that
product/component and/or (?) mark them with something like [overlay-xxx]
so that I don't have to ponder who's maintaining that thing. If they
don't, the bugs end up as invalid b/c the ebuild is not in portage. Easy
enough.

While keeping those bugs in trac bug trackers seemed as a good idea to
me originally, most users are simply unable to do that anyway. We tried
w/ php overlay, didn't work much.


-- 
Best regards,

 Jakub Moc
 mailto:jakub@gentoo.org
 GPG signature:
 http://subkeys.pgp.net:11371/pks/lookup?op=get&search=0xCEBA3D9E
 Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95  B30F 8717 D5FD CEBA 3D9E

 ... still no signature   ;)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2569 bytes --]

On Thu, 2006-03-23 at 14:41 +0000, Stuart Herbert wrote:
> Hi Chris,
> 
> On 3/23/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> > If some random developer goes out there and creates his own fork of
> > catalyst in his overlay, I sure don't want to receive a *single* bug on
> > it.  Ever.
> 
> Your nightmare scenario seems unavoidable.  Enabling per-overlay bug
> tracking doesn't stop users posting bugs in bugzilla.  It just causes
> confusion for users, because they're not sure where to go.  Normally,
> it's not a problem - because the overlay contributors are normally the
> owners of the real package.

No, it does not stop them, but it sure will curb the number of users
posting their bugs to the wrong place.  Remember that only more advanced
users are the ones using overlays.  We won't have Joe Sixpack using an
overlay.  Instead it'll be Bob Developer-to-be.

How is that confusing?  I went to http://overlays.gentoo.org/catalyst-ng
and saw the overlay.  I also saw the link the the bug tracker.

> A hostile fork of Catalyst is very much a special case.

No.  It isn't.  Look in many developer overlays and you'll see packages
that they have made that work how *they* want them to, even if it is
*very* different from what is in the tree.  This is the case for
packages that are not maintained by them, too.  Any ebuild that is done
by someone that isn't the maintainer is a fork.  There's nothing
"hostile" about it.

> What we could do is say that overlays are for package trees only; ie
> they are not general-purpose repositories for holding source trees. 
> That would ensure that your nightmare scenario is even less likely to
> happen, and that if it does, it's through no fault of the overlays
> project :)

It has nothing to do with a source tree.  I could store the source tree,
and tarball, anywhere.  The ebuilds to use these tarballs would be just
as dangerous.

I see no problem with overlays in concept, such as the php overlay that
is very successful.  The main reason that it is successful is because
the same people that maintain php maintain the overlay.  Yes, there are
other contributors, but the maintainers of the overlay are still the
developers.  I see no problem with providing these sorts of overlays to
bridge the gap between contributing users and developers.  I *do* see a
problem with simply allowing random overlays from any developer for
anything.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

On 3/23/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> I see no problem with providing these sorts of overlays to
> bridge the gap between contributing users and developers.  I *do* see a
> problem with simply allowing random overlays from any developer for
> anything.

That's a reasonable point, and it's an experience I've not personally
been on the receiving end of.

But the flip side is reasonable too.  Our governing metastructure is
explicitly clear that:

a) Projects are not mandatory, and
b) That competing projects are allowed

I don't see how we can limit overlays to just packages "owned" by the
developers who own the overlay without violating those two policies.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Jeroen Roovers]

On Thu, 23 Mar 2006 10:31:40 -0500
Chris Gianelloni <wolf31o2@gentoo.org> wrote:

> > Your nightmare scenario seems unavoidable.  Enabling per-overlay bug
> > tracking doesn't stop users posting bugs in bugzilla.  It just
> > causes confusion for users, because they're not sure where to go.
> > Normally, it's not a problem - because the overlay contributors are
> > normally the owners of the real package.
> 
> No, it does not stop them, but it sure will curb the number of users
> posting their bugs to the wrong place.  Remember that only more
> advanced users are the ones using overlays.  We won't have Joe
> Sixpack using an overlay.  Instead it'll be Bob Developer-to-be.

Advanced users can write helpful HOWTOs for everyone to use. Come
and /join #gentoo once in a while and help recover the systems of users
who emerged 0v3rl4y/can-get-gory/package-9999-r1337, as described at
http://forums.g.o/arblegarble. It's fun!


Kind regards,
     JeR
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Chris Bainbridge]

On 23/03/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> On Thu, 2006-03-23 at 14:41 +0000, Stuart Herbert wrote:
> > Your nightmare scenario seems unavoidable.  Enabling per-overlay bug
> > tracking doesn't stop users posting bugs in bugzilla.  It just causes
> > confusion for users, because they're not sure where to go.  Normally,
> > it's not a problem - because the overlay contributors are normally the
> > owners of the real package.
>
> No, it does not stop them, but it sure will curb the number of users
> posting their bugs to the wrong place.  Remember that only more advanced
> users are the ones using overlays.  We won't have Joe Sixpack using an
> overlay.  Instead it'll be Bob Developer-to-be.

If the software a user wants is in an overlay, then the user will be
forced to install the overlay.

Another thing that some people may not have considered - with many
developers using various permutations of overlays, how can you
guarantee that what is being checked into the main tree will build for
a normal user? In order to test that, a developer would have to
disable all overlays, unemerge everything provided by the overlays,
and then build and test with a plain "non-overlay" gentoo. That's a
lot of work; I doubt most developers are doing it.

There is a discussion on the forums at the moment along a similar
topic http://forums.gentoo.org/viewtopic-t-443469.html - the vote
seems to indicate 58% of users are "not really happy with the way the
portage tree is handled".

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Martin Ehmsen]

Chris Bainbridge wrote:
> Another thing that some people may not have considered - with many
> developers using various permutations of overlays, how can you
> guarantee that what is being checked into the main tree will build for
> a normal user? In order to test that, a developer would have to
> disable all overlays, unemerge everything provided by the overlays,
> and then build and test with a plain "non-overlay" gentoo. That's a
> lot of work; I doubt most developers are doing it.

You are wrong in my case.
I have a vanilla root, which used to be just a chroot, but now i'm
trying out vmware, where I test all the changes I make.

I think many other devs test in a similar manner (on another box,
chroot, ...).

/Ehmsen
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Official overlay support

[Duncan]

Chris Bainbridge posted <623652d50603230447l4938aaeam@mail.gmail.com>,
excerpted below,  on Thu, 23 Mar 2006 12:47:13 +0000:

> Adding the ebuilds to overlays is one option, but
> then other users will be expected to find an overlay with their
> package in, and then add it to make.conf.

...  This hints at something I wasn't aware of.  Does it mean that portage
already supports remote overlays, using them as it would a local overlay
only over HTTP/FTP/RSYNC, the the appropriate URL placed in make.conf, or
must the user manually sync the overlay to a local dir using whatever
protocol and let portage access the overlay from there?  I had assumed a
separate manual sync was necessary.  Was I assuming incorrectly?  Can it
be as simple as putting the URL in make.conf?  Wouldn't that be rather
resource intensive in terms of portage access to the remote server?  Or
was the step of acquiring a local copy of that overlay simply omitted and
a reference to that local copy, not the global instance, assumed, when
mentioning putting it in make.conf?

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

Chris Gianelloni wrote:
> I wouldn't mind seeing an actual "unstable" designation added to
> KEYWORDS.  The basic premise would be like package.mask packages where
> things can be done *within the tree* but still has the same air of "this
> might be totally busted at some point" as overlays.  Users would be very
> unlikely to run unstable on their machines.  Heck, we could even make it
> impossible to actually use unstable globally if we wanted to.

Sounds like package.mask / -* right now.

Thanks,
Donnie


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1336 bytes --]

On Thu, 2006-03-23 at 15:51 +0000, Stuart Herbert wrote:
> On 3/23/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> > I see no problem with providing these sorts of overlays to
> > bridge the gap between contributing users and developers.  I *do* see a
> > problem with simply allowing random overlays from any developer for
> > anything.
> 
> That's a reasonable point, and it's an experience I've not personally
> been on the receiving end of.
> 
> But the flip side is reasonable too.  Our governing metastructure is
> explicitly clear that:
> 
> a) Projects are not mandatory, and
> b) That competing projects are allowed
> 
> I don't see how we can limit overlays to just packages "owned" by the
> developers who own the overlay without violating those two policies.

Think about it this way, what if we had two competing products in the
tree that do the same thing, with the same file names?  We would add a
blocker, no?  So what mechanism is there to ensure that there's no
"blocking" issues between an official in-tree project, and these
external overlays that are not in the tree?  With the tree, we have a
well-defined policy on this.  What policy would we use for the overlays?

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Re: Official overlay support

[Rumen Yotov]

[-- Attachment #1: Type: text/plain, Size: 1699 bytes --]

On Thursday 23 March 2006 19:16, Duncan wrote:
> Chris Bainbridge posted <623652d50603230447l4938aaeam@mail.gmail.com>,
>
> excerpted below,  on Thu, 23 Mar 2006 12:47:13 +0000:
> > Adding the ebuilds to overlays is one option, but
> > then other users will be expected to find an overlay with their
> > package in, and then add it to make.conf.
>
> ...  This hints at something I wasn't aware of.  Does it mean that portage
> already supports remote overlays, using them as it would a local overlay
> only over HTTP/FTP/RSYNC, the the appropriate URL placed in make.conf, or
> must the user manually sync the overlay to a local dir using whatever
> protocol and let portage access the overlay from there?  I had assumed a
> separate manual sync was necessary.  Was I assuming incorrectly?  Can it
> be as simple as putting the URL in make.conf?  Wouldn't that be rather
> resource intensive in terms of portage access to the remote server?  Or
> was the step of acquiring a local copy of that overlay simply omitted and
> a reference to that local copy, not the global instance, assumed, when
> mentioning putting it in make.conf?
>
> --
> Duncan - List replies preferred.   No HTML msgs.
> "Every nonfree program has a lord, a master --
> and if you use the program, he is your master."  Richard Stallman in
> http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html
Hi,
Using a remote overlays is rather simple, just do "emerge layman".
Read the einfo and then "man layman".
It works flawlessly, just tested this with one remote overlay.
Beside that "man layman" explains pretty much of it's innerwork.
PS:There's an article in "gentoo-wiki.com" with a list of overlays.
HTH.Rumen

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Official overlay support

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2457 bytes --]

On Thu, 2006-03-23 at 16:40 +0000, Chris Bainbridge wrote:
> On 23/03/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> > On Thu, 2006-03-23 at 14:41 +0000, Stuart Herbert wrote:
> > > Your nightmare scenario seems unavoidable.  Enabling per-overlay bug
> > > tracking doesn't stop users posting bugs in bugzilla.  It just causes
> > > confusion for users, because they're not sure where to go.  Normally,
> > > it's not a problem - because the overlay contributors are normally the
> > > owners of the real package.
> >
> > No, it does not stop them, but it sure will curb the number of users
> > posting their bugs to the wrong place.  Remember that only more advanced
> > users are the ones using overlays.  We won't have Joe Sixpack using an
> > overlay.  Instead it'll be Bob Developer-to-be.
> 
> If the software a user wants is in an overlay, then the user will be
> forced to install the overlay.

It shouldn't be in the overlay, is I think the point many are trying to
make.  If the software is good enough for any of our users, it should be
good enough for the tree.

Yes, I know that this isn't a realistic stance, but we can't go around
thinking that overlays won't negatively impact the tree, either.

> Another thing that some people may not have considered - with many
> developers using various permutations of overlays, how can you
> guarantee that what is being checked into the main tree will build for
> a normal user? In order to test that, a developer would have to
> disable all overlays, unemerge everything provided by the overlays,
> and then build and test with a plain "non-overlay" gentoo. That's a
> lot of work; I doubt most developers are doing it.

Developers should be doing testing in a chroot, really.  I'll definitely
agree with you that many do not.

> There is a discussion on the forums at the moment along a similar
> topic http://forums.gentoo.org/viewtopic-t-443469.html - the vote
> seems to indicate 58% of users are "not really happy with the way the
> portage tree is handled".

No.  It indicates nothing except that 58% of the 80 people who filled
out the poll are "not really happy with the way the portage tree is
handled" which by my counts, isn't even a drop in the bucket of our
number of users, making the statistic completely worthless.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Stefan Schweizer]

On 3/23/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> Think about it this way, what if we had two competing products in the
> tree that do the same thing, with the same file names?  We would add a
> blocker, no?  So what mechanism is there to ensure that there's no
> "blocking" issues between an official in-tree project, and these
> external overlays that are not in the tree?  With the tree, we have a
> well-defined policy on this.  What policy would we use for the overlays?


What about if we just skip your "policies" and let the overlays be a
free place where people can handle issues how they think it is right
for the specific case and not how $super_dev said somewhere. That is
what overlays are about, not?

- Stefan

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 609 bytes --]

On Thu, 23 Mar 2006 19:31:24 +0100 "Stefan Schweizer"
<genstef@gentoo.org> wrote:
| What about if we just skip your "policies" and let the overlays be a
| free place where people can handle issues how they think it is right
| for the specific case and not how $super_dev said somewhere. That is
| what overlays are about, not?

Sounds like a perfect way to break lots and lots of systems. Those
policies are mostly there for good reason.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Re: Official overlay support

[Chris Bainbridge]

On 23/03/06, Rumen Yotov <rumen@qrypto.org> wrote:
> Hi,
> Using a remote overlays is rather simple, just do "emerge layman".
> Read the einfo and then "man layman".
> It works flawlessly, just tested this with one remote overlay.
> Beside that "man layman" explains pretty much of it's innerwork.
> PS:There's an article in "gentoo-wiki.com" with a list of overlays.
> HTH.Rumen

What is the status of those overlays? I believe the php, webapps, and
java ones (at least) are official (in that they're run by gentoo devs)
and bugs are to be reported to bugs.g.o, no?  But the wiki page
http://gentoo-wiki.com/Portage_Overlay_Listing says "NEVER report bugs
at bugs.gentoo.org for these ebuilds." So where should users report
bugs? And how are they to know that?

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Chris Bainbridge]

On 23/03/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> On Thu, 2006-03-23 at 16:40 +0000, Chris Bainbridge wrote:
> > If the software a user wants is in an overlay, then the user will be
> > forced to install the overlay.
>
> It shouldn't be in the overlay, is I think the point many are trying to
> make.  If the software is good enough for any of our users, it should be
> good enough for the tree.

I agree. I would ask, what are the advantages of overlays that
developers find so compelling that they use them rather than the
portage tree? Would it not be a better idea to find a way to bring
those advantages to the tree, rather the proliferation of overlays we
are seeing?

> > There is a discussion on the forums at the moment along a similar
> > topic http://forums.gentoo.org/viewtopic-t-443469.html - the vote
> > seems to indicate 58% of users are "not really happy with the way the
> > portage tree is handled".
>
> No.  It indicates nothing except that 58% of the 80 people who filled
> out the poll are "not really happy with the way the portage tree is
> handled" which by my counts, isn't even a drop in the bucket of our
> number of users, making the statistic completely worthless.

True. Nevertheless, it is the only statistic I have seen regarding
users thoughts on this subject. Of course a larger sample size would
be preferable. There should be more feedback from users - bug voting
on bugzilla would be a start, why has it never been enabled?

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1511 bytes --]

On Thu, 2006-03-23 at 19:31 +0100, Stefan Schweizer wrote:
> On 3/23/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> > Think about it this way, what if we had two competing products in the
> > tree that do the same thing, with the same file names?  We would add a
> > blocker, no?  So what mechanism is there to ensure that there's no
> > "blocking" issues between an official in-tree project, and these
> > external overlays that are not in the tree?  With the tree, we have a
> > well-defined policy on this.  What policy would we use for the overlays?
> 
> 
> What about if we just skip your "policies" and let the overlays be a
> free place where people can handle issues how they think it is right
> for the specific case and not how $super_dev said somewhere. That is
> what overlays are about, not?

I'm fine with that, so long as we keep them *far* from *any* Gentoo
infrastructure.  Once it hits *.gentoo.org then it needs to follow some
basic policy.  Otherwise, it is allowing anyone to completely bypass any
policies we have and allows anyone to cause any kind of breakages that
they want, with exactly 0 repercussions.

I'm sorry, but I am not OK with just standing by and watching us give
complete access to do anything with no accountability.  If you are,
perhaps you really need to rethink your commitment to our users and your
fellow developers.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Jakub Moc]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

Ciaran McCreesh wrote:
> On Thu, 23 Mar 2006 19:31:24 +0100 "Stefan Schweizer"
> <genstef@gentoo.org> wrote:
> | What about if we just skip your "policies" and let the overlays be a
> | free place where people can handle issues how they think it is right
> | for the specific case and not how $super_dev said somewhere. That is
> | what overlays are about, not?
> 
> Sounds like a perfect way to break lots and lots of systems. Those
> policies are mostly there for good reason.

You want to apply policies on overlays? Well no - sorry, overlays are
none of QA's or any other policy business. They are overlays, not
official tree.  If user installs ebuilds from overlay and breaks his
system, then well - not a Gentoo problem. Why should any policies apply?


-- 
Best regards,

 Jakub Moc
 mailto:jakub@gentoo.org
 GPG signature:
 http://subkeys.pgp.net:11371/pks/lookup?op=get&search=0xCEBA3D9E
 Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95  B30F 8717 D5FD CEBA 3D9E

 ... still no signature   ;)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Daniel Ostrow]

[-- Attachment #1: Type: text/plain, Size: 2079 bytes --]

On Thursday 23 March 2006 13:57, Jakub Moc wrote:
> Ciaran McCreesh wrote:
> > On Thu, 23 Mar 2006 19:31:24 +0100 "Stefan Schweizer"
> >
> > <genstef@gentoo.org> wrote:
> > | What about if we just skip your "policies" and let the overlays be a
> > | free place where people can handle issues how they think it is right
> > | for the specific case and not how $super_dev said somewhere. That is
> > | what overlays are about, not?
> >
> > Sounds like a perfect way to break lots and lots of systems. Those
> > policies are mostly there for good reason.
>
> You want to apply policies on overlays? Well no - sorry, overlays are
> none of QA's or any other policy business. They are overlays, not
> official tree.  If user installs ebuilds from overlay and breaks his
> system, then well - not a Gentoo problem. Why should any policies apply?

That is not what Stuart said, he indicated that overlays would be treated as 
supported systems including the use of our bugzilla system to track defects. 
If that is the case it crosses the line into the land of the "official" in 
which case policys start to be applied. While I agree that it would be very 
useful to have any overlays centrally located I do find the term "Official 
overlay" to be insanely oxymoronic in your use of the term overlay, which I 
gather to be "a set of ebuilds not bound by the QA and/or security 
requirements of the portage tree".

If I understand correctly what you are pushing then is a Official overlay 
respository for Unofficial overlays....

You can't have it both ways, either they are wholey Unofficial and do not get 
tracked in bugzilla at all (something which would have to be made VERY clear 
to our users, e.g. a you use it you get to keep the pieces policy, and the 
developer or team in question is the *only* point of contact for fixing 
things) -or- it is an Official overlay with official support which means it 
needs to abide by the rules...

-- 
Daniel Ostrow
Gentoo Foundation Board of Trustees
Gentoo/{PPC,PPC64,DevRel}
dostrow@gentoo.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Duncan Coutts]

On Thu, 2006-03-23 at 13:55 -0500, Chris Gianelloni wrote:

> I'm sorry, but I am not OK with just standing by and watching us give
> complete access to do anything with no accountability.  If you are,
> perhaps you really need to rethink your commitment to our users and your
> fellow developers.

The way the Haskell team manages this is that we don't tell our end
users about our testing overlay. So we don't get bug reports from them.
We have three outside contributers with write access to the overlay
repo. They make changes in consultation with the team. So we're not
giving complete access without accountability.

We have a couple other users who use the overlay but they know what
they're doing. We don't make the overlay that easy to use on purpose
because we don't want inexperienced users using it. So apart from not
advertising it, we don't keep digests in the repo.

I think the point is that these overlays should be a useful way of
getting contributers more closely involved. However we should not
encourage end users to use these overlays without thinking. For example
using more than one at once seems like a really bad idea. Perhaps if we
make them sufficiently hard to use then end users will not use them and
we'll just get the contributers we want.

-- 
Duncan Coutts : Gentoo Developer (Haskell herd team lead)
email         : dcoutts at gentoo dot org

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Stefan Schweizer]

On 3/23/06, Daniel Ostrow <dostrow@gentoo.org> wrote:
> You can't have it both ways, either they are wholey Unofficial and do not get
> tracked in bugzilla at all (something which would have to be made VERY clear
> to our users, e.g. a you use it you get to keep the pieces policy, and the
> developer or team in question is the *only* point of contact for fixing
> things) -or- it is an Official overlay with official support which means it
> needs to abide by the rules...

I think we need both: An aggregation of unofficial and clearly stated
unofficial overlays which are currently in place.
And an official gentoo user and developer overlay, where ebuilds have
to conform to policy.
The difference between official and unofficial: bugs can be on
bugs.gentoo.org or not.

Then both goals would be met: Access to a gentoo-overlay for
non-developers and an aggregation of all gentoo overlays out there.

- Stefan

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Official overlay support

[Rumen Yotov]

[-- Attachment #1: Type: text/plain, Size: 1785 bytes --]

On Thursday 23 March 2006 20:43, Chris Bainbridge wrote:
> On 23/03/06, Rumen Yotov <rumen@qrypto.org> wrote:
> > Hi,
> > Using a remote overlays is rather simple, just do "emerge layman".
> > Read the einfo and then "man layman".
> > It works flawlessly, just tested this with one remote overlay.
> > Beside that "man layman" explains pretty much of it's innerwork.
> > PS:There's an article in "gentoo-wiki.com" with a list of overlays.
> > HTH.Rumen
>
> What is the status of those overlays? I believe the php, webapps, and
> java ones (at least) are official (in that they're run by gentoo devs)
> and bugs are to be reported to bugs.g.o, no?  But the wiki page
> http://gentoo-wiki.com/Portage_Overlay_Listing says "NEVER report bugs
> at bugs.gentoo.org for these ebuilds." So where should users report
> bugs? And how are they to know that?
Hi,
From a very quick scan - yes there're some overlays made/supported by Gentoo 
devs and some others are hosted/maintained by users,so i think for the latter 
case you can't use Bugzilla in a sense that the ebuild is not in the tree.
But looking from another angle (POV) if this is a new version ebuild or an 
ebuild for a new (not in the tree) package, why not report it's 
status/workings/bugs in Bugzilla, generally it'll be an usefull info.
Like now when somebody files a Bug on a new version or makes and attaches 
initial ebuild for a new package.
Of course all depends on the policy (if any is made) about this external 
(official & unofficial) overlays and Bugzilla.
PS: you can yourself see/read there're big differences in opinion even between 
Gentoo devs conserning overlays. So a bug-report assigned to a dev who 
doesn't want to support overlays will probably get "WONTFIX" whatever.
Rumen

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Official overlay support

[Duncan Coutts]

On Thu, 2006-03-23 at 18:55 +0000, Chris Bainbridge wrote:
> On 23/03/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> > On Thu, 2006-03-23 at 16:40 +0000, Chris Bainbridge wrote:
> > > If the software a user wants is in an overlay, then the user will be
> > > forced to install the overlay.
> >
> > It shouldn't be in the overlay, is I think the point many are trying to
> > make.  If the software is good enough for any of our users, it should be
> > good enough for the tree.
> 
> I agree. I would ask, what are the advantages of overlays that
> developers find so compelling that they use them rather than the
> portage tree? Would it not be a better idea to find a way to bring
> those advantages to the tree, rather the proliferation of overlays we
> are seeing?

The advantages we see are:

We use it as a staging area for our herd's ebuilds. We can start with an
untested ebuild and between several team members and outside testers we
can iteratively test and refine the ebuild. This relies on a low latency
between committing changes and other devs and outside testers receiving
those changes. We have a lag of several seconds rather than 30 minutes
for the anoncvs. It means we get much higher QA before ebuilds actually
end up in portage because by the time they get there they have been
reviewed and tested by other team members and outside helpers (often
including testing on several arches).

If we did this in the cvs tree we'd need to keep the packages masked all
the time we were improving them (overhead). We'd need changelog entries
for every change (overhead). We wouldn't be able to share the
development and testing with our outside helpers (due to anoncvs lag).
And of course we wouldn't be able to grant out outside helpers write
access.

So the lower latency helps to run an AT-style system and the write
access allows for a safe intermediate stage in the recruitment process
between AT and dev status.

-- 
Duncan Coutts : Gentoo Developer (Haskell herd team lead)
email         : dcoutts at gentoo dot org

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

On 3/23/06, Daniel Ostrow <dostrow@gentoo.org> wrote:
> That is not what Stuart said, he indicated that overlays would be treated as
> supported systems including the use of our bugzilla system to track defects.
> If that is the case it crosses the line into the land of the "official" in
> which case policys start to be applied.

We do need to apply policies to the overlays.  We can't have an
absolute free-for-all.  Projects and individual developers need to be
responsible for their overlays - just like they are responsible for
the packages they add to the tree.

But we are not going to treat overlays like they're the Portage tree. 
That would undermine the whole point of having the overlays in the
first place, and would bring zero benefit to our users and our
developers.

They are different, and they require a different approach.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Jakub Moc]

[-- Attachment #1: Type: text/plain, Size: 1845 bytes --]

Duncan Coutts wrote:
 > The way the Haskell team manages this is that we don't tell our end
> users about our testing overlay. So we don't get bug reports from them.
> We have three outside contributers with write access to the overlay
> repo. They make changes in consultation with the team. So we're not
> giving complete access without accountability.

Hmmm, that kinda defeats the whole point of having an overlay at all,
IMHO. Of course you can have an overlay strictly for internal
development between a couple of people, but that's not quite what this
debate is about I'd say.

I find overlays really useful for testing stuff before it gets into
portage - that's completely pointless if the overlay is secret. Also,
you are able to find potential future devs among the overlay users,
people who actually submit fixes etc. How are you going to get that if
noone knows about the overlay?


> We have a couple other users who use the overlay but they know what
> they're doing. We don't make the overlay that easy to use on purpose
> because we don't want inexperienced users using it. So apart from not
> advertising it, we don't keep digests in the repo.

Oh well, seems like you have a very specific use for this, probably not
what most users are interested in.

> I think the point is that these overlays should be a useful way of
> getting contributers more closely involved. However we should not
> encourage end users to use these overlays without thinking. For example
> using more than one at once seems like a really bad idea. Perhaps if we
> make them sufficiently hard to use then end users will not use them and
> we'll just get the contributers we want.

As said above, how are you going to get new contributors without people
that are actually using/testing that stuff?

-- 

jakub



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Andres Loeh]

[-- Attachment #1: Type: text/plain, Size: 627 bytes --]

> As said above, how are you going to get new contributors without people
> that are actually using/testing that stuff?

We find the via Bugzilla and/or irc and point them at the overlay.
That way, we more or less know who's using the overlay and make sure
they are briefed a bit before they start using it.

dcoutts has described the current practice we use in the Haskell
team, but that doesn't necessarily mean that it's the only practice
that would work for us. I can imagine that if we can come up with
reasonable policies for o.g.o, we can switch to a slightly different
(i.e., more public) scheme ...

Cheers,
  Andres


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 1992 bytes --]

On Thursday 23 March 2006 15:54, Eric Edgar wrote:
> I personally think this is a bad idea.  I can understand and support
> links to different overlay repositories, however I do not think that
> gentoo should host or support overlays on its own infrastructure.  For
> one thing supporting overlays on our infrastructure looks like we are
> supporting broken ebuilds.  This will also lead to more confusion with
> users who find these official overlays and then the overlays conflict
> with each other and cause problems that leads to comments like well
> gentoo should just know how to fix it and make it all work.  I also
> think that this overlay structure will not provide incentives for people
> to commit to the main tree.  They will get their ebuild in an overlay
> and its hosted on gentoo and distributed to the mirrors.  At that point
> its easy for them to continue to use the overlay.  Over time the overlay
> will diverge more and more from other overlays and even the main tree.

I think that overlays are too useful to not provide. Let me give an example. 
Currently my office workstation is hosting an overlay for ebuilds for 
vmware-server. This overlay came about because I wanted to keep it in an 
overlay/svn repos for myself. This is much preferable than downloading 10+ 
files from a bugzilla bugreport. As I thought others might appreciate it, I 
posted the link at the bug. Another dev requested that the contributor 
(trainee dev actually) get access. I saw no problem with that, so agreed. 
This setup works quite satisfactory. As the repos is special purpose, I have 
no doubt that the ebuilds will finally end up in the tree. Currently the 
software and ebuilds are only beta though so would have to be hard masked.

From this experience I agree with Stuart that it can create a good bridge 
between gentoo and "trusted users".

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Official overlay support

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 1929 bytes --]

On Thursday 23 March 2006 16:31, Chris Gianelloni wrote:
> No.  It isn't.  Look in many developer overlays and you'll see packages
> that they have made that work how *they* want them to, even if it is
> *very* different from what is in the tree.  This is the case for
> packages that are not maintained by them, too.  Any ebuild that is done
> by someone that isn't the maintainer is a fork.  There's nothing
> "hostile" about it.
>
Probably this is the reason that my personal overlay is not "public". For 
example it contains a gtk+ version with the save dialog "fixed".

> I see no problem with overlays in concept, such as the php overlay that
> is very successful.  The main reason that it is successful is because
> the same people that maintain php maintain the overlay.  Yes, there are
> other contributors, but the maintainers of the overlay are still the
> developers.  I see no problem with providing these sorts of overlays to
> bridge the gap between contributing users and developers.  I *do* see a
> problem with simply allowing random overlays from any developer for
> anything.

On the other hand, my personal overlay contains various fixes to packages I 
use, but who are from other developers. As I'm not the maintainer I don't 
always bother to post bugs, and find it wrong to fix it without telling the 
maintainer. Other ebuilds concern packages that are unmaintained or not 
stable enough. It also contains my own kde meta packages that select only 
that part of kde that I want.

I can only assume that other developers have similar overlays too. These 
overlays form actually a wealth of resources that are hidden away. If there 
were a semi-public overlay system in which developers could keep their 
overlays, this might help in getting this out to the public.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Official overlay support

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

Paul de Vrieze wrote:
> I can only assume that other developers have similar overlays too. These 
> overlays form actually a wealth of resources that are hidden away. If there 
> were a semi-public overlay system in which developers could keep their 
> overlays, this might help in getting this out to the public.

Heh, such a thing sort of exists in a non-standardized form at
dev.gentoo.org/~developername/overlay/ -- seems to be the way most
people go.

Donnie


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

[gentoo-dev] Re: Official overlay support

[Duncan]

Chris Bainbridge posted <623652d50603231055k43435b0dp@mail.gmail.com>,
excerpted below,  on Thu, 23 Mar 2006 18:55:15 +0000:

>> No.  It indicates nothing except that 58% of the 80 people who filled
>> out the poll are "not really happy with the way the portage tree is
>> handled" which by my counts, isn't even a drop in the bucket of our
>> number of users, making the statistic completely worthless.
> 
> True. Nevertheless, it is the only statistic I have seen regarding
> users thoughts on this subject. Of course a larger sample size would
> be preferable. There should be more feedback from users - bug voting
> on bugzilla would be a start, why has it never been enabled?

Two comments, one likely fairly obvious:

Obvious:  Both Gentoo users overall and the poll would tend to be
self-selecting, Gentoo users toward those approving Gentoo's handling of
most things -- or they'd be elsewhere, as there /are/ other choices, the
poll toward that segment of Gentoo users not so happy -- since it's widely
known that unhappy folks tend to be more vocal and more likely to be
motivated to find and vote in such a poll than folks happy with things as
they are.

Even having the URL, for instance, I'm reasonably satisfied with the
handling of the tree as it is, and the forums are out of the way, so it's
unlikely I'll find it worth my time to go over there and vote.

The other comment, answering your bug voting question:

>From past threads where bug voting has been discussed, it appears a fairly
vocal segment of developers oppose it.  The reason given is that users
don't necessarily have any sense of global priority (a data-loss
situation on sparc, for instance, with maybe two users affected, vs an
enhancement with a hundred votes that'll take weeks to implement and
test), ease of solution, or knowledge of pending security vulns currently
embargoed, for example, and how they affect developer's priorities.
Neither are they likely to know how many other packages a developer may be
responsible for, or what other factors an after-all volunteer developer
may be balancing in their real-life.  It'll be all too common, they are
afraid, for users to ask why that bug with 100 votes seems to be being
ignored, when all they see is 1 vote from the sparc guy on the bug getting
all the action.

Other developers in favor of bug voting have countered that a developer
can ignore votes if they want, that its only an indicator of interest that
need not be used if it doesn't fit the need.

The counter to that is that the CC lists are already a decent indicator of
interest, for those who want it, without implying the same obligation of
the dev to prioritize bugs with the most votes.

In any case, no consensus in favor large enough to turn on the feature has
ever seemed to emerge, with a number of devs strongly and vocally opposed.

I believe that's a fair summation of the arguments.  My personal opinion,
for whatever it's worth as a user on the dev list, is that the CC point is
a valid one, the CC list should be a pretty decent measure of interest --
I know it has certainly proven so on some of the bugs I've CCed myself to,
as I've seen others pile on too.  Take a look at the CC list on this
(resolved) one on AMD64 performance patches to glibc, for instance:
http://bugs.gentoo.org/show_bug.cgi?id=100289 .  How could voting make
the interest any more evident than all those CCs, and the CCs actually
continue to be useful after the "vote" has been registered, as well.  Bug
voting?  I'd argue we already have bug voting!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Re: Official overlay support

[Duncan]

Rumen Yotov posted <200603232020.37728.rumen@qrypto.org>, excerpted below,
 on Thu, 23 Mar 2006 20:20:30 +0200:

> Using a remote overlays is rather simple, just do "emerge layman".
> Read the einfo and then "man layman".
> It works flawlessly, just tested this with one remote overlay.
> Beside that "man layman" explains pretty much of it's innerwork.
> PS:There's an article in "gentoo-wiki.com" with a list of overlays.

Thanks.  That's now double-underlined on my TODO list.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Official overlay support

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 445 bytes --]

Duncan wrote:
> I believe that's a fair summation of the arguments.  My personal opinion,
> for whatever it's worth as a user on the dev list, is that the CC point is
> a valid one, the CC list should be a pretty decent measure of interest --
> I know it has certainly proven so on some of the bugs I've CCed myself to,
> as I've seen others pile on too.

How can I sort my open bug list by bugs with the most CC's?

Thanks,
Donnie


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 1897 bytes --]

On Thursday 23 March 2006 19:55, Chris Bainbridge wrote:
> I agree. I would ask, what are the advantages of overlays that
> developers find so compelling that they use them rather than the
> portage tree? Would it not be a better idea to find a way to bring
> those advantages to the tree, rather the proliferation of overlays we
> are seeing?

Well. First of all I use a personal overlay to keep all things not valuable or 
suited for the tree. The overlay repos allows to share them on the 4 systems 
I maintain. The second reason for overlays is to test things in unison which 
is not ready for the tree yet (not even masked).

> > No.  It indicates nothing except that 58% of the 80 people who filled
> > out the poll are "not really happy with the way the portage tree is
> > handled" which by my counts, isn't even a drop in the bucket of our
> > number of users, making the statistic completely worthless.
>
> True. Nevertheless, it is the only statistic I have seen regarding
> users thoughts on this subject. Of course a larger sample size would
> be preferable. There should be more feedback from users - bug voting
> on bugzilla would be a start, why has it never been enabled?

You know "there is lies, damn lies, and statistics". In this case the poll 
only says that of the 80 (small sample) respondents, 58% of people disliked 
the handling of the tree. There are three problems with it. First the sample 
is not random at all (only forum users, only users feeling strong enough to 
respond). Second the sample is fairly small in relation to the user base. 
Finally, the question is suggestive, so biassed to disliking of the tree. The 
8% could very well already be caused by suggestive questioning, not to speak 
about the sample selection.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Re: Official overlay support

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 934 bytes --]

On Thursday 23 March 2006 22:49, Donnie Berkholz wrote:
> Duncan wrote:
> > I believe that's a fair summation of the arguments.  My personal opinion,
> > for whatever it's worth as a user on the dev list, is that the CC point
> > is a valid one, the CC list should be a pretty decent measure of interest
> > -- I know it has certainly proven so on some of the bugs I've CCed myself
> > to, as I've seen others pile on too.
>
> How can I sort my open bug list by bugs with the most CC's?

This I don't know. Perhaps this would be a feature enhancement. CC's do 
however have one other property that distinguishes them from votes. People 
who are cc-ed to a bug must accept getting mail on it. Votes can be cast and 
forgotten about. CC's keep asking for attention. You don't add a cc unless 
you're interested.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Re: Making the developer community more open

[Daniel Goller]

[-- Attachment #1: Type: text/plain, Size: 3186 bytes --]

On Wed, 2006-03-22 at 09:15 -0500, Dan Meltzer wrote:
> Asking developers to "proxy" takes almost as much time as it does to
> ask them to maintain a package by themselves. 

wrong

>  The developer is
> directly responsible for anything he commits, so he will have to still
> test the ebuild, still test any revisions, and still follow the
> package to make sure there are no problems.  The writing the ebuild
> part of the process is not that much of the commitment, I don't see
> the point.
> 

we are not just talking about new ebuilds/bumps
having someone do all the work and having to only verify the end results
of the users work is a big help, instead of having to look into the
problem, checking if a fix exists elsewhere, or digging through the
source yourself, you verify the fix solves the problem and does only
that.

and everyone wins

> On 3/22/06, Thomas Cort <linuxgeek@gmail.com> wrote:
> > > > A developer could then take these ebuilds, make sure they
> > > > don't do anything malicious, or break QA, or whatever, and act as the
> > > > bridge between the portage tree and the users actually working on the
> > > > ebuild and keeping things up to date and working.
> >
> > > The easiest way to handle "contrib" as far as that "big warning" is to
> > > make it a separate tree.  That way, folks who want the flexibility get
> > > it, but those who prefer not to "risk it", don't  have to worry about it.
> > > As well, contribs becomes another fertile developer recruitment ground.
> >
> > Why would the packages need a "big warning"/overlay/eclass if they
> > were checked by a developer to make sure they "don't do anything
> > malicious, or break QA, or whatever"? There are many user contributed
> > ebuilds that have made their way into portage after being reviewed by
> > devs that don't have any such warnings.
> >
> > I don't think creating a "contrib" overlay as an official part of
> > Gentoo would be a good idea because making it an official Gentoo
> > project conveys a certain level of quality. If the quality is there,
> > then why not add the ebuilds to portage in the first place? If the
> > quality isn't there, then you will have a lot of unhappy users
> > complaining that an official Gentoo overlay broke their system.
> >
> > Having a non-Gentoo sponsored contrib overlay wouldn't be a good idea
> > either IMO because the contributors wouldn't be contributing to
> > Gentoo, and they wouldn't be interacting as much with the Gentoo
> > developer community. Sure they would learn a lot of the skills
> > required to be a Gentoo developer, but they wouldn't be increasing the
> > value of anything in portage (unless they got a proxy to commit some
> > of their work to portage). Also, there are many overlays out there
> > already. Adding another one won't help with "making the developer
> > community more open". Additionally, I don't personally know of a lot
> > of people who actually use third party overlays except to get an
> > ebuild for a particular package they want or to beta test ebuilds.
> >
> > -Thomas
> >
> > --
> > gentoo-dev@gentoo.org mailing list
> >
> >
> 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Aron Griffis]

[-- Attachment #1: Type: text/plain, Size: 574 bytes --]

Ciaran McCreesh wrote:  [Wed Mar 22 2006, 12:33:10PM EST]
> On Wed, 22 Mar 2006 09:03:38 -0800 Donnie Berkholz
> <spyderous@gentoo.org> wrote:
> | This definitely sounds like a fun idea. It would be even cooler if we
> | were using a distributed SCM on both ends that allowed for easy
> | merging.
> 
> Word of warning to anyone planning to implement one of these that
> includes rsync with cache as a frontend: you will be giving root access
> to your box to any user with commit access.

Could you give some explanation or reference for this?

Thanks,
Aron

[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Aron Griffis]

[-- Attachment #1: Type: text/plain, Size: 2149 bytes --]

Chris Gianelloni wrote:  [Thu Mar 23 2006, 09:41:25AM EST]
> On Thu, 2006-03-23 at 10:09 +0000, Chris Bainbridge wrote:
> > Reduced responsibility for package QA  (I expect "we don't care about
> > overlays" to become a standard response on bugs.g.o)
> 
> You will *definitely* get this from developers that won't be using the
> overlays.
> 
> Let's just say you decide to use a toolchain overlay and it exposes some
> problem in random app foo because you're using gcc 5.1.99 and we only
> have 4.0 in the tree.  You file a bug against package foo without a
> patch.  I'm the maintainer.  You've now made me spend my time supporting
> something that isn't even in the tree, and could be an artifact of the
> overlay itself and something that will *never* end up in the tree.  Why
> should I do this?  What we have done here is actually *reduced* the
> amount of productive work that I can do by forcing me to deal with these
> overlays, even if I choose not to participate.

Some of this could be mitigated with some additional or modified
tools.  For example, emerge --info could be augmented to take
a package argument and list the installed dependency tree for that
package.  The list could also include *where* the package and deps
came from, PORTDIR or an overlay.

The result would be required information in a bug report, similar to
the existing emerge --info requirement.  So if I were submitting
a report about keychain, I would be required to include the result of

    emerge --info keychain

It becomes a lot easier for devs to determine that a problem might be
due to an overlay, then take whatever action they prefer based on that
information.  For some devs, the fact that gcc-5.1.99 breaks their
package might be a welcome early warning.

Another possible enhancement would be to include a checkbox in the bug
report to indicate whether overlays are in use.  Hopefully checked by
the reporter, but alternatively auto-detected by emerge --info in
comment #1, or checked by our ever-vigilant wranglers.  This would
make winnowing of overlay-caused bugs easier.

Just some thoughts...

Aron

[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Re: Making the developer community more open

[Dan Meltzer]

On 3/23/06, Daniel Goller <morfic@gentoo.org> wrote:
> On Wed, 2006-03-22 at 09:15 -0500, Dan Meltzer wrote:
> > Asking developers to "proxy" takes almost as much time as it does to
> > ask them to maintain a package by themselves.
>
> wrong
>
> >  The developer is
> > directly responsible for anything he commits, so he will have to still
> > test the ebuild, still test any revisions, and still follow the
> > package to make sure there are no problems.  The writing the ebuild
> > part of the process is not that much of the commitment, I don't see
> > the point.
> >
>
> we are not just talking about new ebuilds/bumps
> having someone do all the work and having to only verify the end results
> of the users work is a big help, instead of having to look into the
> problem, checking if a fix exists elsewhere, or digging through the
> source yourself, you verify the fix solves the problem and does only
> that.
>
> and everyone wins

So it sounds like you are asking them to do everything developers do,

why not just make them be developers?
>
> > On 3/22/06, Thomas Cort <linuxgeek@gmail.com> wrote:
> > > > > A developer could then take these ebuilds, make sure they
> > > > > don't do anything malicious, or break QA, or whatever, and act as the
> > > > > bridge between the portage tree and the users actually working on the
> > > > > ebuild and keeping things up to date and working.
> > >
> > > > The easiest way to handle "contrib" as far as that "big warning" is to
> > > > make it a separate tree.  That way, folks who want the flexibility get
> > > > it, but those who prefer not to "risk it", don't  have to worry about it.
> > > > As well, contribs becomes another fertile developer recruitment ground.
> > >
> > > Why would the packages need a "big warning"/overlay/eclass if they
> > > were checked by a developer to make sure they "don't do anything
> > > malicious, or break QA, or whatever"? There are many user contributed
> > > ebuilds that have made their way into portage after being reviewed by
> > > devs that don't have any such warnings.
> > >
> > > I don't think creating a "contrib" overlay as an official part of
> > > Gentoo would be a good idea because making it an official Gentoo
> > > project conveys a certain level of quality. If the quality is there,
> > > then why not add the ebuilds to portage in the first place? If the
> > > quality isn't there, then you will have a lot of unhappy users
> > > complaining that an official Gentoo overlay broke their system.
> > >
> > > Having a non-Gentoo sponsored contrib overlay wouldn't be a good idea
> > > either IMO because the contributors wouldn't be contributing to
> > > Gentoo, and they wouldn't be interacting as much with the Gentoo
> > > developer community. Sure they would learn a lot of the skills
> > > required to be a Gentoo developer, but they wouldn't be increasing the
> > > value of anything in portage (unless they got a proxy to commit some
> > > of their work to portage). Also, there are many overlays out there
> > > already. Adding another one won't help with "making the developer
> > > community more open". Additionally, I don't personally know of a lot
> > > of people who actually use third party overlays except to get an
> > > ebuild for a particular package they want or to beta test ebuilds.
> > >
> > > -Thomas
> > >
> > > --
> > > gentoo-dev@gentoo.org mailing list
> > >
> > >
> >
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2.1 (GNU/Linux)
>
> iD8DBQBEIx8o/aM9DdBw91cRAmVoAKC8JtAm2vvWGBG2YMzpI+EGu8RFJwCeOMll
> lCv/CsLde+6MbDHgX8EuKhU=
> =w+ap
> -----END PGP SIGNATURE-----
>
>
>

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

On Thu, 23 Mar 2006 19:57:07 +0100 Jakub Moc <jakub@gentoo.org> wrote:
| > Sounds like a perfect way to break lots and lots of systems. Those
| > policies are mostly there for good reason.
| 
| You want to apply policies on overlays? Well no - sorry, overlays are
| none of QA's or any other policy business. They are overlays, not
| official tree.  If user installs ebuilds from overlay and breaks his
| system, then well - not a Gentoo problem. Why should any policies
| apply?

It is a Gentoo problem, because Gentoo gets innundated with bogus bug
reports when users screw up their systems in weird ways and don't
realise the cause.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Re: Making the developer community more open

[Daniel Goller]

[-- Attachment #1: Type: text/plain, Size: 1739 bytes --]

On Thu, 2006-03-23 at 18:34 -0500, Dan Meltzer wrote:
> On 3/23/06, Daniel Goller <morfic@gentoo.org> wrote:
> > On Wed, 2006-03-22 at 09:15 -0500, Dan Meltzer wrote:
> > > Asking developers to "proxy" takes almost as much time as it does to
> > > ask them to maintain a package by themselves.
> >
> > wrong
> >
> > >  The developer is
> > > directly responsible for anything he commits, so he will have to still
> > > test the ebuild, still test any revisions, and still follow the
> > > package to make sure there are no problems.  The writing the ebuild
> > > part of the process is not that much of the commitment, I don't see
> > > the point.
> > >
> >
> > we are not just talking about new ebuilds/bumps
> > having someone do all the work and having to only verify the end results
> > of the users work is a big help, instead of having to look into the
> > problem, checking if a fix exists elsewhere, or digging through the
> > source yourself, you verify the fix solves the problem and does only
> > that.
> >
> > and everyone wins
> 
> So it sounds like you are asking them to do everything developers do,
> 
> why not just make them be developers?
> >

because they do not want more than take care of their one package and do
not want more than someone taking care of their work, maybe?
or maybe because they don't like a lot of devs due to bad experiences,
and would not want to be any part of the group, but still enjoy working
on packages

there are people who i would love to see as devs, but are not
particularly interested on having to deal with some of the devs, having
a proxy allows them to help gentoo w/o being exposed to the people they
otherwise would have to deal with




[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 771 bytes --]

On Thursday 23 March 2006 22:32, Donnie Berkholz wrote:
> Paul de Vrieze wrote:
> > I can only assume that other developers have similar overlays too.
> > These overlays form actually a wealth of resources that are hidden
> > away. If there were a semi-public overlay system in which developers
> > could keep their overlays, this might help in getting this out to the
> > public.
>
> Heh, such a thing sort of exists in a non-standardized form at
> dev.gentoo.org/~developername/overlay/ -- seems to be the way most
> people go.

Would be easy though if it were a subversion repository. That way one does 
not need to copy it around all the time.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

Hi Andres,

On 3/23/06, Andres Loeh <kosmikus@gentoo.org> wrote:
> dcoutts has described the current practice we use in the Haskell
> team, but that doesn't necessarily mean that it's the only practice
> that would work for us. I can imagine that if we can come up with
> reasonable policies for o.g.o, we can switch to a slightly different
> (i.e., more public) scheme ...

I want to thank both you and Duncan for your explaination of how the
Haskell overlay works.  I would love to get your overlay onto
overlays.g.o, without disrupting the working practices that you've
found successful.

Which means we need to take another look at the vision of all overlays
being publically readable, because having a non-public overlay seems
to be a key part of what you're already doing.

I really don't want o.g.o to carry 'secret' overlays.  But maybe
there's a middle ground that fits with what you need?

If the overlay's changelog is included on o.g.o's front-page, and the
wiki / GuideXML site is publically readable, but we just disallow
anonymous access to the overlay itself (only if requested, this
wouldn't be the default setup) ... how would that work for you?

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

> It is a Gentoo problem, because Gentoo gets innundated with bogus bug
> reports when users screw up their systems in weird ways and don't
> realise the cause.

Convince me that this is something more than just a power play, and
I'll work with you.  But that's the hurdle you'll need to overcome
first.

Second hurdle is that you need to convince me that you "get" what the
overlays are there for.  If you can't, then I can have no confidence
that any policies you bring forward are appropriate for the work we're
trying to enable.

Thrid hurdle is that you need to convince me that you're capable of
treating the overlays differently to how the main tree is treated.  If
you can't, then I'll feel that you hoodwinked me at the second hurdle.

I'm sure you've got a lot to offer, to help make the overlays a
success.  But your agenda has to be appropriate - otherwise you'll
just do more damage that good.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Jakub Moc]

[-- Attachment #1: Type: text/plain, Size: 2163 bytes --]

Ciaran McCreesh wrote:
> On Thu, 23 Mar 2006 19:57:07 +0100 Jakub Moc <jakub@gentoo.org> wrote:
> | > Sounds like a perfect way to break lots and lots of systems. Those
> | > policies are mostly there for good reason.
> | 
> | You want to apply policies on overlays? Well no - sorry, overlays are
> | none of QA's or any other policy business. They are overlays, not
> | official tree.  If user installs ebuilds from overlay and breaks his
> | system, then well - not a Gentoo problem. Why should any policies
> | apply?
> 
> It is a Gentoo problem, because Gentoo gets innundated with bogus bug
> reports when users screw up their systems in weird ways and don't
> realise the cause.

We get innundated with tons of bogus bug reports every day, overlays or
not - see the number of invalid/duplicate bugs flowing every days. We
got a couple of bugs in last two a three days basically stating "ZOMG,
glibc downgrade broke my system, t3h Gentoo bug!!11!!" - so what? They
get marked as invalid, live goes on. This argument really doesn't stand.


As this should be a separate thread, just one reason or example - I'm
really uncomfortable e.g. w/ QA intervening in overlays stuff,
considering the current way QA is being done in Gentoo... Current
non-interactivity policy has clearly influenced multiple ebuilds in
portage to the extent that forced me to read the ebuilds very carefully
multiple times to be sure what the outcome will be with my particular
USE flags. This is a bad thing (TM) for me and I clearly oppose to
having such stuff forced in overlays. I could see a place for QA
volunteers in this overlay stuff, but that would require a fundamentally
different approach from what QA takes now. (If you wish to discuss this
further, move it to a separate thread, please).

Common sense always applies, but generally - overlays are not a place
for bureaucracy...

-- 
Best regards,

 Jakub Moc
 mailto:jakub@gentoo.org
 GPG signature:
 http://subkeys.pgp.net:11371/pks/lookup?op=get&search=0xCEBA3D9E
 Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95  B30F 8717 D5FD CEBA 3D9E

 ... still no signature   ;)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Andres Loeh]

[-- Attachment #1: Type: text/plain, Size: 2088 bytes --]

Hi Stuart.

> > dcoutts has described the current practice we use in the Haskell
> > team, but that doesn't necessarily mean that it's the only practice
> > that would work for us. I can imagine that if we can come up with
> > reasonable policies for o.g.o, we can switch to a slightly different
> > (i.e., more public) scheme ...
> 
> I want to thank both you and Duncan for your explaination of how the
> Haskell overlay works.  I would love to get your overlay onto
> overlays.g.o, without disrupting the working practices that you've
> found successful.
> 
> Which means we need to take another look at the vision of all overlays
> being publically readable, because having a non-public overlay seems
> to be a key part of what you're already doing.

Not really. I discussed this with Duncan and we're perfectly ok with
a publically readable repo. In fact, our overlay is currently publically
readable. It's just not very well-known or advertised.

If we change the situation, we'll have to write up a few guidelines specific
to our repository.

Here's a list of things that I think are essential or highly helpful to
our working process:

* We should be allowed to continue using darcs for our version management.
  If that's not possible on Gentoo infra, we should be allowed to host on
  another machine and just have a pointer or ChangeLog on o.g.o.

* It should be possible for us to assign commit permissions to any people
  we think are qualified without any formalities and delay.

> If the overlay's changelog is included on o.g.o's front-page, and the
> wiki / GuideXML site is publically readable, but we just disallow
> anonymous access to the overlay itself (only if requested, this
> wouldn't be the default setup) ... how would that work for you?

It would work, of course, and it would help prevent certain complaints,
but it's not absolutely necessary. If "on request" is chosen, it's also
important that read access can be given by us without any delay, i.e.,
without going through any formal process.

Cheers,
  Andres

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2011 bytes --]

On Fri, 2006-03-24 at 08:59 +0000, Stuart Herbert wrote:
> > It is a Gentoo problem, because Gentoo gets innundated with bogus bug
> > reports when users screw up their systems in weird ways and don't
> > realise the cause.
> 
> Convince me that this is something more than just a power play, and
> I'll work with you.  But that's the hurdle you'll need to overcome
> first.

Perhaps because he's not the only one saying it?

Really, people, can we leave our personal bullshit out of a technical
discussion *just once*?

> Second hurdle is that you need to convince me that you "get" what the
> overlays are there for.  If you can't, then I can have no confidence
> that any policies you bring forward are appropriate for the work we're
> trying to enable.

They are there to speed development and to allow users to contribute
more directly.  They should not be readable by the public, otherwise we
run into the problem of users that don't know what they are doing and
followed some half-way written guide on the forums using ebuilds from
these overlays, breaking their systems, and filing bugs.  This is *not*
acceptable.  I see no problems with allowing users to gain read and even
write access to overlays, but it must be done with a certain level of
caution of the main tree, or you'll have a very hard time getting
support from the developer community at large.

> Thrid hurdle is that you need to convince me that you're capable of
> treating the overlays differently to how the main tree is treated.  If
> you can't, then I'll feel that you hoodwinked me at the second hurdle.
> 
> I'm sure you've got a lot to offer, to help make the overlays a
> success.  But your agenda has to be appropriate - otherwise you'll
> just do more damage that good.

Again, try to keep this technical discussion technical and leave your
personal biases out of it.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]

On Fri, 2006-03-24 at 10:16 +0100, Jakub Moc wrote:
> As this should be a separate thread, just one reason or example - I'm
> really uncomfortable e.g. w/ QA intervening in overlays stuff,
> considering the current way QA is being done in Gentoo... Current
> non-interactivity policy has clearly influenced multiple ebuilds in
> portage to the extent that forced me to read the ebuilds very carefully
> multiple times to be sure what the outcome will be with my particular
> USE flags. This is a bad thing (TM) for me and I clearly oppose to
> having such stuff forced in overlays. I could see a place for QA
> volunteers in this overlay stuff, but that would require a fundamentally
> different approach from what QA takes now. (If you wish to discuss this
> further, move it to a separate thread, please).

Except nobody says that QA needs to intervene in an overlay.

My request is *simple*.

If the overlay is accessible to the general public, then it *cannot*
break other packages in the tree via its use.  If the overlay is
"private" (meaning it has a restricted access list, developers and users
are both welcome) then it can break whatever policy that it wants.

> Common sense always applies, but generally - overlays are not a place
> for bureaucracy...

Nor are they a place to allow a free-for-all where people can commit
anything that can cause any amount of damage to the tree, while still
being "official" and hosted on Gentoo infrastructure.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1649 bytes --]

On Fri, 2006-03-24 at 12:46 +0100, Andres Loeh wrote:
> > If the overlay's changelog is included on o.g.o's front-page, and the
> > wiki / GuideXML site is publically readable, but we just disallow
> > anonymous access to the overlay itself (only if requested, this
> > wouldn't be the default setup) ... how would that work for you?
> 
> It would work, of course, and it would help prevent certain complaints,
> but it's not absolutely necessary. If "on request" is chosen, it's also
> important that read access can be given by us without any delay, i.e.,
> without going through any formal process.

See, I have no problem with this.  The operation of the overlay *should*
lie completely with the overlay owners.  You *should* be able to add
*anyone* that you feel is worth adding to read *or* write access, with
no further process.

As I've said, my only request is a single policy that before an overlay
can become publicly readable on overlays.gentoo.org (which is Gentoo
infrastructure) that it does not break packages in the main tree that
are not in the overlay.

If this single policy were in place, then I would fully support
overlays.gentoo.org being created.

My main point is I don't want one of my tree packages to break because
some ricer told some n00b to use some crazy ebuild from some random
overlay that isn't really fit for the general masses.  If we take at
least *some* measures to prevent this, then I'm OK with it.  Allowing a
free-for-all in the overlays is not acceptable.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 558 bytes --]

On Friday 24 March 2006 14:55, Chris Gianelloni wrote:
> My main point is I don't want one of my tree packages to break because
> some ricer told some n00b to use some crazy ebuild from some random
> overlay that isn't really fit for the general masses.  If we take at
> least *some* measures to prevent this, then I'm OK with it.  Allowing a
> free-for-all in the overlays is not acceptable.

I think that this is a reasonable requirement.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Official overlay support

[Aron Griffis]

Stuart,

I like the idea of overlays but your email here is completely bogus.
Ciaran just explained why overlays are a Gentoo problem, rebutting
Jakub's assertion that there's no need for policies.  I don't see any
agenda here, so either you're pulling in external context, or you're
reading into it.

Aron

Stuart Herbert wrote:  [Fri Mar 24 2006, 03:59:54AM EST]
> > It is a Gentoo problem, because Gentoo gets innundated with bogus bug
> > reports when users screw up their systems in weird ways and don't
> > realise the cause.
> 
> Convince me that this is something more than just a power play, and
> I'll work with you.  But that's the hurdle you'll need to overcome
> first.
> 
> Second hurdle is that you need to convince me that you "get" what the
> overlays are there for.  If you can't, then I can have no confidence
> that any policies you bring forward are appropriate for the work we're
> trying to enable.
> 
> Thrid hurdle is that you need to convince me that you're capable of
> treating the overlays differently to how the main tree is treated.  If
> you can't, then I'll feel that you hoodwinked me at the second hurdle.
> 
> I'm sure you've got a lot to offer, to help make the overlays a
> success.  But your agenda has to be appropriate - otherwise you'll
> just do more damage that good.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Aron Griffis]

[-- Attachment #1: Type: text/plain, Size: 623 bytes --]

Chris Gianelloni wrote:  [Fri Mar 24 2006, 08:55:30AM EST]
> As I've said, my only request is a single policy that before an overlay
> can become publicly readable on overlays.gentoo.org (which is Gentoo
> infrastructure) that it does not break packages in the main tree that
> are not in the overlay.

This makes sense, but what's the content of such a policy?  Take your
gcc-5.1.99 example: say it "breaks" lots of packages in portage.  Is
it not allowed to be in a publicly accessible overlay in that case?
If that's not the policy, then what policy allows gcc-5.1.99 but still
covers the ground you want covered?

Aron

[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

Hi Andres,

On 3/24/06, Andres Loeh <kosmikus@gentoo.org> wrote:
> Here's a list of things that I think are essential or highly helpful to
> our working process:
>
> * We should be allowed to continue using darcs for our version management.
>  If that's not possible on Gentoo infra, we should be allowed to host on
>  another machine and just have a pointer or ChangeLog on o.g.o.

Unless I've missed it, no-one else has spoken up and suggested what
the second VCS should be.  I'll have a play with darcs as soon as I
can, and talk with infra about whether we can support it or not.  Is
it okay if I come and pick your brains to help with this?

> * It should be possible for us to assign commit permissions to any people
>  we think are qualified without any formalities and delay.

Absolutely.  This is one of the corner-stones of the project.

> It would work, of course, and it would help prevent certain complaints,
> but it's not absolutely necessary. If "on request" is chosen, it's also
> important that read access can be given by us without any delay, i.e.,
> without going through any formal process.

The only formallity is that the request will need to come from the
project lead listed on your project's page under
www.g.o/proj/<lang>/<project>/.  I need to talk to infra first, but in
principle I don't see a problem with project leads being allowed to
delegate that power.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Alec Warner]

Chris Gianelloni wrote:
> On Fri, 2006-03-24 at 08:59 +0000, Stuart Herbert wrote:
> 
>>>It is a Gentoo problem, because Gentoo gets innundated with bogus bug
>>>reports when users screw up their systems in weird ways and don't
>>>realise the cause.
>>
>>Convince me that this is something more than just a power play, and
>>I'll work with you.  But that's the hurdle you'll need to overcome
>>first.
> 
> 
> Perhaps because he's not the only one saying it?
> 
> Really, people, can we leave our personal bullshit out of a technical
> discussion *just once*?
> 
> 
>>Second hurdle is that you need to convince me that you "get" what the
>>overlays are there for.  If you can't, then I can have no confidence
>>that any policies you bring forward are appropriate for the work we're
>>trying to enable.
> 
> 
> They are there to speed development and to allow users to contribute
> more directly.  They should not be readable by the public, otherwise we
> run into the problem of users that don't know what they are doing and
> followed some half-way written guide on the forums using ebuilds from
> these overlays, breaking their systems, and filing bugs.  This is *not*
> acceptable.  I see no problems with allowing users to gain read and even
> write access to overlays, but it must be done with a certain level of
> caution of the main tree, or you'll have a very hard time getting
> support from the developer community at large.
> 

+1

At least in my mind the overlays should be developmental overlays; not
for public comsumption.  This doesn't mean "don't tell anyone about it
so that no one shows up."  It means "interested users will probably
inquire about helping out, etc...and can be granted access fairly easily."

-Alec Warner
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

Hi Chris,

On 3/24/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> As I've said, my only request is a single policy that before an overlay
> can become publicly readable on overlays.gentoo.org (which is Gentoo
> infrastructure) that it does not break packages in the main tree that
> are not in the overlay.
>
> If this single policy were in place, then I would fully support
> overlays.gentoo.org being created.

I accept the principle.  I've just one question about the practice.

Overlays will (probably more often than not) contain packages that
have problems.  For example, I'm thinking of when we developed the new
PHP packages, the packages delivered PHP in a new way, with different
USE flags.  We deliberately added a blocker, so that dev-lang/php and
dev-php/php (the version from the tree) couldn't be installed at the
same time.  This meant that, until the webapps had their DEPs updated,
if you had PHP from the overlay installed, it would block webapps from
the tree from installing (because they depended on dev-php/php).

Do you consider a practice like that to violate the policy you seek?

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

On Fri, 24 Mar 2006 10:16:15 +0100 Jakub Moc <jakub@gentoo.org> wrote:
| We get innundated with tons of bogus bug reports every day, overlays
| or not - see the number of invalid/duplicate bugs flowing every days.
| We got a couple of bugs in last two a three days basically stating
| "ZOMG, glibc downgrade broke my system, t3h Gentoo bug!!11!!" - so
| what? They get marked as invalid, live goes on. This argument really
| doesn't stand.

They get marked as invalid after how long? There're some really subtle
ways in which libraries can screw things up. I've dealt with far too
many bug reports where it took a heck of a lot of debugging before it
became clear that the cause was some dodgy external stuff. And that
was with me understanding the packages in question -- there's no way
bug wranglers could've figured it out.

| As this should be a separate thread, just one reason or example - I'm
| really uncomfortable e.g. w/ QA intervening in overlays stuff,
| considering the current way QA is being done in Gentoo...

I'm really uncomfortable with QA intervening anywhere. It would be far
nicer if the appropriate developers ensured that they weren't breaking
anything.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Andres Loeh]

[-- Attachment #1: Type: text/plain, Size: 2134 bytes --]

> > > If the overlay's changelog is included on o.g.o's front-page, and the
> > > wiki / GuideXML site is publically readable, but we just disallow
> > > anonymous access to the overlay itself (only if requested, this
> > > wouldn't be the default setup) ... how would that work for you?
> > 
> > It would work, of course, and it would help prevent certain complaints,
> > but it's not absolutely necessary. If "on request" is chosen, it's also
> > important that read access can be given by us without any delay, i.e.,
> > without going through any formal process.
> 
> See, I have no problem with this.  The operation of the overlay *should*
> lie completely with the overlay owners.  You *should* be able to add
> *anyone* that you feel is worth adding to read *or* write access, with
> no further process.

Ok.

> As I've said, my only request is a single policy that before an overlay
> can become publicly readable on overlays.gentoo.org (which is Gentoo
> infrastructure) that it does not break packages in the main tree that
> are not in the overlay.
> 
> If this single policy were in place, then I would fully support
> overlays.gentoo.org being created.

I see your point. Then I'd suggest to have o.g.o host two classes of
overlays: (A) publically readable ones that fulfill basic policies and don't
break the main tree, and (B) others where read and write access can be
granted on request by the overlay owners, but isn't available automatically.

Every overlay would belong to class B at first -- in fact, o.g.o could go
online only supporting B. We can take our time and work out goog guidelines
for class A repos, and then gradually change some of the overlays to class
A status.

> My main point is I don't want one of my tree packages to break because
> some ricer told some n00b to use some crazy ebuild from some random
> overlay that isn't really fit for the general masses.  If we take at
> least *some* measures to prevent this, then I'm OK with it.  Allowing a
> free-for-all in the overlays is not acceptable.

Yes, as I said, I understand that.

Cheers,
  Andres

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Andres Loeh]

[-- Attachment #1: Type: text/plain, Size: 1714 bytes --]

> On 3/24/06, Andres Loeh <kosmikus@gentoo.org> wrote:
> > Here's a list of things that I think are essential or highly helpful to
> > our working process:
> >
> > * We should be allowed to continue using darcs for our version management.
> >  If that's not possible on Gentoo infra, we should be allowed to host on
> >  another machine and just have a pointer or ChangeLog on o.g.o.
> 
> Unless I've missed it, no-one else has spoken up and suggested what
> the second VCS should be.  I'll have a play with darcs as soon as I
> can, and talk with infra about whether we can support it or not.  Is
> it okay if I come and pick your brains to help with this?

Sure. Best place to find any of us is #gentoo-haskell ...

> > * It should be possible for us to assign commit permissions to any people
> >  we think are qualified without any formalities and delay.
> 
> Absolutely.  This is one of the corner-stones of the project.

Great.

> > It would work, of course, and it would help prevent certain complaints,
> > but it's not absolutely necessary. If "on request" is chosen, it's also
> > important that read access can be given by us without any delay, i.e.,
> > without going through any formal process.
> 
> The only formallity is that the request will need to come from the
> project lead listed on your project's page under
> www.g.o/proj/<lang>/<project>/.  I need to talk to infra first, but in
> principle I don't see a problem with project leads being allowed to
> delegate that power.

At the moment, Haskell is only a herd and a team, not a project. But this
is certainly something that can be addressed, should it be necessary to
change that.

Cheers,
  Andres

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Official overlay support

[Jakub Moc]

[-- Attachment #1: Type: text/plain, Size: 1482 bytes --]

Chris Gianelloni wrote:
> On Fri, 2006-03-24 at 12:46 +0100, Andres Loeh wrote:

> As I've said, my only request is a single policy that before an overlay
> can become publicly readable on overlays.gentoo.org (which is Gentoo
> infrastructure) that it does not break packages in the main tree that
> are not in the overlay.
> 
> If this single policy were in place, then I would fully support
> overlays.gentoo.org being created.

How are you going to enforce that policy? Heh, you can't do that
preemptively even in the main tree. :) Every day there are bugs about
package A upgrade breaking packages B, C and D. I can also create an
overlay with two completely innocent ebuilds, get it opened and then
keep filing it w/ ricer food like patched-to-hell toolchain stuff?

How are you going to verify that ebuild X in overlay doesn't break
anything in the tree? And that its next revision won't do it. You
volunteer for such job? Great. Or not? Well, then there's no point in
suggesting a policy that can just never work...

So - what you are telling us is that you *assume* that people are going
to publish ebuilds *known* to break in-portage stuff on overlays.g.o.?
Weird assumption really... :/


-- 
Best regards,

 Jakub Moc
 mailto:jakub@gentoo.org
 GPG signature:
 http://subkeys.pgp.net:11371/pks/lookup?op=get&search=0xCEBA3D9E
 Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95  B30F 8717 D5FD CEBA 3D9E

 ... still no signature   ;)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

On 3/24/06, Ciaran McCreesh <ciaranm@gentoo.org> wrote:
> I'm really uncomfortable with QA intervening anywhere. It would be far
> nicer if the appropriate developers ensured that they weren't breaking
> anything.

+1

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Jakub Moc]

[-- Attachment #1: Type: text/plain, Size: 1421 bytes --]

Ciaran McCreesh wrote:
> On Fri, 24 Mar 2006 10:16:15 +0100 Jakub Moc <jakub@gentoo.org> wrote:
> | We get innundated with tons of bogus bug reports every day, overlays
> | or not - see the number of invalid/duplicate bugs flowing every days.
> | We got a couple of bugs in last two a three days basically stating
> | "ZOMG, glibc downgrade broke my system, t3h Gentoo bug!!11!!" - so
> | what? They get marked as invalid, live goes on. This argument really
> | doesn't stand.
> 
> They get marked as invalid after how long? There're some really subtle
> ways in which libraries can screw things up. I've dealt with far too
> many bug reports where it took a heck of a lot of debugging before it
> became clear that the cause was some dodgy external stuff. And that
> was with me understanding the packages in question -- there's no way
> bug wranglers could've figured it out.

Yeah, and the point is? It happens every day, there are already tons of
third-party overlays used by Gentoo users, but once this thread about
"official" overlays started, you came here to tell us "wow, this all
will cause terrible borkage and flood developers w/ tons of stupid
invalid bugs, we need policies"?

I really don't see how overlays run mostly by Gentoo devs would cause
any more borkage than totally uncontrolled third-party overlays run by
whomever creates and publishes them, sorry.


-- 

jakub


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

On 3/24/06, Alec Warner <antarus@gentoo.org> wrote:
> At least in my mind the overlays should be developmental overlays; not
> for public comsumption.  This doesn't mean "don't tell anyone about it
> so that no one shows up."  It means "interested users will probably
> inquire about helping out, etc...and can be granted access fairly easily."

That undermines the idea of using the overlays to attract current
non-contributors.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Andrej Kacian]

[-- Attachment #1: Type: text/plain, Size: 906 bytes --]

Dňa Fri, 24 Mar 2006 17:15:37 +0100
Jakub Moc <jakub@gentoo.org> napísal:

> Yeah, and the point is? It happens every day, there are already tons
> of third-party overlays used by Gentoo users, but once this thread
> about "official" overlays started, you came here to tell us "wow,
> this all will cause terrible borkage and flood developers w/ tons of
> stupid invalid bugs, we need policies"?
> 
> I really don't see how overlays run mostly by Gentoo devs would cause
> any more borkage than totally uncontrolled third-party overlays run by
> whomever creates and publishes them, sorry.

One of the reasons might be that while many users have enough sense not
to report bugs too eagerly when using third-party overlays of varying
quality, they would do so if they were using an official overlay -
which, with your no-policy approach, would undoubtedly crawl with bugs.

-- 
Andrej

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 278 bytes --]

Re: [gentoo-dev] Official overlay support

[Stuart Herbert]

On 3/24/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> Again, try to keep this technical discussion technical and leave your
> personal biases out of it.

It's not meant as a personal critisism of Ciaran.  Ciaran's being very
helpful in this thread.  It just happens that it was his post that
created the opportunity for me to make that comment.

I stand by the critera I stated.  They're exactly what you are doing,
and exactly what Ciaran is doing - failure avoidance.

I am *not* going to keep this just on a technical level.  (I will,
however, keep this discussion professional).  The overlays are a
social tool just as much as a technical one.  The social aspect keeps
getting ignored in this thread, or dismissed as unimportant.  That
frustrates me, I don't mind admitting.

Best regards,
Stu
--

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1683 bytes --]

On Fri, 2006-03-24 at 09:47 -0500, Aron Griffis wrote:
> Chris Gianelloni wrote:  [Fri Mar 24 2006, 08:55:30AM EST]
> > As I've said, my only request is a single policy that before an overlay
> > can become publicly readable on overlays.gentoo.org (which is Gentoo
> > infrastructure) that it does not break packages in the main tree that
> > are not in the overlay.
> 
> This makes sense, but what's the content of such a policy?  Take your
> gcc-5.1.99 example: say it "breaks" lots of packages in portage.  Is
> it not allowed to be in a publicly accessible overlay in that case?
> If that's not the policy, then what policy allows gcc-5.1.99 but still
> covers the ground you want covered?

I see your point here and honestly do not have an answer.  I don't want
to limit what people can do in the overlays so much as reduce the
"collateral damage" that can be done.  Honestly stuff like the toolchain
is a bit of an exception only because that information all shows up on
an "emerge --info" already.

I really can't think of much besides kernel + toolchain that can have
such devastating effects to the rest of the tree.  The only other
massive breakages would be via eclasses, which was my main target.

Does anyone have any ideas how we could resonably reduce problems
reported from things such as toolchain breakages in an overlay, yet
still not punish the people running the overlay by disallowing it?  I
surely wouldn't want to limit the toolchain maintainers from being able
to enjoy the use of an overlay if they wished it.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Making the developer community more open

[Daniel Drake]

George Shapovalov wrote:
> Please take a look at #1523 (note the number ;)), it addresses
> essentially this issue, or pretty similar.

Thanks, I'd never heard of that, and it's very interesting. Exactly the 
kind of thing I am looking for too, in terms of scope/crazyness :)

It was obviously planned a *long* time ago, and I have some doubts as to 
whether a system would stand now. What are you thoughts on this? How 
much of it do you think would still work in the present day?

Daniel
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: Official overlay support

[Duncan]

Chris Gianelloni posted <1143227928.17575.44.camel@cgianelloni.nuvox.net>,
excerpted below,  on Fri, 24 Mar 2006 14:18:48 -0500:

> On Fri, 2006-03-24 at 09:47 -0500, Aron Griffis wrote:
>> Chris Gianelloni wrote:  [Fri Mar 24 2006, 08:55:30AM EST]
>> > As I've said, my only request is a single policy that before an overlay
>> > can become publicly readable on overlays.gentoo.org (which is Gentoo
>> > infrastructure) that it does not break packages in the main tree that
>> > are not in the overlay.
>> 
>> This makes sense, but what's the content of such a policy?  Take your
>> gcc-5.1.99 example: say it "breaks" lots of packages in portage.  Is
>> it not allowed to be in a publicly accessible overlay in that case?
>> If that's not the policy, then what policy allows gcc-5.1.99 but still
>> covers the ground you want covered?
> 
> I see your point here and honestly do not have an answer.  I don't want
> to limit what people can do in the overlays so much as reduce the
> "collateral damage" that can be done.  Honestly stuff like the toolchain
> is a bit of an exception only because that information all shows up on
> an "emerge --info" already.

With current in-tree gccs there's another exception as well -- the
slotting and eselect/gcc-config functionality makes it very easy to switch
gccs where necessary.  It's that functionality that allowed me to test out
gcc-4.0 and 4.1 with few problems, all of which were easily resolved with
a quick gcc switch, save for one with KDE and libstdc++ that was easy
enough to work around once I was aware of it.

I'd suggest that the "no break tree" policy apply here to the extent that
gcc in an official overlay must continue to support the usual slotting and
eselect/gcc-config functionality, thus allowing a quick reconfigure to a
generally working system.

Of course, there's a wrench thrown in the next time gcc decides to go with
a seriously incompatible c++ implimentation, but I think the usual die
unless some exotic I_WANT_TO_BREAK_MY_GENTOO_SYSTEM_WITH_A_L33T_GCC
variable is set, with an appropriately dire warning in the die, covers
that eventuality.

The rest of the toolchain...  Kernel, I've never run into a problem with
support because I choose to grab my kernels directly off of kernel.org,
and I think that should continue.  Is binutils fully slotted and
binutils-config or the like yet working?

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Official overlay support

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 963 bytes --]

On Friday 24 March 2006 20:18, Chris Gianelloni wrote:

> I really can't think of much besides kernel + toolchain that can have
> such devastating effects to the rest of the tree.  The only other
> massive breakages would be via eclasses, which was my main target.

glibc is a good candidate. And portage a second one.

> Does anyone have any ideas how we could resonably reduce problems
> reported from things such as toolchain breakages in an overlay, yet
> still not punish the people running the overlay by disallowing it?  I
> surely wouldn't want to limit the toolchain maintainers from being able
> to enjoy the use of an overlay if they wished it.

Perhaps we could ask people who run overlays with dangerous ebuilds, to have 
these ebuilds protected by some environment variables. (The var must be set 
for the ebuild to work.)

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Official overlay support

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 402 bytes --]

On Friday 24 March 2006 16:41, Andres Loeh wrote:
> At the moment, Haskell is only a herd and a team, not a project. But this
> is certainly something that can be addressed, should it be necessary to
> change that.

In my regards you are a project, just not one that has a project page.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] Official overlay support

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1479 bytes --]

On Sat, Mar 25, 2006 at 08:37:24PM +0100, Paul de Vrieze wrote:
> On Friday 24 March 2006 20:18, Chris Gianelloni wrote:
> > I really can't think of much besides kernel + toolchain that can have
> > such devastating effects to the rest of the tree.  The only other
> > massive breakages would be via eclasses, which was my main target.
> glibc is a good candidate. And portage a second one.
*libc in general.
binutils
coreutils (Screwing up this is really fun, sort/xargs/tail etc.)

And a general class, the reason I've had stuff in my own overlays:
- Trying to develop clean/safe automated upgrade paths for complex
  packages.
Early versions of these tend to do nasty things to data (openldap was
esp. painful).

> > Does anyone have any ideas how we could resonably reduce problems
> > reported from things such as toolchain breakages in an overlay, yet
> > still not punish the people running the overlay by disallowing it?  I
> > surely wouldn't want to limit the toolchain maintainers from being able
> > to enjoy the use of an overlay if they wished it.
> Perhaps we could ask people who run overlays with dangerous ebuilds, to have 
> these ebuilds protected by some environment variables. (The var must be set 
> for the ebuild to work.)
Only if portage can check the variable before starting to compile any
packages.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 241 bytes --]

Re: [gentoo-dev] Official overlay support

[Michael Cummings]

[-- Attachment #1: Type: text/plain, Size: 607 bytes --]

On Friday 24 March 2006 10:41, Andres Loeh wrote:
> At the moment, Haskell is only a herd and a team, not a project. 

Then stand up my brother! perl wasn't so different, so we snuck in a project 
page. no one said anything. i'm going with we're a project to govern the 
direction and maintenance of the herd until i hear otherwise ;)

if 'they' (the forces of evil in the universe, the consortium, what have you) 
turn to ban us as projects, we could always come together as the 
'languages-you-use-and-abuse project'. might be fun that way. we could have 
pizza socials and such.

~mcummings

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]