[gentoo-dev] Last rites: net-nntp/inn

[gentoo-dev] Last rites: net-nntp/inn

[Markos Chandras]

[-- Attachment #1: Type: Text/Plain, Size: 231 bytes --]

# Markos Chandras <hwoarang@gentoo.org> (11 Jan 2010)
# Fails with -Wl,--as-needed
# bug #182782. Removal in 30 days
net-nntp/inn

-- 
Markos Chandras (hwoarang)
Gentoo Linux Developer
Web: http://hwoarang.silverarrow.org

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Last rites: net-nntp/inn

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 301 bytes --]

On Monday 11 January 2010 16:05:16 Markos Chandras wrote:
> # Markos Chandras <hwoarang@gentoo.org> (11 Jan 2010)
> # Fails with -Wl,--as-needed
> # bug #182782. Removal in 30 days
> net-nntp/inn

is as-needed support really a valid reason for punting a package ?  i dont 
think it is.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[gentoo-dev] Re: Last rites: net-nntp/inn

[Arnaud Launay]

[-- Attachment #1: Type: text/plain, Size: 801 bytes --]

Hello,

Le Mon, Jan 11, 2010 at 11:05:16PM +0200, Markos Chandras a écrit:
> # Markos Chandras <hwoarang@gentoo.org> (11 Jan 2010)
> # Fails with -Wl,--as-needed
> # bug #182782. Removal in 30 days
> net-nntp/inn

As a newsmaster, I'm a bit concerned by this.
By viewing bug #182782 , it seems to me that only inn <=2.4.* is
concerned by this bug, and that >=2.5 does not have it.

But, if I understand this announce correctly, the complete inn
port will be dropped to oblivion.

Wouldn't it be better to stabilize inn 2.5 (there's even a 2.5.1
release out there, with a quite reactive support/dev team, to
which this "bug" could be pushed and corrected), than to just
drop inn entirely ?

Or maybe I'm just plain wrong, in which case I will happily stand
corrected.

	Arnaud.

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Last rites: net-nntp/inn

[Jeroen Roovers]

On Mon, 11 Jan 2010 17:31:08 -0500
Mike Frysinger <vapier@gentoo.org> wrote:

> On Monday 11 January 2010 16:05:16 Markos Chandras wrote:
> > # Markos Chandras <hwoarang@gentoo.org> (11 Jan 2010)
> > # Fails with -Wl,--as-needed
> > # bug #182782. Removal in 30 days
> > net-nntp/inn
> 
> is as-needed support really a valid reason for punting a package ?  i
> dont think it is.

Bad research - he simply lists the wrong reason. Lack of maintainer
attention would have been a better description.

The bug in question should have been closed and kept closed long ago,
so the 2.5 years mentioned on the bug is wrong as well...

I'm looking into this now.


     jer

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Jeroen Roovers]

On Tue, 12 Jan 2010 00:30:24 +0100
Arnaud Launay <asl@launay.org> wrote:

> But, if I understand this announce correctly, the complete inn
> port will be dropped to oblivion.

Yes, and that shouldn't (and won't) happen.

> Wouldn't it be better to stabilize inn 2.5 (there's even a 2.5.1
> release out there, with a quite reactive support/dev team, to
> which this "bug" could be pushed and corrected), than to just
> drop inn entirely ?

I'm working on getting 2.5.1 in the tree (and fixing a USE=python and
some other issues while I'm at it).


     jer

[gentoo-dev] Re: Last rites: net-nntp/inn

[Duncan]

Arnaud Launay posted on Tue, 12 Jan 2010 00:30:24 +0100 as excerpted:

> Hello,
> 
> Le Mon, Jan 11, 2010 at 11:05:16PM +0200, Markos Chandras a écrit:
>> # Markos Chandras <hwoarang@gentoo.org> (11 Jan 2010) # Fails with
>> -Wl,--as-needed
>> # bug #182782. Removal in 30 days
>> net-nntp/inn
> 
> As a newsmaster, I'm a bit concerned by this. By viewing bug #182782 ,
> it seems to me that only inn <=2.4.* is concerned by this bug, and that
> >=2.5 does not have it.
> 
> But, if I understand this announce correctly, the complete inn port will
> be dropped to oblivion.
> 
> Wouldn't it be better to stabilize inn 2.5 (there's even a 2.5.1 release
> out there, with a quite reactive support/dev team, to which this "bug"
> could be pushed and corrected), than to just drop inn entirely ?
> 
> Or maybe I'm just plain wrong, in which case I will happily stand
> corrected.

While I'm an --as-needed user myself, add my post here too.

A work-around is a work-around and shouldn't close the bug, agree with 
flame-eyes there, but it seems a bit much to yank something as standard 
as INN from the tree over this, especially when there's no indication it 
applies to the ~arch version 2.5, only 2.4.x, in either bug (182782, 
248145, refer to the log for the latter).

Not only is there no indication that 2.5 is affected, but if it is, 
where's the mention of the upstream bug number?

Replacements?  There's leafnode, and checking, I see openntpd4 and 
twisted news.  Still, inn is rather like sendmail for news.  Do we 
/really/ want to kill it?

IMO, the backing on this one seems thin and the timing premature.  
Perhaps it does need to go, but if so, there's certainly a stronger case 
to be made for it than those couple bugs and the last-rites masking 
comment.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Richard Freeman]

On 01/11/2010 06:30 PM, Arnaud Launay wrote:
>
> As a newsmaster, I'm a bit concerned by this.

Yeah, inn seems like a really high-profile package to mask for removal. 
  It would be conspicuous in its absence.

Would it make sense to post on -dev BEFORE masking packages like this? 
I'm sure there are lots of people who would chip in before something 
like this dies.

Right now lots of users are going to get errors due to a masked package 
until somebody takes the initiative to fix it.  I suspect that nobody 
wants to poke their head up and risk getting it shot off by doing 
something like that...

Perhaps Gentoo needs a little more of Wikipedia's "Be Bold" attitude and 
a little less of their "delete first and ask questions later" attitude.

Note - I'm not suggesting the problem shouldn't be fixed - I'm just 
suggesting that in this case the solution is worse than the original 
problem.

Rich

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Jeroen Roovers]

On Tue, 12 Jan 2010 02:02:14 +0100
Jeroen Roovers <jer@gentoo.org> wrote:

> I'm working on getting 2.5.1 in the tree (and fixing a USE=python and
> some other issues while I'm at it).

net-nntp/inn-2.5.1 is in the tree and fixes many (QA) issues. Please
track bug #300650 [1] if you want to stay informed of its
stabilisation, which should be performed in about a month unless more
non-trivial issues get uncovered. I applied all of the QA changes to
2.5.0 for convenience. I also removed the package.mask, and closed
both(!) open bug reports as FIXED (in 2.5.*).


     jer


[1] https://bugs.gentoo.org/show_bug.cgi?id=300650

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Jeremy Olexa]

On Mon, 11 Jan 2010 20:36:37 -0500, Richard Freeman <rich0@gentoo.org>
wrote:
> On 01/11/2010 06:30 PM, Arnaud Launay wrote:
>>
>> As a newsmaster, I'm a bit concerned by this.
> 
> Yeah, inn seems like a really high-profile package to mask for removal. 
>   It would be conspicuous in its absence.
> 
> Would it make sense to post on -dev BEFORE masking packages like this? 
> I'm sure there are lots of people who would chip in before something 
> like this dies.

(A general reply, not targeted towards you, Rich)

Speaking on behalf of the treecleaners:
The fact is, some of us have never heard of "inn" and until Gentoo has
some sort of "popularity tracking" software/tool, the treecleaners will
continue to mask unmaintained software. We can't possible know about every
package in the tree and if it looks like it is unmaintained (open bugs w/o
action) then we will mask it for removal unless someone fixes it and
maintains it.

Let's all move on here and be happy that someone is now maintaining such a
popular package. Thanks jer/rej - I'll add you to metadata so it doesn't
become unmaintained again :) Wasn't there a GSoC project on popularity of
packages? Let's get it implemented already! ;)

-Jeremy

> 
> Right now lots of users are going to get errors due to a masked package 
> until somebody takes the initiative to fix it.  I suspect that nobody 
> wants to poke their head up and risk getting it shot off by doing 
> something like that...
> 
> Perhaps Gentoo needs a little more of Wikipedia's "Be Bold" attitude and

> a little less of their "delete first and ask questions later" attitude.
> 
> Note - I'm not suggesting the problem shouldn't be fixed - I'm just 
> suggesting that in this case the solution is worse than the original 
> problem.
> 
> Rich

[gentoo-dev] Thanks for the rescue! Was: Last rites: net-nntp/inn

[Duncan]

Jeroen Roovers posted on Tue, 12 Jan 2010 03:22:05 +0100 as excerpted:

> net-nntp/inn-2.5.1 is in the tree and fixes many (QA) issues. [& etc]

Thanks! =:^)

(Not to be an aoler and metoo, but I asked some time ago and the 
consensus seemed to be that thanks were good even if they meant an extra 
email or two, and this package is worth it, so... Yes!  Thanks!)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Markos Chandras]

[-- Attachment #1: Type: Text/Plain, Size: 1136 bytes --]

On Tuesday 12 January 2010 04:22:05 Jeroen Roovers wrote:
> On Tue, 12 Jan 2010 02:02:14 +0100
> 
> Jeroen Roovers <jer@gentoo.org> wrote:
> > I'm working on getting 2.5.1 in the tree (and fixing a USE=python and
> > some other issues while I'm at it).
> 
> net-nntp/inn-2.5.1 is in the tree and fixes many (QA) issues. Please
> track bug #300650 [1] if you want to stay informed of its
> stabilisation, which should be performed in about a month unless more
> non-trivial issues get uncovered. I applied all of the QA changes to
> 2.5.0 for convenience. I also removed the package.mask, and closed
> both(!) open bug reports as FIXED (in 2.5.*).
> 
> 
>      jer
> 
> 
> [1] https://bugs.gentoo.org/show_bug.cgi?id=300650
> 
Thanks for saving this package. As Jeremy said, there is absolutely no way to 
measure the popularity of a package. So if it has no maintainer, and open bugs 
we have to mask it and announce it here. It is up to you whether you want to 
save it or not

Anyway, sorry for the inconvenience
-- 
Markos Chandras (hwoarang)
Gentoo Linux Developer
Web: http://hwoarang.silverarrow.org

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Richard Freeman]

On 01/11/2010 10:43 PM, Jeremy Olexa wrote:
> (A general reply, not targeted towards you, Rich)

No prob - my post wasn't really directed personally at anybody.

>
> Speaking on behalf of the treecleaners:
> The fact is, some of us have never heard of "inn" and until Gentoo has
> some sort of "popularity tracking" software/tool, the treecleaners will
> continue to mask unmaintained software.

Yikes - just goes to show how NNTP is starting to fade into the past.  :)

I'm not sure if it would cause undue overhead, but perhaps a solution 
would be to:

1.  Open a bug stating that the package will be discarded - assign to 
the maintainer.  This gives the maintainer a chance to wake up.  You can 
even do this without having to try to contact them first which might 
save you a step if you're doing that now.

2.  Periodically post a list of packages that have said bugs logged for 
more than two weeks on -dev-announce - reference the bug number.  That 
gives the community at large a chance to pick up the package.

3.  In another two weeks, if some dev hasn't stepped in to maintain, 
then mask as usual.  Don't announce this since anybody who cares should 
have CC'ed themselves on the bug.

4.  Of course, security issues / etc take priority and appropriate 
action is taken quickly (try to find a maintainer, but mask otherwise).

I'd think that if you tagged bugs appropriately you could largely 
automate #2 and #3 - just query for bugs over a certain age with a given 
keyword or whatever.

This would probably lengthen the time needed to get rid of a package, 
but it wouldn't really increase the work needed by too much.  You 
already announce on the list that you're masking packages - now you'd 
announce two weeks earlier and skip the announcement when the mask is made.

This is just a suggestion, but it does eliminate the need to try to make 
judgment calls about whether a given package is or isn't "important."

Rich

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Jeroen Roovers]

On Tue, 12 Jan 2010 18:32:06 +0200
Markos Chandras <hwoarang@gentoo.org> wrote:

> Thanks for saving this package. As Jeremy said, there is absolutely
> no way to measure the popularity of a package. So if it has no
> maintainer, and open bugs we have to mask it and announce it here. It
> is up to you whether you want to save it or not

I don't think the (perceived) popularity of the package has anything to
do with it.

I do think maybe treecleaner@ needs to set up policies with regard to
methods of investigation, thoroughness, and transparency. In the case
at hand, treecleaner shouldn't have been called in (you're not the
bloody cavalry you know! ;-) in the first place, and should certainly
not have acted (so quickly).

It's not clear to me generally what you (treecleaner@) all do and why
you do it - but it *is* clear that it's very easy to `rm -r *' to get
rid of some old stuff and that you may end up regretting it later.

Particularly, it looks like the net-mail, net-news and netmon herds are
understaffed and have been for a while, and I see a general shift of
developers towards desktop oriented packages and away from the nuts and
bolts that make it all go.

I think (but have no facts apart from talking to people and handling
network package related bugs in every way possible) that our userbase
is still much more technically oriented. If that's all true, then doing
some `rm net-*/*' cleanups may well end up hurting Gentoo as you would
drive out more of the networking oriented people (users and developers)
that I feel we still need to support, and turn into Yet Another Desktop
Oriented Distro (which we also need, but that's already covered quite
well).

ISTR treecleaner@ already had some policy in place that requires some
$period to pass before you mask for removal. Maybe you should announce
an upcoming mask nice and early to keep that shock wave from reaching
users straight away.


Regards,
     jer

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Markos Chandras]

[-- Attachment #1: Type: Text/Plain, Size: 3003 bytes --]

On Tuesday 12 January 2010 20:21:59 Jeroen Roovers wrote:
> On Tue, 12 Jan 2010 18:32:06 +0200
> 
> Markos Chandras <hwoarang@gentoo.org> wrote:
> > Thanks for saving this package. As Jeremy said, there is absolutely
> > no way to measure the popularity of a package. So if it has no
> > maintainer, and open bugs we have to mask it and announce it here. It
> > is up to you whether you want to save it or not
> 
> I don't think the (perceived) popularity of the package has anything to
> do with it.
> 
> I do think maybe treecleaner@ needs to set up policies with regard to
> methods of investigation, thoroughness, and transparency. In the case
> at hand, treecleaner shouldn't have been called in (you're not the
> bloody cavalry you know! ;-) in the first place, and should certainly
> not have acted (so quickly).
> 
> It's not clear to me generally what you (treecleaner@) all do and why
> you do it - but it *is* clear that it's very easy to `rm -r *' to get
> rid of some old stuff and that you may end up regretting it later.
> 
> Particularly, it looks like the net-mail, net-news and netmon herds are
> understaffed and have been for a while, and I see a general shift of
> developers towards desktop oriented packages and away from the nuts and
> bolts that make it all go.
> 
> I think (but have no facts apart from talking to people and handling
> network package related bugs in every way possible) that our userbase
> is still much more technically oriented. If that's all true, then doing
> some `rm net-*/*' cleanups may well end up hurting Gentoo as you would
> drive out more of the networking oriented people (users and developers)
> that I feel we still need to support, and turn into Yet Another Desktop
> Oriented Distro (which we also need, but that's already covered quite
> well).
So what do you suggest? Have old, unmaintained and broken ( or forgotten ) 
packages under those categories in order to preserve the "personality" of 
Gentoo? IMHO ( this is not a treecleaners@ opinion, i m just talking for my 
self ), announcing and masking a package is a good way to inform and wake up 
everybody to take care of this package if they really really want to stay on 
portage. Having broken and unmaintained packages on tree, just to say that we 
have plenty of packages on portage is not acceptable policy imho. So if you 
want a package, plz take care of it :)
> 
> ISTR treecleaner@ already had some policy in place that requires some
> $period to pass before you mask for removal. Maybe you should announce
> an upcoming mask nice and early to keep that shock wave from reaching
> users straight away.
Having open bugs for months isn't a way to let everybody know that this 
package is broken for long time, so it is a valid candidate for removal? 
Should we send that via e-mail as well? 
> 
> 
> Regards,
>      jer
> 

-- 
Markos Chandras (hwoarang)
Gentoo Linux Developer
Web: http://hwoarang.silverarrow.org

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Richard Freeman]

On 01/12/2010 01:30 PM, Markos Chandras wrote:
> IMHO ( this is not a treecleaners@ opinion, i m just talking for my
> self ), announcing and masking a package is a good way to inform and wake up
> everybody to take care of this package if they really really want to stay on
> portage.

I agree with the announce part, and the THREAT of masking.  I just don't 
think that the masking should happen at the same time as the announcement.

> Having open bugs for months isn't a way to let everybody know that this
> package is broken for long time, so it is a valid candidate for removal?
> Should we send that via e-mail as well?

I don't think an open bug constitutes notice.  It is valid notice to the 
maintainer of the package, but not to the larger community.

I probably have 100-200 packages installed, and I'd probably be annoyed 
if any of them went away (I'm still missing kdirstat, but that isn't 
really the kde team's fault).  If an important one was about to go away 
I might step in to maintain it, and I'm sure a lot of other people feel 
the same way.  At the same time, I can't monitor the bugs on 100-200 
packages - that is the reason for having a maintainer.

I think the concern is that some maintainers don't respond in a timely 
manner.  Now, I don't know that maintainers have an obligation to fix 
every bug within a certain timeframe - some packages are problematic and 
I'm not sure that we should discard a 98% solution in favor of a 0% one 
because we don't have a 100% one.  However, serious issues should be in 
scope for treecleaners, but the first goal should be to find a 
maintainer, and only if that fails should we consider masking.

Also - if a maintainer can't be found we might also try to coordinate 
with the Sunrise folks to see if they're willing to take it over.

We should also solicit proxy-maintainers among the user community when 
we announce pending removals.  That could be very helpful with something 
like inn:  I use it VERY lightly and I'm not a news guru, but I am a 
dev.  I bet we have users who are news gurus and who care about inn, but 
they aren't Gentoo devs.  Together we could probably cover the gaps and 
I'm sure devs would be more willing to pick up a package if they knew 
they had some help with the nuances of the package itself.  If that 
falls apart later, at least an active dev is assigned to the package, 
and they can always decide to try to find a new maintainer or kill the 
package (saving treecleaners the work of doing so).

In short - treecleaners should be about getting packages back into the 
mainstream maintenance model, with removal being the fallback option if 
that doesn't work.  They shouldn't have to go out of their way to do 
this, but an advance announcement and some coordination is probably a 
good idea.

Rich

[gentoo-dev] Re: Last rites: net-nntp/inn

[Arnaud Launay]

[-- Attachment #1: Type: text/plain, Size: 1716 bytes --]

Le Tue, Jan 12, 2010 at 08:30:26PM +0200, Markos Chandras a écrit:
> So if you want a package, plz take care of it :)

From a "user" point of view, having submitted ebuilds, patches
ideas and questions here and there on bugs.go, I must admit I end
up putting up my "contributions" on my local /usr/local/portage
because it never makes it to the official tree.

I love Gentoo, but sometimes I feel like I'm using a debian stable :)

I do not say everything should be bloody-edge at all times, but
as a hoster, I use a lot of those so-called "servers" -- apache,
mysql, cyrus-imapd, inn, postfix, amavisd, maia (which has a
working, if not perfect, ebuild standing in bugs.go for years now
-- #130068 -- and which is in "maintainer-needed" status), etc) ,
and I have to put them under our local tree to use them.

For example, I've one for pondus, a very small gnome app which
tracks your weight... I never bothered submitting it to bgo.

I wonder how much time flies before users begin to stop posting
their corrections to bugs.go and keep them local, because they
never get committed.

I agree that some management is needed, but I feel like it's
becoming more and more complicated to participate in the project
-- either you have to write a document showing you understand how
the management works (why should I care ? I just want to add a
patch), either your patch/ebuild is "ignored" for years.

It might not be the case for every herd, though.

It seems to me it was easier to participate in 2004...

BTW, I'm not looking to troll here, so please: no flamewar. It's
just my humble opinion, I don't want to get people mad, and I can
very much be completely wrong.

	Arnaud.

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Markos Chandras]

[-- Attachment #1: Type: Text/Plain, Size: 1484 bytes --]

On Tuesday 12 January 2010 21:40:37 Arnaud Launay wrote:
> Le Tue, Jan 12, 2010 at 08:30:26PM +0200, Markos Chandras a écrit:
> > So if you want a package, plz take care of it :)
> 
> From a "user" point of view, having submitted ebuilds, patches
> ideas and questions here and there on bugs.go, I must admit I end
> up putting up my "contributions" on my local /usr/local/portage
> because it never makes it to the official tree.
> 
> I love Gentoo, but sometimes I feel like I'm using a debian stable :)
> 
> I do not say everything should be bloody-edge at all times, but
> as a hoster, I use a lot of those so-called "servers" -- apache,
> mysql, cyrus-imapd, inn, postfix, amavisd, maia (which has a
> working, if not perfect, ebuild standing in bugs.go for years now
> -- #130068 -- and which is in "maintainer-needed" status), etc) ,
> and I have to put them under our local tree to use them.
> 
If you feel like it, become a proxy-maintainer and poke a developer to put 
your ebuilds on tree. Have you ever heard of that ? :)
> For example, I've one for pondus, a very small gnome app which
> tracks your weight... I never bothered submitting it to bgo.
> 
Same rule applies here
> I wonder how much time flies before users begin to stop posting
> their corrections to bugs.go and keep them local, because they
> never get committed.

> 	Arnaud.
> 

-- 
Markos Chandras (hwoarang)
Gentoo Linux Developer
Web: http://hwoarang.silverarrow.org

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Last rites: net-nntp/inn

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 671 bytes --]

On Monday 11 January 2010 20:00:40 Jeroen Roovers wrote:
> On Mon, 11 Jan 2010 17:31:08 -0500 Mike Frysinger wrote:
> > On Monday 11 January 2010 16:05:16 Markos Chandras wrote:
> > > # Markos Chandras <hwoarang@gentoo.org> (11 Jan 2010)
> > > # Fails with -Wl,--as-needed
> > > # bug #182782. Removal in 30 days
> > > net-nntp/inn
> >
> > is as-needed support really a valid reason for punting a package ?  i
> > dont think it is.
> 
> Bad research - he simply lists the wrong reason. Lack of maintainer
> attention would have been a better description.

which is still not a good enough reason for removal if the version in the tree 
still works
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1226 bytes --]

On Monday 11 January 2010 22:43:18 Jeremy Olexa wrote:
> On Mon, 11 Jan 2010 20:36:37 -0500, Richard Freeman wrote:
> > On 01/11/2010 06:30 PM, Arnaud Launay wrote:
> >> As a newsmaster, I'm a bit concerned by this.
> >
> > Yeah, inn seems like a really high-profile package to mask for removal.
> >   It would be conspicuous in its absence.
> >
> > Would it make sense to post on -dev BEFORE masking packages like this?
> > I'm sure there are lots of people who would chip in before something
> > like this dies.
> 
> (A general reply, not targeted towards you, Rich)
> 
> Speaking on behalf of the treecleaners:
> The fact is, some of us have never heard of "inn" and until Gentoo has
> some sort of "popularity tracking" software/tool, the treecleaners will
> continue to mask unmaintained software. We can't possible know about every
> package in the tree and if it looks like it is unmaintained (open bugs w/o
> action) then we will mask it for removal unless someone fixes it and
> maintains it.

you need to fix your filter then.  an "open bug" is not an acceptable reason 
for masking a package.  if you're going to clean a package, you need to 
research actual reasons to mask & punt.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Ben de Groot]

2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
> If you feel like it, become a proxy-maintainer and poke a developer to put
> your ebuilds on tree. Have you ever heard of that ? :)

Proxy-maintainership should be given a MUCH higher profile in Gentoo,
in my opinion. It is a virtually unknown option.

Another thing that works in my experience, but this is up to the
herds/projects, is having an official overlay where devs and users can
work closely together. This allows users to commit ebuilds and
patches, while there is quality control by the involved devs. Devs can
keep an eye on such overlays and move stuff to portage when they are
ready. Sunrise is the most obvious example for this, for
maintainer-wanted packages. But it works equally well for us in the Qt
project with qting-edge, and I believe kde and pro-audio have the same
experience.

But I also believe we need a better structure to handle
maintainer-needed, maintainer-wanted and nominally maintained but
ignored packages. Maybe we should form a team, which would be
dedicated to take care of such things, and which would have a review
policy for user submitted ebuilds and patches in bugzilla. A bit like
treecleaners, but bringing life instead of death. What do you think?

Cheers,
-- 
Ben de Groot
Gentoo Linux developer (qt, media, lxde, desktop-misc)
______________________________________________________

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Tomáš Chvátal]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dne 12.1.2010 21:33, Mike Frysinger napsal(a):
> On Monday 11 January 2010 22:43:18 Jeremy Olexa wrote:
>> On Mon, 11 Jan 2010 20:36:37 -0500, Richard Freeman wrote:
>>> On 01/11/2010 06:30 PM, Arnaud Launay wrote:
>>>> As a newsmaster, I'm a bit concerned by this.
>>>
>>> Yeah, inn seems like a really high-profile package to mask for removal.
>>>   It would be conspicuous in its absence.
>>>
>>> Would it make sense to post on -dev BEFORE masking packages like this?
>>> I'm sure there are lots of people who would chip in before something
>>> like this dies.
>>
>> (A general reply, not targeted towards you, Rich)
>>
>> Speaking on behalf of the treecleaners:
>> The fact is, some of us have never heard of "inn" and until Gentoo has
>> some sort of "popularity tracking" software/tool, the treecleaners will
>> continue to mask unmaintained software. We can't possible know about every
>> package in the tree and if it looks like it is unmaintained (open bugs w/o
>> action) then we will mask it for removal unless someone fixes it and
>> maintains it.
> 
> you need to fix your filter then.  an "open bug" is not an acceptable reason 
> for masking a package.  if you're going to clean a package, you need to 
> research actual reasons to mask & punt.
> -mike
Dont be joking,
Your approach of adding new packages to main tree is that you add them
with empty metadata.xml and we have to remove them in few years because
they are steaming piles of bugs...

Lack of maintainer and open bugs are valid reasons.
And since WE want to enable as-needed as default at some time we need to
work on the bugs -> if packages have no maintainer and fails it we will
have to punt them (that bug was open for 2 years+, and clearly there
were even newer versions none bothered to bump to). If anyone picks them
up and fix in the mask for cleaning that's great, because that is the
reason for having that mask, so people can be loud about removal. We
could simply punt things at the moment we want without any notice if we
would not care about user responses for such actions.

Currently there are 109 reason [1] why as-needed cant be done, if the
package has maintainer he should work on it, otherwise byes, there are
quite large unmaintained areas in the tree we have to care about.

[1]
http://bugs.gentoo.org/buglist.cgi?bug_id=182324,249450,226909,299077,247869,248195,248437,285747,280705,247931,297193,278310,207605,284921,247067,248586,277655,246755,277206,249295,248548,247043,248678,278423,247088,282426,247777,246729,246961,274385,278104,300515,248345,277169,248356,277227,248169,295199,247761,277769,247444,294396,247768,247844,276303,278069,276250,246726,257996,247731,247054,277925,276873,294971,278100,297025,248549,247779,276295,247712,260226,280922,248556,248163,248192,298152,274700,265643,257918,277938,287933,248143,248571,276928,226863,247991,226885,248152,248573,247044,296631,248351,248552,247748,226917,246875,248555,294738,277794,277050,246970,248159,248605,247919,276506,297409,277640,248357,294878,248579,132992,248577,248551,278086,276796,248411,299478,248580,276302

- --------
Tomáš Chvátal
Gentoo Linux Developer [KDE/Overlays/QA/X11]
E-Mail          : scarabeus@gentoo.org
GnuPG FP        : 94A4 5CCD 85D3 DE24 FE99 F924 1C1E 9CDE 0341 4587
GnuPG ID        : 03414587
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAktM4NAACgkQHB6c3gNBRYeCSQCfSWnP07SLw9sBLUENdN9ZAEYT
kcoAoLSM6ohZ3wzn47LdcEWDq8oTGQZk
=1n2p
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Denis Dupeyron]

2010/1/12 Tomáš Chvátal <scarabeus@gentoo.org>:
> Dont be joking,
[...]

Mmmh? Take a deep breath, a long walk, a large beer, or whatever
works. Because you need it.

Denis.

[gentoo-dev] Re: Last rites: net-nntp/inn

[Duncan]

Ben de Groot posted on Tue, 12 Jan 2010 21:35:45 +0100 as excerpted:

> 2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
>> If you feel like it, become a proxy-maintainer and poke a developer to
>> put your ebuilds on tree. Have you ever heard of that ? :)
> 
> Proxy-maintainership should be given a MUCH higher profile in Gentoo, in
> my opinion. It is a virtually unknown option.

Yay! =:^)

> Another thing that works in my experience, but this is up to the
> herds/projects, is having an official overlay where devs and users can
> work closely together.

Again! =:^)

> But I also believe we need a better structure to handle
> maintainer-needed, maintainer-wanted and nominally maintained but
> ignored packages. Maybe we should form a team, which would be dedicated
> to take care of such things, and which would have a review policy for
> user submitted ebuilds and patches in bugzilla. A bit like treecleaners,
> but bringing life instead of death. What do you think?

Well, sunrise was supposed to be that for packages not yet in the tree.  
The devs shepherding that work hard with the users doing the ebuilds to 
get them up to tree quality, so they're ready for devs to take on with 
little work (or to go into proxy maintainership), when it's that 
package's time.  I know I've used a handful of sunrise packages that 
ultimately ended up in the tree, which is pretty good, considering I 
don't even have the sunrise overlay enabled unless there's something I 
specifically want from it.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: Last rites: net-nntp/inn

[Duncan]

Richard Freeman posted on Tue, 12 Jan 2010 14:07:38 -0500 as excerpted:

> On 01/12/2010 01:30 PM, Markos Chandras wrote:
>> IMHO ( this is not a treecleaners@ opinion, i m just talking for my
>> self ), announcing and masking a package is a good way to inform and
>> wake up everybody to take care of this package if they really really
>> want to stay on portage.
> 
> I agree with the announce part, and the THREAT of masking.  I just don't
> think that the masking should happen at the same time as the
> announcement.

FWIW, I feel for the treecleaners.  It's a job with little thanks and 
lots of chance to make someone mad at you, but I'm glad /someone's/ doing 
it! =:^)

So going with this idea...  Isn't the treecleaner masking 30-day at 
present?  What about extending that just a bit, to 5 weeks total, while 
reducing the actual masking to 4 weeks, with the extra week a wait time 
between the traditional last-rites mail and the masking?

In the case of the INNs of the tree, that should prevent masking 
entirely, since popular packages will certainly have someone raising the 
roof on just the warning, within a day or two.  That was certainly the 
case here.  No masking means ordinary users won't have to ever know it 
happened.

Or is that extra step going to throw a spanner into the works for 
treecleaners?  As I said, I definitely appreciate the job they're doing, 
and wouldn't want to make their life harder.  But this could well reduce 
the fallout when the INNs of the tree come up, and that just might make 
it easier to handle, even if tracking that extra step /is/ a bit more 
work.

Treecleaners?

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: Last rites: net-nntp/inn

[Arnaud Launay]

Le Tue, Jan 12, 2010 at 10:37:19PM +0000, Duncan a écrit:
> FWIW, I feel for the treecleaners.  It's a job with little
> thanks and lots of chance to make someone mad at you, but I'm
> glad /someone's/ doing it! =:^)

Yeah. I'm glad each time I see old things getting deleted,
abandoned software and such. So, yeah, thanks, treecleaners.
Your job is as easy as a sysadmin's one: no one knows you exist,
except when someone needs to scream at you...

> In the case of the INNs of the tree, that should prevent
> masking entirely, since popular packages will certainly have
> someone raising the roof on just the warning, within a day or
> two.  That was certainly the case here.  No masking means
> ordinary users won't have to ever know it happened.

Well, as it happens, I knew it was being masked because I got the
mail warning from gentoo-dev-announce, which is unfortunately a
bit badly advertised.

Anyway, I would have scratched my head far, far more if I had to
understand WTF portage would complain about an inn masked... I
don't care when it's small, unused package -- last time it was
lprof, and I didn't really care, it was a bit still here from a
package I installed years ago, and which passed through depclean.

So, what about something like:
* mail on gentoo-dev-announce, saying "heads up. mask in one week"
* one week later, mask and "classical" mail "foo/bar masked"

I have absolutely no idea how much work it requires, so I won't
complain if TC says it's too complicated/unpratical/etc.

BTW, I have no knowledge of the concept of proxy-maintainer, I'll
look at it tomorrow, it's 2am here... :)  I don't even think I
ever heard of it before, but I didn't brush my gentoo-fu for a
few years, that may explain...

	Arnaud.

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Jeroen Roovers]

On Tue, 12 Jan 2010 21:51:28 +0100
Tomáš Chvátal <scarabeus@gentoo.org> wrote:

> > you need to fix your filter then.  an "open bug" is not an
> > acceptable reason for masking a package.  if you're going to clean
> > a package, you need to research actual reasons to mask & punt.
> > -mike
> Dont be joking,
> Your approach of adding new packages to main tree is that you add them
> with empty metadata.xml and we have to remove them in few years
> because they are steaming piles of bugs...

Er, say whah? Flinging mud?

Mike's got a very valid point, in that you don't mask a package because
of an open bug. All the rest of what you added below is about
--as-needed, which doesn't apply to the bug in case and which is still
no valid reason to mask a package anyway. End of discussion, plz?


Yah,
     jer

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Jeroen Roovers]

On Tue, 12 Jan 2010 22:37:19 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

> So going with this idea...  Isn't the treecleaner masking 30-day at 
> present?  What about extending that just a bit, to 5 weeks total,
> while reducing the actual masking to 4 weeks, with the extra week a
> wait time between the traditional last-rites mail and the masking?

No, masking for removal should take 30 days. I strongly feel that
before treecleaner@ does any masking, an announcement should go to
-dev@ and -announce@ a pretty long time in advance, maybe two months,
especially with the two cockups a month that I am counting now.


     jer

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Jeroen Roovers]

On Wed, 13 Jan 2010 02:18:59 +0100
Arnaud Launay <asl@launay.org> wrote:

> I have absolutely no idea how much work it requires, so I won't
> complain if TC says it's too complicated/unpratical/etc.

rm -r * is easy.

> BTW, I have no knowledge of the concept of proxy-maintainer, I'll
> look at it tomorrow, it's 2am here... :)  I don't even think I
> ever heard of it before, but I didn't brush my gentoo-fu for a
> few years, that may explain...

It should probably be documented at the official places [1][2].


[1] http://devmanual.gentoo.org
[2] http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml [3]
[3] Why weren't these merged years ago?

[gentoo-dev] Re: Last rites: net-nntp/inn

[Duncan]

Jeroen Roovers posted on Wed, 13 Jan 2010 06:48:18 +0100 as excerpted:

> On Tue, 12 Jan 2010 22:37:19 +0000 (UTC) Duncan <1i5t5.duncan@cox.net>
> wrote:
> 
>> So going with this idea...  Isn't the treecleaner masking 30-day at
>> present?  What about extending that just a bit, to 5 weeks total, while
>> reducing the actual masking to 4 weeks, with the extra week a wait time
>> between the traditional last-rites mail and the masking?
> 
> No, masking for removal should take 30 days. I strongly feel that before
> treecleaner@ does any masking, an announcement should go to -dev@ and
> -announce@ a pretty long time in advance, maybe two months, especially
> with the two cockups a month that I am counting now.

30-day masking /does/ give the folks updating once a month at least one 
warning, so I can see the case for that.

But... IMO extending the pre-mask warning another full 30 days... is 
asking for trouble going the /other/ way.  It's not urgent enough to 
require immediate action... which can unfortunately cause people to put 
it off and forget about it.  Then it's masked 30 days later... and we're 
back where we were!

So I'd say a week to 15 days pre-mask warning... and 14-15 days is 
stretching it.  A week is just about right, short enough to require 
urgent action and thus front-burnering, long enough that if there's a 
clear objection to be made, it should be very clear within 2-3 days, and 
there's another 4 days to actually do something about it before the 
masking.

Just my (non-gentoo-dev) opinion, however.  I'm acutely aware I'm not the 
one doing the work, so if that opinion doesn't match dev-reality, feel 
free to ignore it.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 830 bytes --]

On Tuesday 12 January 2010 15:51:28 Tomáš Chvátal wrote:
> Your approach of adding new packages to main tree is that you add them
> with empty metadata.xml and we have to remove them in few years because
> they are steaming piles of bugs...

not that this is relevant at all to my point, but when i notice such bugs, 
i'll fix them.  but if no one cc's me, then i dont generally notice.

> Lack of maintainer

incorrect

> open bugs are valid reasons.

the *type* of bug matters, not the existence of an open bug.  take any number 
of the trivial QA bugs open related to an extra empty dir or split docs -- 
none are valid for punting an otherwise perfectly working package.

> And since WE want to enable as-needed as default at some time we need to
> work on the bugs

which isnt going to happen
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[gentoo-dev] Re: Last rites: net-nntp/inn

[Arnaud Launay]

Le Wed, Jan 13, 2010 at 06:52:09AM +0100, Jeroen Roovers a écrit:
> > BTW, I have no knowledge of the concept of proxy-maintainer, I'll
> It should probably be documented at the official places [1][2].
> [1] http://devmanual.gentoo.org
> [2] http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml [3]
> [3] Why weren't these merged years ago?

Well, nothing about "proxy-maintainer" here.
I just found one doc at
http://dev.gentoo.org/~antarus/projects/proxy-maint/

which kind of explain what is a proxy maintainer (more or less),
but does not explain how to become one...

Le Tue, Jan 12, 2010 at 09:49:10PM +0200, Markos Chandras a écrit:
> If you feel like it, become a proxy-maintainer and poke a
> developer to put your ebuilds on tree. Have you ever heard of that ? :)

No problem. Just tell me who I need to poke :) Would that be
antarus, saying "hey, I'm mostly in servers, how may I be of
service" ?

	Arnaud.

[gentoo-dev] proxy maintainership and gentoo-x86 scm

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 644 bytes --]

On Tuesday 12 January 2010 15:35:45 Ben de Groot wrote:
> 2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
> > If you feel like it, become a proxy-maintainer and poke a developer to
> > put your ebuilds on tree. Have you ever heard of that ? :)
> 
> Proxy-maintainership should be given a MUCH higher profile in Gentoo,
> in my opinion. It is a virtually unknown option.

i think our current work flows also significantly impede the smooth running of 
this.  if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a 
much smoother ride for Gentoo devs to pull from a proxy maintainer and push on 
their behalf.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Richard Freeman]

On 01/13/2010 09:24 AM, Mike Frysinger wrote:
> On Tuesday 12 January 2010 15:51:28 Tomáš Chvátal wrote:
>> And since WE want to enable as-needed as default at some time we need to
>> work on the bugs
>
> which isnt going to happen

This isn't really intended to point fingers at anybody in particular - I 
haven't personally investigated the complexity of fixing the as-needed 
issue for this particular package.

I think that logging as-needed bugs is certainly a value-add.

I think that tracking a blocker for as-needed is a value-add.

However, if we want to turn as-needed into a QA issue and try to enforce 
it, I think that this should really be run past the council and 
documented.  It wouldn't hurt to also document tips for solving the 
problem and the proper way to mask as-needed if it just isn't going to 
work (even if we make as-needed the default that doesn't mean that we 
can't mask it if we have to).

I think that devs should make good-faith efforts to fix as-needed 
issues, but if the problem is with the overall upstream design and major 
work is involved, that is an UPSTREAM problem.  Sure, it is nice if 
somebody wants to be an upstream contributor and fix their problems for 
them, but I'm not sure that it is worth the Gentoo resources in every 
single case.  Maybe for system packages or common dependencies we might 
push a little harder.

In any case, when this kind of controversy exists the best solution is 
to make a proposal and ask the council to render a decision.  It isn't 
productive to have battles on the mailing list about whether something 
should or shouldn't be policy.

I don't mean to suggest that QA or treecleaners or whatever absolutely 
must run everything they do past the council.  However, when we run into 
genuine disagreements between projects/herds/devs that is the ultimate 
escalation path.

Package mask is not a very good way to try to hit devs with a cluestick 
anyway - the main victims of this sort of approach tend to be the users.

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Richard Freeman]

On 01/13/2010 10:06 AM, Arnaud Launay wrote:
> which kind of explain what is a proxy maintainer (more or less),
> but does not explain how to become one...
>

We don't really have any official process around this.  Things like 
sunrise and proxy-maintainers are good ways to get new blood into the 
contributor pool, however, and I think they'd be good things to look 
into.  Maybe I'll try to brainstorm some ideas of how to generate 
involvement (I'll post on -project if I come up with something).

> Le Tue, Jan 12, 2010 at 09:49:10PM +0200, Markos Chandras a écrit:
>> If you feel like it, become a proxy-maintainer and poke a
>> developer to put your ebuilds on tree. Have you ever heard of that ? :)
>
> No problem. Just tell me who I need to poke :) Would that be
> antarus, saying "hey, I'm mostly in servers, how may I be of
> service" ?

Yup - some kind of clearinghouse and communications tool wouldn't hurt. 
  You can always post in irc or a list and see if you get any takers. 
You can also post them in bugzilla under maintainer-wanted, but it 
doesn't get much visibility there.

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Ben de Groot]

2010/1/13 Mike Frysinger <vapier@gentoo.org>:
> On Tuesday 12 January 2010 15:35:45 Ben de Groot wrote:
>> 2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
>> > If you feel like it, become a proxy-maintainer and poke a developer to
>> > put your ebuilds on tree. Have you ever heard of that ? :)
>>
>> Proxy-maintainership should be given a MUCH higher profile in Gentoo,
>> in my opinion. It is a virtually unknown option.
>
> i think our current work flows also significantly impede the smooth running of
> this.  if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
> their behalf.
> -mike

I couldn't agree more!

-- 
Ben de Groot
Gentoo Linux developer (qt, media, lxde, desktop-misc)
______________________________________________________

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[justin]

[-- Attachment #1: Type: text/plain, Size: 959 bytes --]

On 13/01/10 22:02, Ben de Groot wrote:
> 2010/1/13 Mike Frysinger <vapier@gentoo.org>:
>> On Tuesday 12 January 2010 15:35:45 Ben de Groot wrote:
>>> 2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
>>>> If you feel like it, become a proxy-maintainer and poke a developer to
>>>> put your ebuilds on tree. Have you ever heard of that ? :)
>>>
>>> Proxy-maintainership should be given a MUCH higher profile in Gentoo,
>>> in my opinion. It is a virtually unknown option.
>>
>> i think our current work flows also significantly impede the smooth running of
>> this.  if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
>> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
>> their behalf.
>> -mike
> 
> I couldn't agree more!
> 

So what is the current mood to switch to git. As there is no traffic on
the scm-ml I assume that there is no progress. But shouldn' we start to
move on? Now!?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

[gentoo-dev] Re: proxy maintainership and gentoo-x86 scm

[Christian Faulhammer]

[-- Attachment #1: Type: text/plain, Size: 456 bytes --]

Hi,

justin <jlec@j-schmitz.net>:
> So what is the current mood to switch to git. As there is no traffic
> on the scm-ml I assume that there is no progress. But shouldn' we
> start to move on? Now!?

 Mood is good, but someone has to do the work.  And robbat2 cannot do
all of it.

V-Li

-- 
Christian Faulhammer, Gentoo Lisp project
<URL:http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode

<URL:http://gentoo.faulhammer.org/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Nirbheek Chauhan]

On Wed, Jan 13, 2010 at 9:24 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> i think our current work flows also significantly impede the smooth running of
> this.  if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
> their behalf.
>

In theory, yes. In practice, git is too slow to handle 30,000 files.
Even simple operations like git add become painful even if you put the
whole of portage on tmpfs since git does a stat() on every single file
in the repository with every operation.

Simple test: do a git init followed by git add && git commit -m
"Initial commit" in your portage dir (.gitignore packages/ and
distfiles/)

Once this is done, you can test how it'll feel like to use a DSCM on
portage (without history). Unless you have a really fast SSD and
processor, you'll want to go back to the good old days of CVS with its
network-bound latencies on just 5-6 files in the current dir.

Besides this, there is the problem of accommodating people who use a
subtree of gentoo-x86, and those who don't want the entire CVS history
on their hard drives. In summation, robbat2 needs *our* help in the
following:

a) Push functionality in shallow clones (patches exist upstream)
b) Partial-tree checkouts (patches exist upstream)
c) Optimize git so it can handle 30,000 files
    - Maybe maintain a cache of directory timestamps and only stat()
directories?
    - Implement recursive timestamps on directories in various
filesystems and then in git (via xattrs perhaps)? People want to do
this for things like Tracker too. Prelim patches might exist.
d) Implement scripts/infra for people to fetch repository (shallow and
deep) bundles to initialize their local git clones (similar to portage
snapshots)
    - git clone from scratch taxes the server too much, just like
rsync from scratch
e) Server-side scripts for pushing to CIA.vc for pretty stats like we do in CVS
    - We want this for overlays right now too.
f) (Optional) Fix http cloning in git to make it "smarter" to help
people behind firewalls get anonymous clones (patches exist upstream)

Did I miss something Robin?

-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Markos Chandras]

[-- Attachment #1: Type: Text/Plain, Size: 1182 bytes --]

On Wednesday 13 January 2010 23:02:17 Ben de Groot wrote:
> 2010/1/13 Mike Frysinger <vapier@gentoo.org>:
> > On Tuesday 12 January 2010 15:35:45 Ben de Groot wrote:
> >> 2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
> >> > If you feel like it, become a proxy-maintainer and poke a developer to
> >> > put your ebuilds on tree. Have you ever heard of that ? :)
> >>
> >> Proxy-maintainership should be given a MUCH higher profile in Gentoo,
> >> in my opinion. It is a virtually unknown option.
> >
> > i think our current work flows also significantly impede the smooth
> > running of this.  if we had were using a dscm (git) on gentoo-x86, i feel
> > like it'd be a much smoother ride for Gentoo devs to pull from a proxy
> > maintainer and push on their behalf.
> > -mike
> 
> I couldn't agree more!
> 
My experience as developer this years implies that decisions about these 
things take way too long to get implemented ( or even discussed ). So imho, 
the best way to promote the proxy-maintainer thing, is individually using our 
blogs or any other means.
-- 
Markos Chandras (hwoarang)
Gentoo Linux Developer
Web: http://hwoarang.silverarrow.org

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Nguyen Thai Ngoc Duy]

On 1/14/10, Nirbheek Chauhan <nirbheek@gentoo.org> wrote:
> On Wed, Jan 13, 2010 at 9:24 PM, Mike Frysinger <vapier@gentoo.org> wrote:
>  > i think our current work flows also significantly impede the smooth running of
>  > this.  if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
>  > much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
>  > their behalf.
>  >
>
>
> In theory, yes. In practice, git is too slow to handle 30,000 files.
>  Even simple operations like git add become painful even if you put the
>  whole of portage on tmpfs since git does a stat() on every single file
>  in the repository with every operation.

What you need is "git update-index --assume-unchanged". That feature
was introduced exactly to reduce stat().

BTW, if you know you only work in certain directories, doing "git diff
--stat <dir>", "git diff --cached --stat <dir>" instead of "git
status" would also help. Make aliases for them ("git dis" and "git
dics" in my ~/.gitconfig) so you don't have to type full command every
time.

"git commit <dir>" and "git status <dir>" still do full tree lstat().
I can try to make a patch or two to reduce lstat() in such cases.

Does that help?
-- 
Duy

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Ben de Groot]

2010/1/14 Nirbheek Chauhan <nirbheek@gentoo.org>:
> In theory, yes. In practice, git is too slow to handle 30,000 files.
> Even simple operations like git add become painful even if you put the
> whole of portage on tmpfs since git does a stat() on every single file
> in the repository with every operation.
> [...]
> Besides this, there is the problem of accommodating people who use a
> subtree of gentoo-x86, and those who don't want the entire CVS history
> on their hard drives. In summation, robbat2 needs *our* help in the
> following:

Thanks! That was a nice overview of the remaining issues. Has anyone
tested Mercurial to see how it compares, especially with respect to
these issues?

Cheers,
-- 
Ben de Groot
Gentoo Linux developer (qt, media, lxde, desktop-misc)
______________________________________________________

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Ben de Groot]

2010/1/14 Markos Chandras <hwoarang@gentoo.org>:
> My experience as developer this years implies that decisions about these
> things take way too long to get implemented ( or even discussed ). So imho,
> the best way to promote the proxy-maintainer thing, is individually using our
> blogs or any other means.

I'm not saying we should wait for a move to a DVCS. That is obviously
going to take some time still. I think we should do both: promote the
proxy-maintenance possibility, and at the same time work on DVCS
migration, which will ultimately make such work easier.

Cheers,
-- 
Ben de Groot
Gentoo Linux developer (qt, media, lxde, desktop-misc)
______________________________________________________

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

["Paweł Hajdan, Jr."]

[-- Attachment #1: Type: text/plain, Size: 1506 bytes --]

On 1/14/10 1:49 PM, Nirbheek Chauhan wrote:
> Besides this, there is the problem of accommodating people who use a
> subtree of gentoo-x86, and those who don't want the entire CVS history
> on their hard drives. In summation, robbat2 needs *our* help in the
> following:
> 
> a) Push functionality in shallow clones (patches exist upstream)
> b) Partial-tree checkouts (patches exist upstream)
> c) Optimize git so it can handle 30,000 files
>     - Maybe maintain a cache of directory timestamps and only stat()
> directories?
>     - Implement recursive timestamps on directories in various
> filesystems and then in git (via xattrs perhaps)? People want to do
> this for things like Tracker too. Prelim patches might exist.
> d) Implement scripts/infra for people to fetch repository (shallow and
> deep) bundles to initialize their local git clones (similar to portage
> snapshots)
>     - git clone from scratch taxes the server too much, just like
> rsync from scratch
> e) Server-side scripts for pushing to CIA.vc for pretty stats like we do in CVS
>     - We want this for overlays right now too.
> f) (Optional) Fix http cloning in git to make it "smarter" to help
> people behind firewalls get anonymous clones (patches exist upstream)
> 
> Did I miss something Robin?

It would be nice to post that info to a webpage. That could increase a
chance of a volunteer contributing some help.

Note in advance: I don't know git internals, so can't help at this moment.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Pacho Ramos]

[-- Attachment #1: Type: text/plain, Size: 319 bytes --]

El jue, 14-01-2010 a las 18:04 +0100, "Paweł Hajdan, Jr." escribió:
> 
> It would be nice to post that info to a webpage. That could increase a
> chance of a volunteer contributing some help.

I agree, maybe that way other people (from forums for example) could
help if they know about git (or elected system)

[-- Attachment #2: Esta parte del mensaje está firmada digitalmente --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Daniel Bradshaw]

Excuse me butting in... I'm just a little confused.
Not that this is anything new, I'm just ... well, confused.

On 01/14/2010 12:49 PM, Nirbheek Chauhan wrote:
> In theory, yes. In practice, git is too slow to handle 30,000 files.
> Even simple operations like git add become painful even if you put the
> whole of portage on tmpfs since git does a stat() on every single file
> in the repository with every operation.
>    
My understanding is that git was developed as the SCM for the kernel 
project.
A quick check in an arbitary untouched kernel in /usr/src/ suggests a 
file [1] count of 25300.

Assuming that my figure isn't out by an order of magnitude, how does the 
kernel team get along with git and 25k files but it is deathly slow for 
our 30k?
Or, to phrase the question better... what are they doing that allows 
them to manage?

Regards,
Daniel

[1] `find -type f | wc -l`, so all regular files

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Nirbheek Chauhan]

On Thu, Jan 14, 2010 at 7:17 PM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> What you need is "git update-index --assume-unchanged". That feature
> was introduced exactly to reduce stat().
>
> BTW, if you know you only work in certain directories, doing "git diff
> --stat <dir>", "git diff --cached --stat <dir>" instead of "git
> status" would also help. Make aliases for them ("git dis" and "git
> dics" in my ~/.gitconfig) so you don't have to type full command every
> time.
>

This is very interesting; I did not know about this feature! Thanks
for pointing it out :)

I'll try this stuff out and report back once I have my portage tmpfs
created again.

> "git commit <dir>" and "git status <dir>" still do full tree lstat().
> I can try to make a patch or two to reduce lstat() in such cases.
>

That would definitely compliment the --stat option to git diff et al,
making git more usable on repos with a huge no. of files. Now that I
think about it, why does git <command> <dir> need to do a full tree
stat at all? Doesn't the added specification of <dir> mean "I'm only
interested in this dir for this command, other stuff doesn't matter"?

> Does that help?

Quite helpful indeed; now if only someone would implement recursive
timestamps for directories... ;)

-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Nirbheek Chauhan]

On Fri, Jan 15, 2010 at 2:01 AM, Daniel Bradshaw <daniel@the-cell.co.uk> wrote:
> On 01/14/2010 12:49 PM, Nirbheek Chauhan wrote:
>>
>> In theory, yes. In practice, git is too slow to handle 30,000 files.
>> Even simple operations like git add become painful even if you put the
>> whole of portage on tmpfs since git does a stat() on every single file
>> in the repository with every operation.
>>
>
> My understanding is that git was developed as the SCM for the kernel
> project.
> A quick check in an arbitary untouched kernel in /usr/src/ suggests a file
> [1] count of 25300.
>
> Assuming that my figure isn't out by an order of magnitude, how does the
> kernel team get along with git and 25k files but it is deathly slow for our
> 30k?
> Or, to phrase the question better... what are they doing that allows them to
> manage?
>

My bad. I did the tests a while back, and the number "30,000" is
actually for the no. of ebuilds in portage. The no. of files is
actually ~113,000 (difference comes because every package has a
manifest+changelog+metadata.xml+patches). OTOH, the no. of directories
is "just" ~20,000, so  if git would only do a stat() on directories,
it would get into the "usable" circle.

Also, since git does a stat on directories as well as files, you can
say that every command has to do ~133,000 stats, which is damn slow
even when cached.

-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Nirbheek Chauhan]

On Fri, Jan 15, 2010 at 3:51 AM, Nirbheek Chauhan <nirbheek@gentoo.org> wrote:
> My bad. I did the tests a while back, and the number "30,000" is
> actually for the no. of ebuilds in portage. The no. of files is
> actually ~113,000 (difference comes because every package has a
> manifest+changelog+metadata.xml+patches).

Further refinement: ~92,000

Removed metadata/ (-28,000 which won't be around in the git tree), and
ChangeLog (-13,000 which would be redundant, and should be
auto-generated alongwith metadata prior to distribution via rsync.
Hey, this is something that needs to be done too!)

-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Daniel Bradshaw]

On 01/14/2010 10:21 PM, Nirbheek Chauhan wrote:
> On Fri, Jan 15, 2010 at 2:01 AM, Daniel Bradshaw<daniel@the-cell.co.uk>  wrote:
>    
>> On 01/14/2010 12:49 PM, Nirbheek Chauhan wrote:
>>      
>>> In theory, yes. In practice, git is too slow to handle 30,000 files.
>>> Even simple operations like git add become painful even if you put the
>>> whole of portage on tmpfs since git does a stat() on every single file
>>> in the repository with every operation.
>>>
>>>        
>> My understanding is that git was developed as the SCM for the kernel
>> project.
>> A quick check in an arbitary untouched kernel in /usr/src/ suggests a file
>> [1] count of 25300.
>>
>> Assuming that my figure isn't out by an order of magnitude, how does the
>> kernel team get along with git and 25k files but it is deathly slow for our
>> 30k?
>> Or, to phrase the question better... what are they doing that allows them to
>> manage?
>>
>>      
> My bad. I did the tests a while back, and the number "30,000" is
> actually for the no. of ebuilds in portage. The no. of files is
> actually ~113,000 (difference comes because every package has a
> manifest+changelog+metadata.xml+patches). OTOH, the no. of directories
> is "just" ~20,000, so  if git would only do a stat() on directories,
> it would get into the "usable" circle.
>
> Also, since git does a stat on directories as well as files, you can
> say that every command has to do ~133,000 stats, which is damn slow
> even when cached.
>
>    

Ah, so one side is off by a fair bit.
Thanks for the clarification.

Regards,
Daniel

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Nirbheek Chauhan]

On Thu, Jan 14, 2010 at 10:34 PM, "Paweł Hajdan, Jr."
<phajdan.jr@gentoo.org> wrote:
> It would be nice to post that info to a webpage. That could increase a
> chance of a volunteer contributing some help.
>

That list is incomplete, a more complete todo list can be found by
looking at the archives at archives.gentoo.org/gentoo-scm/

I'll compile a proper list and put it up somewhere, thanks for the suggestion.


-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 2423 bytes --]

On Fri, Jan 15, 2010 at 03:59:00AM +0530, Nirbheek Chauhan wrote:
> On Fri, Jan 15, 2010 at 3:51 AM, Nirbheek Chauhan <nirbheek@gentoo.org> wrote:
> > My bad. I did the tests a while back, and the number "30,000" is
> > actually for the no. of ebuilds in portage. The no. of files is
> > actually ~113,000 (difference comes because every package has a
> > manifest+changelog+metadata.xml+patches).
> Further refinement: ~92,000
> 
> Removed metadata/ (-28,000 which won't be around in the git tree), and
> ChangeLog (-13,000 which would be redundant, and should be
> auto-generated alongwith metadata prior to distribution via rsync.
> Hey, this is something that needs to be done too!)
ChangeLog != commit logs.

There is frequently additional information in the CVS commit messages
that isn't in the ChangeLogs. ChangeLogs aren't always updated reliably
either, esp for ebuild cleanups (hi vapier).

The actual performance of git itself isn't the largest problem.
The migration issues, esp. the speed of the conversion are.

My status of the migration side itself hasn't changed since the end of
October:
http://archives.gentoo.org/gentoo-scm/msg_e0a0a41200c1fc6a0fda68b4ff9d2c61.xml

That top item is the largest blocker. The actual conversion time is down
to 9 hours, but with more than that again in setting it up. I'd like to
get the conversion time down to UNDER 4 hours. It's mostly
single-threaded, and we've got lots of cores available, it just needs
parallelization. We're basically dead in the water during the
conversion, there is NO incremental support at all.

Side-project to the above: Is there anything link Psyco for Python
acceleration that works on 64-bit machines? Psyco itself has a warning
on the frontpage of no 64-bit support.

pre-upload-hook: ford_prefect, can I have something by Jan 21st please?
Even just the core C infrastructure for the hook.

For the partial tree users, the support is actually IN 1.6.6 series. It
needs a little more cooking I think however, there are some patches
queued for it, that will probably end up in 1.6.6.1 or similar.

We DO still need somebody that cares about CVS access to test with
git-cvssserver against the existing conversion.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 330 bytes --]

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 660 bytes --]

On 01/15/2010 12:54 AM, Robin H. Johnson wrote:
> 
> That top item is the largest blocker. The actual conversion time is down
> to 9 hours, but with more than that again in setting it up. I'd like to
> get the conversion time down to UNDER 4 hours. It's mostly
> single-threaded, and we've got lots of cores available, it just needs
> parallelization. We're basically dead in the water during the
> conversion, there is NO incremental support at all.
> 

Is a day really an issue? If there's something extremely urgent while
the conversion is going on, you can just turn CVS back on and then let's
try again some other day.

Regards,
Petteri


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Nirbheek Chauhan]

On Fri, Jan 15, 2010 at 4:24 AM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> On Fri, Jan 15, 2010 at 03:59:00AM +0530, Nirbheek Chauhan wrote:
>> ChangeLog (-13,000 which would be redundant, and should be
>> auto-generated alongwith metadata prior to distribution via rsync.
>> Hey, this is something that needs to be done too!)
> ChangeLog != commit logs.
>
> There is frequently additional information in the CVS commit messages
> that isn't in the ChangeLogs. ChangeLogs aren't always updated reliably
> either, esp for ebuild cleanups (hi vapier).
>

All the more reason to just chuck manual ChangeLogs in favour of
auto-generated ones. Atleast, that's what gnome projects do since they
moved to git [ http://live.gnome.org/Git/ChangeLog ].

However, this will entail a change in how commit messages are
formatted; git commit messages need to be very different from CVS/svn
ones.

> The actual performance of git itself isn't the largest problem.
> The migration issues, esp. the speed of the conversion are.
>
> My status of the migration side itself hasn't changed since the end of
> October:
> http://archives.gentoo.org/gentoo-scm/msg_e0a0a41200c1fc6a0fda68b4ff9d2c61.xml
>
> That top item is the largest blocker. The actual conversion time is down
> to 9 hours, but with more than that again in setting it up. I'd like to
> get the conversion time down to UNDER 4 hours. It's mostly
> single-threaded, and we've got lots of cores available, it just needs
> parallelization. We're basically dead in the water during the
> conversion, there is NO incremental support at all.
>

Actually, this has confused me for a while. Sorry if this is a dumb
question, but why do we care about the conversion speed if we can just
convert it once, make the old cvs repo read-only and be done with it?
Are we concerned about the window during which cvs access will have to
be blocked and devs will sit around twiddling thumbs?

> Side-project to the above: Is there anything link Psyco for Python
> acceleration that works on 64-bit machines? Psyco itself has a warning
> on the frontpage of no 64-bit support.
>

I had written Pysco off as dead and started looking forward to Unladen
Swallow :)

http://code.google.com/p/unladen-swallow/wiki/ProjectPlan

> We DO still need somebody that cares about CVS access to test with
> git-cvssserver against the existing conversion.
>

This will fit in quite badly with the proposed changes to make
Manifest have only distfile manifests when using git, and to not have
ChangeLogs for the simple reason that they (Manifests and ChangeLogs)
invariably cause merge conflicts. And then there was also the plan to
not edit headers for files to prevent extra commits (which are even
more useless in git).


-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

[gentoo-dev] Re: proxy maintainership and gentoo-x86 scm

[Paul Arthur]

On 2010-01-14, Nirbheek Chauhan <nirbheek@gentoo.org> wrote:
> On Wed, Jan 13, 2010 at 9:24 PM, Mike Frysinger <vapier@gentoo.org> wrote:
>> i think our current work flows also significantly impede the smooth running of
>> this.  if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
>> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
>> their behalf.
>
> In theory, yes. In practice, git is too slow to handle 30,000 files.
> Even simple operations like git add become painful even if you put the
> whole of portage on tmpfs since git does a stat() on every single file
> in the repository with every operation.
>
> Simple test: do a git init followed by git add && git commit -m
> "Initial commit" in your portage dir (.gitignore packages/ and
> distfiles/)
>
> Once this is done, you can test how it'll feel like to use a DSCM on
> portage (without history). Unless you have a really fast SSD and
> processor, you'll want to go back to the good old days of CVS with its
> network-bound latencies on just 5-6 files in the current dir.

Ouch.  I wanted to test this in a fairly bad scenario, so I gave it a
try on my old, low-spec fileserver.

aravis-root /usr/portage # time git add .
real    19m1.333s
user    0m53.230s
sys     1m9.350s

aravis-root /usr/portage # time git commit -m "Initial commit"
real    19m44.700s
user    0m23.740s
sys     0m49.320s

Then, with no changes whatsoever:
aravis-root /usr/portage # time git status
real    4m26.454s
user    0m2.090ssys     0m6.380s

Finally, a ray of hope:
aravis-root /usr/portage/app-emulation # time git add xen-tools-gdbserver/
real    0m0.978s
user    0m0.400s
sys     0m0.180s

But no:
aravis-root /usr/portage/app-emulation # time git commit -m "Commit 2"
real    3m18.502s
user    0m1.830s
sys     0m7.530s

Now, this is fairly close to a worst-case scenario, being an old
computer with a slow drive. A new computer with faster drives (in
RAID 1+0) is much more reasonable.

lasaraleen portage # time git add .
real    0m26.002s
user    0m6.256s
sys     0m6.572s

lasaraleen portage # time git commit -m "Initial commit"
real    0m27.371s
user    0m3.704s
sys     0m3.856s

lasaraleen app-emulation # time git commit -m "Commit 2"
real    0m1.374s
user    0m0.468s
sys     0m0.904s


-- 
Having to infer what Unix is solely from a copy of the GNU Manifesto is
not really an exercise you want to undertake. 
    --AdB

Re: [gentoo-dev] Re: proxy maintainership and gentoo-x86 scm

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 2132 bytes --]

On Thu, Jan 14, 2010 at 07:07:01PM -0500, Paul Arthur wrote:
> On 2010-01-14, Nirbheek Chauhan <nirbheek@gentoo.org> wrote:
> > On Wed, Jan 13, 2010 at 9:24 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> >> i think our current work flows also significantly impede the smooth running of
> >> this.  if we had were using a dscm (git) on gentoo-x86, i feel like it'd be a
> >> much smoother ride for Gentoo devs to pull from a proxy maintainer and push on
> >> their behalf.
> >
> > In theory, yes. In practice, git is too slow to handle 30,000 files.
> > Even simple operations like git add become painful even if you put the
> > whole of portage on tmpfs since git does a stat() on every single file
> > in the repository with every operation.
> >
> > Simple test: do a git init followed by git add && git commit -m
> > "Initial commit" in your portage dir (.gitignore packages/ and
> > distfiles/)
> >
> > Once this is done, you can test how it'll feel like to use a DSCM on
> > portage (without history). Unless you have a really fast SSD and
> > processor, you'll want to go back to the good old days of CVS with its
> > network-bound latencies on just 5-6 files in the current dir.
> 
> Ouch.  I wanted to test this in a fairly bad scenario, so I gave it a
> try on my old, low-spec fileserver.
You didn't repack or at least run git-gc between the huge add and your
everyday operations. Do that, and then measure the ops with both cold
and hot cache. 

The initial packing and adding are very intensive, even on fast
machines, but that's because they are dealing with a lot of small pieces
of data. Packing has benefited immensely from being fully multi-threaded
in Git.

I'd love somebody to do the SoC stats again:
http://www.gentoo.org/proj/en/infrastructure/cvs-migration.xml?style=printable

Using the git repo conversion I did:
http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 330 bytes --]

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Max Arnold]

On Thu, Jan 14, 2010 at 07:07:01PM -0500, Paul Arthur wrote:
> Ouch.  I wanted to test this in a fairly bad scenario, so I gave it a
> try on my old, low-spec fileserver.

Just out of curiosity did several tests with Mercurial:

$ mkdir /scratch/tmp
$ time tar --use-compress-program=lzma -xf portage-20100114.tar.lzma -C /scratch/tmp 
real    1m3.696s
user    0m25.082s
sys     0m12.549s

$ cd /scratch/tmp/portage
$ hg init --time
Time: real 0.070 secs (user 0.040+0.000 sys 0.000+0.000)

$ hg status --time | wc -l
Time: real 9.920 secs (user 6.290+0.000 sys 1.970+0.000)
113272

$ hg add --time | wc -l
Time: real 23.050 secs (user 20.450+0.000 sys 2.300+0.000)
113272

$ hg commit --time -m "Initial commit"
Time: real 758.010 secs (user 354.250+0.000 sys 93.400+0.000)

$ cd ..
$ hg clone --noupdate --time portage portage-work
Time: real 34.530 secs (user 9.160+0.000 sys 9.750+0.000)

$ cd portage-work
$ hg update --time
113272 files updated, 0 files merged, 0 files removed, 0 files unresolved
Time: real 538.330 secs (user 218.140+0.000 sys 74.520+0.000)

$ mkdir dev-util/hg-test
$ touch dev-util/hg-test/Manifest
$ hg status --time        
? dev-util/hg-test/Manifest
Time: real 6.350 secs (user 4.520+0.000 sys 1.310+0.000)

$ hg add --time
adding dev-util/hg-test/Manifest
Time: real 10.250 secs (user 8.610+0.000 sys 1.390+0.000)

$ hg commit --time -m "added hg-test"
Time: real 17.370 secs (user 15.400+0.000 sys 1.430+0.000)

$ hg out --time ../portage | grep changeset | wc -l
Time: real 0.930 secs (user 0.690+0.000 sys 0.070+0.000)
1

$ hg push --time ../portage
pushing to ../portage
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
Time: real 6.100 secs (user 5.450+0.000 sys 0.330+0.000)

$ cd ../portage
$ hg update --time
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
Time: real 96.390 secs (user 14.950+0.000 sys 5.420+0.000)

$ hg log --time | grep changeset | wc -l
Time: real 0.530 secs (user 0.460+0.000 sys 0.060+0.000)
2

This is on a rather slow box (nettop with VIA C7 1200 MHz CPU, 1G RAM and 5400 RPM 2.5" drive)

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 948 bytes --]

On Thursday 14 January 2010 18:25:35 Petteri Räty wrote:
> On 01/15/2010 12:54 AM, Robin H. Johnson wrote:
> > That top item is the largest blocker. The actual conversion time is down
> > to 9 hours, but with more than that again in setting it up. I'd like to
> > get the conversion time down to UNDER 4 hours. It's mostly
> > single-threaded, and we've got lots of cores available, it just needs
> > parallelization. We're basically dead in the water during the
> > conversion, there is NO incremental support at all.
> 
> Is a day really an issue? If there's something extremely urgent while
> the conversion is going on, you can just turn CVS back on and then let's
> try again some other day.

indeed ... we've already tolerated long downtimes in different infrastructure 
pieces in our history.  devs can live with advertised downtimes since the 
timing is known.  it's the unknown/indefinite things that annoy people.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Nguyen Thai Ngoc Duy]

On 1/15/10, Nirbheek Chauhan <nirbheek@gentoo.org> wrote:
>  > "git commit <dir>" and "git status <dir>" still do full tree lstat().
>  > I can try to make a patch or two to reduce lstat() in such cases.
>  >
>
>
> That would definitely compliment the --stat option to git diff et al,
>  making git more usable on repos with a huge no. of files. Now that I
>  think about it, why does git <command> <dir> need to do a full tree
>  stat at all? Doesn't the added specification of <dir> mean "I'm only
>  interested in this dir for this command, other stuff doesn't matter"?

Probably because the difference is too small to notice on smaller-size
projects, or because people tend to do whole-tree operations so "git
<command> <dir>"'s performance does not catch the developers' eyes.

Anyway, stat()ing 80k files takes about 1 second on my machine, still
tolerable. There is whole-tree open() in "git status" to check for
untracked files, that contributes more on "git status" slowness. How
long on average did a Git operation take on your tmpfs?
-- 
Duy

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Victor Ostorga]

On Tue, 12 Jan 2010 21:35:45 +0100
Ben de Groot <yngwin@gentoo.org> wrote:

> 2010/1/12 Markos Chandras <hwoarang@gentoo.org>:
> > If you feel like it, become a proxy-maintainer and poke a developer
> > to put your ebuilds on tree. Have you ever heard of that ? :)
> 
> Proxy-maintainership should be given a MUCH higher profile in Gentoo,
> in my opinion. It is a virtually unknown option.
> 
> Another thing that works in my experience, but this is up to the
> herds/projects, is having an official overlay where devs and users can
> work closely together. This allows users to commit ebuilds and
> patches, while there is quality control by the involved devs. Devs can
> keep an eye on such overlays and move stuff to portage when they are
> ready. Sunrise is the most obvious example for this, for
> maintainer-wanted packages. But it works equally well for us in the Qt
> project with qting-edge, and I believe kde and pro-audio have the same
> experience.
> 
> But I also believe we need a better structure to handle
> maintainer-needed, maintainer-wanted and nominally maintained but
> ignored packages. Maybe we should form a team, which would be
> dedicated to take care of such things, and which would have a review
> policy for user submitted ebuilds and patches in bugzilla. A bit like
> treecleaners, but bringing life instead of death. What do you think?
> 

I agree with the creation of a maintainer-needed team, but to be
honest, all those packages ended in maintainer-needed by lack of
manpower.

Even with the manpower problem, there are people helping with those
packages, such as Patrick, Samuli, Diego and myself. 

I am in the maintainer-needed CC so I always try to commit patches and
do fixes to those packages, although lately I have not had much time
available. So, if anyone have a patch for a maintainer-needed package,
feel free to ping me :)

Víctor

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Ben de Groot]

I think we have a bigger problem with packages that have a maintainer,
at least nominally, but said maintainer does not actually maintain the
package anymore.

-- 
Ben de Groot
Gentoo Linux developer (qt, media, lxde, desktop-misc)
______________________________________________________

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Thilo Bangert]

[-- Attachment #1: Type: Text/Plain, Size: 651 bytes --]

Ben de Groot <yngwin@gentoo.org> said:
> I think we have a bigger problem with packages that have a maintainer,
> at least nominally, but said maintainer does not actually maintain the
> package anymore.

full ack. i was thinking that maybe we need an 'easy-fix' team, which can 
do all the easy fixes, which have been laying around for way too long and 
which aparently are "easy to fix" (ie. only waiting for somebody to 
commit)...

the details would stil have to be thought through, but anything which 
improves the felt responsitivity for our users can only be good.
Did you know: Gentoo is dying! ;-)

kind regards
Thilo

> 


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Richard Freeman]

On 01/17/2010 03:20 PM, Thilo Bangert wrote:
> Ben de Groot<yngwin@gentoo.org>  said:
>> I think we have a bigger problem with packages that have a maintainer,
>> at least nominally, but said maintainer does not actually maintain the
>> package anymore.
>
> full ack. i was thinking that maybe we need an 'easy-fix' team, which can
> do all the easy fixes, which have been laying around for way too long and
> which aparently are "easy to fix" (ie. only waiting for somebody to
> commit)...

I think this is a good idea.  Alternatively, perhaps if we just make a 
policy that it is acceptable for a dev to touch a package and leave it 
with QA problems, as long as it ends up with no more QA problems than it 
started with.  Of course, devs are encouraged to fix QA problems 
whenever they can.

This way devs will be more willing to make small fixes that generally 
improve the distro without being worried about being chewed out because 
they didn't go through the package with a fine-tooth comb looking for 
issues.

That will at least get poorly-maintained packages trending in the right 
direction.  Along with that I think we need to address the problem of 
packages that list a maintainer but which aren't maintained.

Perhaps a solution would be to have some way to periodically ping 
maintainers to confirm they're still looking at a package.  That could 
take many forms - a simple one would be to ask the maintainer to touch 
the metadata.xml file or something like that (obviously the manipulation 
has to be something that is noticed by CVS).  On the other hand we don't 
want needless churn.  Then an automated process can ping maintainers if 
they haven't done this, and after a delay the package would be set to 
maintainer-wanted.

Actively-maintained projects would be allowed to use automation to keep 
packages they track marked fresh.  However, the project does then become 
accountable for actually maintaining them.

This would go along with an (unwritten but already in place) policy that 
anybody listed as a maintainer has to actually maintain the package, 
with consequences for not doing so to some defined standard.  This 
standard would not be zero-bugs, but certainly anything real flagged by 
repoman or a violation of a written QA policy would be expected to be 
cleaned up in a timely manner.

The goal is to make the maintainer field meaningful.

Then we could figure out a policy for maintainer-wanted builds.  My 
feeling is that as long as they build, don't have security issues, and 
don't have QA issues that genuinely get in the way of some Gentoo 
initiative, they can stay around.  Once any of the above ceases to be 
the case they're announced on-list and then treecleaned.

The bottom line though is that we can write all the rules we want - 
ultimately Gentoo needs warm bodies to fix bugs.  No amount of process 
will get around that.  What process can do for us, however, is to 
increase visibility of issues so that QA doesn't waste time hunting down 
inactive devs and so that people running servers or whatever can tell if 
a package is actively maintained.

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 625 bytes --]

On Sunday 17 January 2010 15:20:46 Thilo Bangert wrote:
> Ben de Groot <yngwin@gentoo.org> said:
> > I think we have a bigger problem with packages that have a maintainer,
> > at least nominally, but said maintainer does not actually maintain the
> > package anymore.
> 
> full ack. i was thinking that maybe we need an 'easy-fix' team, which can
> do all the easy fixes, which have been laying around for way too long and
> which aparently are "easy to fix" (ie. only waiting for somebody to
> commit)...

when the bug wranglers re-assign to maintainer-needed, have them tag it with 
SIMPLE or something
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Thilo Bangert]

[-- Attachment #1: Type: Text/Plain, Size: 1368 bytes --]

Mike Frysinger <vapier@gentoo.org> said:
> On Sunday 17 January 2010 15:20:46 Thilo Bangert wrote:
> > Ben de Groot <yngwin@gentoo.org> said:
> > > I think we have a bigger problem with packages that have a
> > > maintainer, at least nominally, but said maintainer does not
> > > actually maintain the package anymore.
> >
> > full ack. i was thinking that maybe we need an 'easy-fix' team, which
> > can do all the easy fixes, which have been laying around for way too
> > long and which aparently are "easy to fix" (ie. only waiting for
> > somebody to commit)...
> 
> when the bug wranglers re-assign to maintainer-needed, have them tag it
>  with SIMPLE or something

no - i wasn't talking about maintainer-needed bugs. it is my impression, 
that many apparently maintained packages have simple bugs lingering for 
extended periods of time. Fx. users reporting bugs AND supplying fixes, 
ready to be committed.

i dont want to step on anyones toes, which is why this may need to be put 
in place carefully - but i do think it would improve average quality of 
the tree.

the easy stuff, that is maintainer-needed is done regularly by a number of 
people. of course, "easy stuff" is different for different people.

the SIMPLE keyword would definitively work, though. can somebody add it to 
bugzilla?

thanks
Thilo

> -mike
> 


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Ben de Groot]

2010/1/17 Thilo Bangert <bangert@gentoo.org>:
> no - i wasn't talking about maintainer-needed bugs. it is my impression,
> that many apparently maintained packages have simple bugs lingering for
> extended periods of time. Fx. users reporting bugs AND supplying fixes,
> ready to be committed.
>
> i dont want to step on anyones toes, which is why this may need to be put
> in place carefully - but i do think it would improve average quality of
> the tree.

What about something like: if a bug has been open for 2 months without
any apparent maintainer activity, anyone can step in and commit a fix?

Cheers,
-- 
Ben de Groot
Gentoo Linux developer (qt, media, lxde, desktop-misc)
______________________________________________________

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Richard Freeman]

On 01/17/2010 08:23 PM, Ben de Groot wrote:
> What about something like: if a bug has been open for 2 months without
> any apparent maintainer activity, anyone can step in and commit a fix?
>

How about - anybody at any time can at their discretion post a comment 
in a bug asking if there are objections to committing a fix.  If there 
is no reply from the maintainer/project/etc in two days go ahead and do 
it.  Do check devaway first to see if it makes sense to wait a little 
longer, and use discretion on the more critical packages.

Re: [gentoo-dev] proxy maintainership and gentoo-x86 scm

[Arun Raghavan]

2010/1/15 Robin H. Johnson <robbat2@gentoo.org>:
[...]
> pre-upload-hook: ford_prefect, can I have something by Jan 21st please?
> Even just the core C infrastructure for the hook.

Sorry about dropping the ball on this for so long - 21st would be
hard, but I should be able to get this to you by the weekend.

Cheers,
-- 
Arun Raghavan
http://arunraghavan.net/
(Ford_Prefect | Gentoo) & (arunsr | GNOME)

Re: [gentoo-dev] Re: Last rites: net-nntp/inn

[Petteri Räty]

On 18.1.2010 4.17, Richard Freeman wrote:
> On 01/17/2010 08:23 PM, Ben de Groot wrote:
>> What about something like: if a bug has been open for 2 months without
>> any apparent maintainer activity, anyone can step in and commit a fix?
>>
>
> How about - anybody at any time can at their discretion post a comment
> in a bug asking if there are objections to committing a fix. If there is
> no reply from the maintainer/project/etc in two days go ahead and do it.
> Do check devaway first to see if it makes sense to wait a little longer,
> and use discretion on the more critical packages.
>
>

This process is already the current policy although longer than two days 
is prudent. For urgent fixes you wouldn't follow this process any way 
and for others it doesn't hurt to wait.

Regards,
Petteri