[gentoo-dev] Gentoo: State of the Union

[gentoo-dev] Gentoo: State of the Union

[Molle Bestefich]

> > Having a live tree requires people to be perfect.  People are not
> > perfect and requiring it is ridiculous.  I love having commits in my
> > local tree within the hour, but having a stable and unstable branch
> > makes a lot of sense.
>
> Does it?  How does having a stable and unstable branch differ from
> having stable and unstable keywords?

Agreed.  That doesn't make sense.

Subversion has what is known as pre-commit hooks.
Using those it's very easy to:
 * prevent (most or some) committers from designating ebuilds as stable
 * allow committers to designate ebuilds as stable under a certain path only
 * strictly limit a commiters access to a part of the tree

Slightly harder, but could be done too:
 * deny commits if it breaks ebuild dependencies

If you want central control of what's happening in the repository,
Subversion seems like the way to go.

> SVN requires at least 2x the tree size for storage on the local machine,

This is a feature, not a bug.
It allows you to keep operations local, fx. do a diff without being
online with the repository.

> checkouts take something akin to an order of magnitude longer than CVS.

In the past Subversion performance was sub-par.  I haven't
systematically tested performance, but I would expect it to be much
better now.

Performance is gradually improved, see fx. r18867, r18944 and r19420:
http://svn.collab.net/viewvc/svn?view=rev&revision=18867
http://svn.collab.net/viewvc/svn?view=rev&revision=18944
http://svn.collab.net/viewvc/svn?view=rev&revision=19420

GIT might perform better, since Linus specifically emphasized
non-O(n^2) performance in the design.  But being a decentralized tool,
I'm not sure how well it fits the needs here.

> The former is annoying, but liveable, but
> the latter is a deal-breaker.

I don't think so.  You really rarely do a complete checkout.

> - No changeset/merge tracking

Solutions exist, including svnmerge and svk.
"Official" solution actively worked on at the moment, check out the
svn dev list.

A benefit of Subversion that I personally like is that FSFS
repositories are extremely easy to rsync.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo: State of the Union

[Stuart Herbert]

Quoting Molle Bestefich <molle.bestefich@gmail.com>:

>> Does it?  How does having a stable and unstable branch differ from
>> having stable and unstable keywords?
>
> Agreed.  That doesn't make sense.

It does if you have a separate stable tree per-arch.  With the current  
tree design, it's too easy to break an arch that you've no intention  
of affecting.

However, if you have a separate tree per-arch, that tree can be tested  
and approved for release as a single unit.

 From a SCM point of view, arches are a subset of the full Gentoo  
tree.  They would fit very well into a branching model - and  
Subversion's support for branching would make it a breeze for us to  
support without overloading the arch teams.

> If you want central control of what's happening in the repository,
> Subversion seems like the way to go.

Better to fix the design of the repository, so that it can't be broken  
in the first place, than to try putting band-aids in place to cover  
the gaps.

> I don't think so.  You really rarely do a complete checkout.

I'd be interested to see some stats from our CVS logs to back that up.

Best regards,
Stu
-- 
Stuart Herbert                                         stuart@gentoo.org
Gentoo Developer                                  http://www.gentoo.org/
                                           http://blog.stuartherbert.com/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo: State of the Union

[Thomas Cort]

[-- Attachment #1: Type: text/plain, Size: 775 bytes --]

On Thu, 04 May 2006 11:44:18 +0100
Stuart Herbert <stuart@gentoo.org> wrote:
> However, if you have a separate tree per-arch, that tree can be tested  
> and approved for release as a single unit.

How big would this tree be? Would it be every package? How will this make the arch teams' life easier and/or the users' experience better? I still fail to see how this will work any differently than what we have today. Today we have ~arch, someone tests the package for stability on an arch system, and marks it stable. The tester is effectively integrating the package into a stable system. Maybe you could explain how the procedure might go with branches for getting a package from just put into the tree to ~arch to arch and what happens with version bumps.

~tcort

[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Gentoo: State of the Union

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]

On Thu, 2006-05-04 at 11:44 +0100, Stuart Herbert wrote:
>  From a SCM point of view, arches are a subset of the full Gentoo  
> tree.  They would fit very well into a branching model - and  
> Subversion's support for branching would make it a breeze for us to  
> support without overloading the arch teams.

Are you kidding me?  What about people that commit for multiple arches?
They're now going to have to do the same commit over $x number of trees?
How exactly will that not overload the arch teams?

The more I hear about all of these great features of qall of these
alternative SCM's, the more I think that somebody just has a hard-on for
getting rid of CVS and plans on doing it, no matter the cost to
efficiency and other developers.  No, I'm pointing to anyone in
particular.  It just seems that everyone wants to blame CVS for our
problems, when our problems are almost entirely cultural.

Seriously, if I were forced to commit to multiple trees, or branches, or
whatever, I'd simply leave the project since it would be such an
enormous waste of time.  I'm sure lots of others feel the same.  Our
version control system is supposed to be a tool to help us get our work
done, not a hindrance.

-- 
Chris Gianelloni
Release Engineering - Strategic Lead
x86 Architecture Team
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Gentoo: State of the Union

[Stuart Herbert]

On 5/4/06, Chris Gianelloni <wolf31o2@gentoo.org> wrote:
> On Thu, 2006-05-04 at 11:44 +0100, Stuart Herbert wrote:
> >  From a SCM point of view, arches are a subset of the full Gentoo
> > tree.  They would fit very well into a branching model - and
> > Subversion's support for branching would make it a breeze for us to
> > support without overloading the arch teams.
>
> Are you kidding me?  What about people that commit for multiple arches?
> They're now going to have to do the same commit over $x number of trees?
> How exactly will that not overload the arch teams?

Talking about an SVN perspective ... provided the trees live in a
single repository (which would make a lot of sense), it would be very
straightforward to provide a tool to copy a particular ebuild & its
files from an unstable tree simultaneously to the other trees.  The
difficulties with such a tool would be taking over the right files/
contents (something which is solveable), and what to do about signing
(where a distribtued system like Git probably makes much more sense
anyway).

Given such a tool, you could promote a version of a package to any
number of per-arch trees at the same time.

> The more I hear about all of these great features of qall of these
> alternative SCM's, the more I think that somebody just has a hard-on for
> getting rid of CVS and plans on doing it, no matter the cost to
> efficiency and other developers.

What we're talking about here is a step in the development cycle
commonly called 'integration', where something's taken from the
development bucket, put into the 'release' bucket, and tested to
ensure that it plays nice with everything else already in the
'release' bucket.  It should be listed in RUP, CMM, or whatever
development methodology you use locally in your day job.

Adopting this approach would be far more painful with CVS than with
(say) subversion.  Branch management in subversion is infinitely
easier than with CVS.

Best regards,
Stu

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Gentoo: State of the Union

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 884 bytes --]

On Thursday 04 May 2006 14:17, Stuart Herbert wrote:
> Talking about an SVN perspective ... provided the trees live in a
> single repository (which would make a lot of sense), it would be very
> straightforward to provide a tool to copy a particular ebuild & its
> files from an unstable tree simultaneously to the other trees.  The
> difficulties with such a tool would be taking over the right files/
> contents (something which is solveable), and what to do about signing
> (where a distribtued system like Git probably makes much more sense
> anyway).

The files content problem should be solved anyway. That way one could even 
make selfcontained ebuild packages including source. A way of 
distributing that would be more appropriate for external ebuilds.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

[gentoo-dev] Re: Gentoo: State of the Union

[Molle Bestefich]

> > If you want central control of what's happening in the repository,
> > Subversion seems like the way to go.
>
> Better to fix the design of the repository, so that it can't be
> broken in the first place, than to try putting band-aids in place
> to cover the gaps.

Wow, way out of scope.
Sure, a Portage that's inherently unbreakable would be nice.
And infinitely more difficult to carry out than what I was proposing.

Also, the two (QA inline in commit process vs. Perfect Portage) does
not preclude each other.  While one may be better long term you
can still do the other to improve quality short term.

(Not that you would need to - if you had a QA step in the commit
 process to make sure that ebuild dependencies were never broken,
 why would you want to create a whole new Portage?

I was trying to say that a QA tool in form of a SVN pre-commit hook
seems like a perfect fit.  The entire infrastructure to run an
external application to check a commit before carrying it out,
approve the commit, send appropriate error messages back to the SCM
client and display them to the user etc. is already in place (and
has been for years) in the Subversion server and any of the client
tools.

The amount of work involved between inventing a couple of rules that
committed ebuilds must obey to actually implementing an enforcement
of said rules is _very_ overcomeable with SVN.

I wasn't even proposing that Gentoo actually needs a QA tool inline
in the process, just saying that it can easily be done [with SVN] :-).


PS. I resent calling it a "band-aid".  It'd be a QA tool built on the
very well-thought-out foundation for quality assurance of commits that
SVN pre-commit hooks happens to be.

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: Gentoo: State of the Union

[Jan Kundrát]

[-- Attachment #1: Type: text/plain, Size: 605 bytes --]

Molle Bestefich wrote:
> I was trying to say that a QA tool in form of a SVN pre-commit hook
> seems like a perfect fit.  The entire infrastructure to run an
> external application to check a commit before carrying it out,
> approve the commit, send appropriate error messages back to the SCM
> client and display them to the user etc. is already in place (and
> has been for years) in the Subversion server and any of the client
> tools.

It's already possible with CVS (unless the gentoo/xml/htdocs/ uses some
uber-tweaked implementation :) ).

Cheers,
-jkt

-- 
cd /local/pub && more beer > /dev/mouth

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 258 bytes --]

[gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

[-- Attachment #1: Type: text/plain, Size: 5634 bytes --]

This is a follow up to Mark's (halcy0n's) thread regarding QA Policies and
seemant's letter on herds, teams, and projects.

I believe the way Gentoo is doing things is broken.  There I have said it.  The
entire project has reached a level of being too political and trying to solve
certain problems in the wrong way.

Some of these problems are intermixed.  Please consider them starting points
for discussion.

__Problem: Developer Growth__

I find that developer growth as being a problem.  Adding a developer to gentoo
should be as easy as 1. has the user contributed numerous (~5+) patches that
helps the project move forward.  If yes, then commit access should be given.
Adding a developer is usually quite a chore.  There are numerous reasons why
this is a problem: having a live tree, taking a test, and not defining within
policy when a person could possibly get commit access. 

All these reasons leave the project stagnant and lacking developers.

Why do people have to take a test?  Is it to make sure they won't break the
tree?  If it is, then the solution of a test is wrong.  We do want to make sure
our developers understand gentoo, but I argue that the bugtracker is all we
need.  As long as a person is adding value to gentoo and they have "proven"
themselves, then they *should* have commit access. 

Perhaps its because of a live tree...

__Problem: Live Tree__

Having a live tree requires people to be perfect.  People are not perfect and
requiring it is ridiculous.  I love having commits in my local tree within the
hour, but having a stable and unstable branch makes a lot of sense.  

CVS doesn't do branching nor tags very well... 

__Problem: CVS__

CVS is one of the worst application ever created.  The portage tree needs to
move to subversion.  A lot of the problems within the project would be solved
by using a better SCM system.  The previous problems regarding the Live Tree
and Developer Growth would be solved, IMHO, by just switching.  Branches Work.
Tags Work.  Reverts work.  Moves work.  I don't see any reason not to use it.
It just plain works.

Projects (gentoo/bsd, embedded, hardened) could choose to keep their own
branches of the portage tree and merge with trunk as needed.  Projects could
stick to traditional solutions like profiles if they so wished. 

Some will probably ask who will merge between branches.  We can do that easily
ourselves.  If I think a package is good to go, then svn merge -r1123:1124 to
the branch. 

Huge projects like Apache, GCC, and KDE already use SVN.

__Problem: QA Policies__ 

http://article.gmane.org/gmane.linux.gentoo.devel/37544

It seems that the QA Policies are a product of a Live Tree, and going partially
non-live would solve the problems listed. 

Everyone here is on the same team.  There will be some breakages in the tree
and those can be dealt with.  Like Seemant [1] said, herds are just groups of
like *packages*.  The QA Policy is wrong when it says cross-team assistance; we
are all on the *same* team.  The tree should naturally work.  If it doesn't
then that is a bug for all of us.

Conflict resolution should not be a subproject.  It should *not* exist at all.
Rules need to be in place to avoid conflict.  Having some sort of voting
structure for all the developers (this doesn't mean requiring everyone to vote)
and not just the council or devrel makes a lot of sense for most things.  If I
don't like how someone is acting within the project there should be a vote and
then see if that person is kicked out.  No trial, no anything besides a vote.
And if I lose I have to deal with it.  Either stay with the project, or find
something else.  This solution just works.

Gentoo should be a fun environment.  The previous paragraph should be taken as
a last resort.

__Problem: GLEPs__

I dislike GLEPs.  Usually they sit on the website for a long long time not
doing anything.  My vote (+1) is get rid of gleps and do everything by email
and a vote by the developers.  AFAIK, the board votes on the GLEPs.  Bad Idea.
It stifles things from getting done, and there is no ownership of who is going
to implement the idea.

A new idea proposal should be mailed to a mailinglist (-innovation?) with
details of timeline to completion, impact, and who is doing the implementation.
If it sounds like a good one, then there is a vote and things proceed.  I like
progress.

__Problem: Voting__

Gentoo has over 200 developers.  People are generally against the voting idea,
but I'm not sure why.  I think voting should work like this:  if 30 developers
(or someother specified number) vote yes to an idea then that idea passes.  It
doesn't require everyone to vote, be at home, be on the computer, and not be on
vacation.

The Apache Foundation already has a decent page regarding this:
http://www.apache.org/foundation/voting.html

The Apache Foundation has over 1300 developers; they must be doing something
right.

If someone misses a vote, too bad.  You weren't there and progress has been
made.  I equate this to leaving on vacation from work.  My input is missed
while away, but decisions have been made in my absence.

=-=-=-=-=-

What is interesting is that Source Mage Linux has already voted on a proposal
similar to mine[2].  I truly think that making some changes in the "gentoo way"
would benefit us and make gentoo a truly better distribution.

Ryan
Gentoo Developer

[1] http://article.gmane.org/gmane.linux.gentoo.devel/37599 
[2] http://lists.ibiblio.org/pipermail/sm-discuss/2006-April/014069.html

[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]

Re: [gentoo-dev] Gentoo: State of the Union

[Dan Armak]

[-- Attachment #1: Type: text/plain, Size: 902 bytes --]

On Friday 28 April 2006 23:42, Ryan Phillips wrote:
> svn
>  + Atomic Commits
>  + Merging/tagging/brancing is a simple "copy" operation
>    http://svnbook.red-bean.com/en/1.1/ch04.html
>  + lots of benefits
>    http://svnbook.red-bean.com/nightly/en/svn.intro.features.html
>    there is more I'm sure people can come up with
>  - 2x Drive space

- No changeset/merge tracking

If we have a lot of active branches and a lot of merging between them, 
changeset tracking could be a major plus. I've never used git but I've heard 
that it, and other distributed development-style SCMs, have changeset/merge 
tracking features that are really helpful. Could someone who's used them 
comment on this?

-- 
Dan Armak
Gentoo Linux developer (KDE)
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD  0069 508D 9143 8D5F 8951

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Gentoo: State of the Union

[Fernando J. Pereda]

[-- Attachment #1: Type: text/plain, Size: 1859 bytes --]

On Sat, Apr 29, 2006 at 03:02:46PM +0300, Dan Armak wrote:
> On Friday 28 April 2006 23:42, Ryan Phillips wrote:
> > svn
> >  + Atomic Commits
> >  + Merging/tagging/brancing is a simple "copy" operation
> >    http://svnbook.red-bean.com/en/1.1/ch04.html
> >  + lots of benefits
> >    http://svnbook.red-bean.com/nightly/en/svn.intro.features.html
> >    there is more I'm sure people can come up with
> >  - 2x Drive space
> 
> - No changeset/merge tracking
> 
> If we have a lot of active branches and a lot of merging between them, 
> changeset tracking could be a major plus. I've never used git but I've heard 
> that it, and other distributed development-style SCMs, have changeset/merge 
> tracking features that are really helpful. Could someone who's used them 
> comment on this?

Yes, Git tracks merges unless the merge is trivial (aka fast-forward).

When you merge branch 'a' into branch 'b' and 'b' is a strict subset of
'a' we call that a fast-forward, since there is no merge anywhere:

 o---o---o "b"
          \
           x---x---x "a"

If you merge 'a' into 'b' (git checkout b ; git pull . a) the result is:

 o---o---o---x---x---x "a" = "b"

However in the following case, a proper merge is required:

 o---o---o---#---# "b"
          \
           x---x---x "a"

'b' is not a strict subset of 'a' because the commits marked with # do
not exist in 'a', so the thing ends up being:

 o---o---o---#---#---@ "b"
          \         /
           x---x---x "a"

The commit marked with @ is a special comit called a 'merge'.

I hope that clarifies the merge tracking part.

However I don't know what do you mean with 'changeset tracking'.

Cheers,
Ferdy

-- 
Fernando J. Pereda Garcimartín
Gentoo Developer (Alpha,net-mail,mutt,git)
20BB BDC3 761A 4781 E6ED  ED0B 0A48 5B0C 60BD 28D4

[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] Gentoo: State of the Union

[Dan Armak]

[-- Attachment #1: Type: text/plain, Size: 897 bytes --]

On Saturday 29 April 2006 15:21, Fernando J. Pereda wrote:
> The commit marked with @ is a special comit called a 'merge'.
> I hope that clarifies the merge tracking part.
You just described what merging is. Svn can do that too with svn merge. But, 
if I merge changesets from branch A to B selectively, skipping some along the 
way, can I later ask git to:

- list the changesets remaining in A that I haven't merged to B yet? 
- list the branches, from a given list, which have/haven't merged a given 
changeset?

Those are things svn can't do.

>
> However I don't know what do you mean with 'changeset tracking'.

I didn't mean that 'changeset tracking' is different from 'merge tracking'.

-- 
Dan Armak
Gentoo Linux developer (KDE)
Public GPG key: http://dev.gentoo.org/~danarmak/danarmak-gpg-public.key
Fingerprint: DD70 DBF9 E3D4 6CB9 2FDD  0069 508D 9143 8D5F 8951

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] Re: Gentoo: State of the Union

[R Hill]

Dan Armak wrote:
> On Saturday 29 April 2006 15:21, Fernando J. Pereda wrote:
>> The commit marked with @ is a special comit called a 'merge'.
>> I hope that clarifies the merge tracking part.
> You just described what merging is. Svn can do that too with svn merge. But, 
> if I merge changesets from branch A to B selectively, skipping some along the 
> way, can I later ask git to:
> 
> - list the changesets remaining in A that I haven't merged to B yet? 
> - list the branches, from a given list, which have/haven't merged a given 
> changeset?
> 
> Those are things svn can't do.
> 
>> However I don't know what do you mean with 'changeset tracking'.
> 
> I didn't mean that 'changeset tracking' is different from 'merge tracking'.

Subversion dev Daniel Berlin was recently hired by Google, and according to his 
blog[i] one of his first duties is to implement merge tracking in svn.

--de.


[i] http://www.dberlin.org/blog/

-- 
gentoo-dev@gentoo.org mailing list