[gentoo-dev] On banning merge commits

[gentoo-dev] On banning merge commits

[Patrice Clement]

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

Hi gents

After yet another discussion about git in the #gentoo-dev channel tonight, the
topic of merge commits came up for the umpteenth time.

We all seem to agree merge commits are really bad design, add clutter to the
log graph after a while and should be banned altogether from reaching the
central repository.

As of now, no policy is in place yet to keep developers from pushing merge
commits.

What is the correct course of action? I would very much like it to be worded in
a document (GLEP and/or Wiki page) so that confusion is avoided and we all are
on the same page on this topic.

Cheers,

-- 
Patrice Clement
Gentoo Linux developer
http://www.gentoo.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] On banning merge commits

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 835 bytes --]

On Sun, 8 May 2016 01:52:22 +0200
Patrice Clement <monsieurp@gentoo.org> wrote:

> Hi gents
> 
> After yet another discussion about git in the #gentoo-dev channel tonight, the
> topic of merge commits came up for the umpteenth time.
> 
> We all seem to agree merge commits are really bad design, add clutter to the
> log graph after a while and should be banned altogether from reaching the
> central repository.
> 
> As of now, no policy is in place yet to keep developers from pushing merge
> commits.
> 
> What is the correct course of action? I would very much like it to be worded in
> a document (GLEP and/or Wiki page) so that confusion is avoided and we all are
> on the same page on this topic.

You start by accepting my retirement.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] On banning merge commits

[cbergstrom]

Don't be crazy - I know many developer groups which dislike merge commits. That nonlinear work flow is just a mess long term.

  Original Message  
From: Michał Górny
Sent: Sunday, May 8, 2016 13:09
To: Patrice Clement
Reply To: gentoo-dev@lists.gentoo.org
Cc: gentoo-dev@lists.gentoo.org
Subject: Re: [gentoo-dev] On banning merge commits

On Sun, 8 May 2016 01:52:22 +0200
Patrice Clement <monsieurp@gentoo.org> wrote:

> Hi gents
> 
> After yet another discussion about git in the #gentoo-dev channel tonight, the
> topic of merge commits came up for the umpteenth time.
> 
> We all seem to agree merge commits are really bad design, add clutter to the
> log graph after a while and should be banned altogether from reaching the
> central repository.
> 
> As of now, no policy is in place yet to keep developers from pushing merge
> commits.
> 
> What is the correct course of action? I would very much like it to be worded in
> a document (GLEP and/or Wiki page) so that confusion is avoided and we all are
> on the same page on this topic.

You start by accepting my retirement.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

Re: [gentoo-dev] On banning merge commits

[Greg KH]

On Sun, May 08, 2016 at 01:44:43PM +0800, cbergstrom@pathscale.com wrote:
> Don't be crazy - I know many developer groups which dislike merge
> commits. That nonlinear work flow is just a mess long term.

Really?  What "mess" does it cause?

Are things harder to bisect?  Harder to determine what came before what?
Harder to do future development?  Harder because it is unfamiliar
compared to the cvs workflow?

Or just "messier" when you look at the graph of the tree?

What is the _real_ reason that you don't like merges?

thanks,

greg k-h

[gentoo-dev] Re: On banning merge commits

[Duncan]

cbergstrom posted on Sun, 08 May 2016 13:44:43 +0800 as excerpted:

> Don't be crazy - I know many developer groups which dislike merge
> commits. That nonlinear work flow is just a mess long term.

Said by someone who apparently can't figure out reasonable quote then 
reply-in-context, or even appropriate quote nesting, either. <shrug>

>   Original Message
> From: Michał Górny Sent: Sunday, May 8, 2016 13:09 To: Patrice Clement
> Reply To: gentoo-dev@lists.gentoo.org Cc: gentoo-dev@lists.gentoo.org
> Subject: Re: [gentoo-dev] On banning merge commits
> 
> On Sun, 8 May 2016 01:52:22 +0200 Patrice Clement <monsieurp@gentoo.org>
> wrote:
> 
>> Hi gents
>> 
>> After yet another discussion about git in the #gentoo-dev channel
>> tonight, the topic of merge commits came up for the umpteenth time.
>> 
>> We all seem to agree merge commits are really bad design, add clutter
>> to the log graph after a while and should be banned altogether from
>> reaching the central repository.
>> 
>> As of now, no policy is in place yet to keep developers from pushing
>> merge commits.
>> 
>> What is the correct course of action? I would very much like it to be
>> worded in a document (GLEP and/or Wiki page) so that confusion is
>> avoided and we all are on the same page on this topic.
> 
> You start by accepting my retirement.

Agreed with mgorny here.

In fact, I'd suggest banning *non-*merge push-commits.  It's forcing the 
rebases that is really bad design, removing useful information from the 
log graph, etc.

The reason gentoo's git logs are such a mess now is that they're mostly 
flat.  Initial commits are normally one atomic issue per initial commit, 
good, but then instead of naturally grouping them by subject with a well 
commented merge commit, as is done for example with the mainline kernel 
tree, they're unnaturally forced into a flat list by rebases instead of 
the more natural merges, horribly *bad*!

Of course that loses all the rich information that the merges carry, 
including, when merges are done right, grouping individual atomic commits 
by general task/project, and appropriate merge comments outlining what 
was actually merged.

With the kernel I can git log --graph ORIG_HEAD.. and by default search 
only for merges into the mainline/lefthand master branch.  That gives me 
a very good overview of what subsystems were merged, and for the ones 
that interest me, I can read the merge commit comment and have a good 
idea what was merged from that subsystem.  For the ones that are more 
interesting, I can drill down into the merge and read individual commit 
comments.  If even that isn't enough, I can easily git show <commit> 
using the commit ID from the log, and get the full diff.

Try doing that with gentoo.  It doesn't work because everything's flat; 
there's too few merge commits.  For instance, I've no interested in arm, 
and with the kernel git log, I can skip the majority of arm-affecting-
only commits by just skipping the merge commits as soon as I see it's 
from the arm tree, while with the gentoo git log, all those arm 
stabilizations appear as hundreds of individual flat commits on the main 
branch, instead of a small handful of merge-commits, with merge-commit 
comments such as "arm stabilizations for app-foo/bar and deps".

So on gentoo, what I end up doing instead is seeing what packages want to 
update, and only searching the git log for the interesting ones.  Works 
well enough for that, but I've effectively no overview of what's going on 
in general as I do for the kernel tree, as I miss all the interesting 
commits that didn't happen to show up as package update candidates spit 
out by portage, even in areas I'd otherwise track rather heavily, like 
kde, where I track the kde overlay git log, but miss the gentoo repo side 
due to the problems created by lack of appropriate merge commits as 
explained above.


Or to put it a different way, if we're not going to use git's rich 
distributed branch development and tracking, forcing everything to single 
chain on the main tree, why did we bother switching to git in the first 
place?  That was available on cvs, or if we wanted more features, 
subversion, etc.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] On banning merge commits

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1539 bytes --]

Hi all,

On Sun, 8 May 2016 01:52:22 +0200 Patrice Clement wrote:
> Hi gents
> 
> After yet another discussion about git in the #gentoo-dev channel tonight, the
> topic of merge commits came up for the umpteenth time.
> 
> We all seem to agree merge commits are really bad design, add clutter to the
> log graph after a while and should be banned altogether from reaching the
> central repository.

No, not all of us.
 
> As of now, no policy is in place yet to keep developers from pushing merge
> commits.
> 
> What is the correct course of action? I would very much like it to be worded in
> a document (GLEP and/or Wiki page) so that confusion is avoided and we all are
> on the same page on this topic.

Sometimes merge commits are desired. But they should be used
wisely. Of course if used randomly and without thought they may
become disaster.

I see nothing wrong if developer makes series of large non-trivial
changes in a separate branch and then merges this branch with
current head after good testing. Another case is when some user
submits a long branch of patches and developer merges it from
external repository (with proper testing of course).

If one have problems reading logs with merge commits, use gitk
or similar tools. I had the same hate relationship towards
non-linear workflow after switching from cvs and svn to git, but
with time comes the understanding that git workflow is the right
way to go, it just takes time to get accustomed to it.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Re: On banning merge commits

[Kent Fredric]

On 8 May 2016 at 20:58, Duncan <1i5t5.duncan@cox.net> wrote:
> Or to put it a different way, if we're not going to use git's rich
> distributed branch development and tracking, forcing everything to single
> chain on the main tree, why did we bother switching to git in the first
> place?  That was available on cvs, or if we wanted more features,
> subversion, etc.


I think the annoyance is more having two histories, where on one side,
you've got the high-traffic
gentoo work flow happening, and then  you have a merge commit ....

And that merge commit may have only a single commit on it, and its
parent is god-knows how many days old.

So the "graph" looks *massive* when it is really only a single commit
and its merge commit.

I think the most productive thing here is not to ban "merge commits"
as such, but ban merge commits where the "merge base" ( that is, the
common ancestor of the left and right parents of the merge commit )
leaves a significant number of commits on the "left" side of the
equation.

Personally, I could live with "0 commits on left", because that would
give a bunching effect.

The "Real History" would still be linear if you followed only the
right parents, but you'd get a simplified view with grouping if you
followed only the left parents.

However, a limit of say, 10 commits on left, I could also live with.

The essential idea being to minimise the amount of congnitive effort a
human has when trying to explore the history and understand what
"actually happened" from a master perspective.

"Long histories that go for days only to merge one commit" tend to
harm this, and I think that's the essential irritation.

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] On banning merge commits

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 1495 bytes --]

On 05/08/2016 01:21 AM, Greg KH wrote:
> On Sun, May 08, 2016 at 01:44:43PM +0800, cbergstrom@pathscale.com wrote:
>> Don't be crazy - I know many developer groups which dislike merge
>> commits. That nonlinear work flow is just a mess long term.
> 
> Really?  What "mess" does it cause?
> 
> Are things harder to bisect?  Harder to determine what came before what?
> Harder to do future development?  Harder because it is unfamiliar
> compared to the cvs workflow?
> 
> Or just "messier" when you look at the graph of the tree?
> 
> What is the _real_ reason that you don't like merges?
> 
> thanks,
> 
> greg k-h
> 

I don't have a strong -- or even clear -- opinion on the matter, but I
could imagine it being a bother to see a bunch of "merge commit" commit
messages in `git log` and not really have much to go on as far as "who
submitted this, who approved it, what does it fix, etc". As far as I
know, there's only the committer information and any GPG-signing that
may have accompanied it, as we do in Gentoo. If I have any details
wrong, please clarify.

I've heard about a way to "redo" history in a git repository as well,
especially before pushing. Could that be a way to mitigate merge
commits, so they are more informative and conform to our commit message
convention?

Sincerely,

a neutral party
-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] On banning merge commits

[Amadeusz Żołnowski]

[-- Attachment #1: Type: text/plain, Size: 216 bytes --]

I am working at the moment on debundling ejabberd. It will come with ~30
packages and I will do "git merge --no-ff ejabberd-debundled" because it
will actually look less messy.

Thanks,
-- Amadeusz Żołnowski

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 950 bytes --]

[gentoo-dev] Re: On banning merge commits

[Duncan]

Kent Fredric posted on Sun, 08 May 2016 21:25:38 +1200 as excerpted:

> On 8 May 2016 at 20:58, Duncan <1i5t5.duncan@cox.net> wrote:
>> Or to put it a different way, if we're not going to use git's rich
>> distributed branch development and tracking, forcing everything to
>> single chain on the main tree, why did we bother switching to git in
>> the first place?  That was available on cvs, or if we wanted more
>> features, subversion, etc.
> 
> I think the annoyance is more having two histories, where on one side,
> you've got the high-traffic gentoo work flow happening, and then  you
> have a merge commit ....
> 
> And that merge commit may have only a single commit on it, and its
> parent is god-knows how many days old.
> 
> So the "graph" looks *massive* when it is really only a single commit
> and its merge commit.
> 
> I think the most productive thing here is not to ban "merge commits" as
> such, but ban merge commits where the "merge base" ( that is, the common
> ancestor of the left and right parents of the merge commit ) leaves a
> significant number of commits on the "left" side of the equation. [...]

> "Long histories that go for days only to merge one commit" tend to harm
> this, and I think that's the essential irritation.

OK, that I can agree with. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] On banning merge commits

[Dirkjan Ochtman]

On Sun, May 8, 2016 at 7:09 AM, Michał Górny <mgorny@gentoo.org> wrote:
>> What is the correct course of action? I would very much like it to be worded in
>> a document (GLEP and/or Wiki page) so that confusion is avoided and we all are
>> on the same page on this topic.
>
> You start by accepting my retirement.

Hey Michał,

Patrice is just opening up a discussion on a mailing list. Clearly
there are opposing viewpoints on whether we should use merge commits
and/or how often. Making vague threats without any rationale/reasoning
about why you would dislike any change in this direction is completely
unproductive for the Gentoo community. I would be very interested in
what your proposed way forward is, and why.

Cheers,

Dirkjan

Re: [gentoo-dev] Re: On banning merge commits

[Dirkjan Ochtman]

On Sun, May 8, 2016 at 11:25 AM, Kent Fredric <kentfredric@gmail.com> wrote:
> The essential idea being to minimise the amount of congnitive effort a
> human has when trying to explore the history and understand what
> "actually happened" from a master perspective.
>
> "Long histories that go for days only to merge one commit" tend to
> harm this, and I think that's the essential irritation.

This makes sense to me.

Personally, I think rebases are easy in the vast majority of the cases
(specificially for our gentoo tree, where actual conflicts are often
unlikely). However, for long-running branches (either in time, which I
think doesn't happen that often, or in work), making an explicit merge
could still make sense.

(I guess this is slightly different than somehow limiting the length
of the left-sized twig.)

Cheers,

Dirkjan

Re: [gentoo-dev] On banning merge commits

[Andreas K. Hüttel]

Am Sonntag, 8. Mai 2016, 07:09:31 schrieb Michał Górny:
> > What is the correct course of action? I would very much like it to be
> > worded in a document (GLEP and/or Wiki page) so that confusion is
> > avoided and we all are on the same page on this topic.
> 
> You start by accepting my retirement.

I think you should take a vacation for a while... Preferably somewhere 
tropical, with no internet access and lots of beach... 

-- 
Andreas K. Hüttel
Gentoo Linux developer (council, perl, libreoffice)
dilfridge@gentoo.org
http://www.akhuettel.de/

Re: [gentoo-dev] On banning merge commits

[Andreas K. Hüttel]

Am Sonntag, 8. Mai 2016, 01:52:22 schrieb Patrice Clement:
>
> What is the correct course of action? I would very much like it to be
> worded in a document (GLEP and/or Wiki page) so that confusion is avoided
> and we all are on the same page on this topic.
> 

OK here's my 2ct: 

* I'm not opposed to merge commits in principle, and see a few cases where 
they have advantages.

* Git has the provisions for nonlinear history, and just not liking spaghetti 
is no sufficient reason to castrate the entire version control system.

* However... as the past months have shown, when using merges it is much 
easier to accidentally mess up the entire tree than using rebases alone.

* So, in an ideal world we would use merges wisely and sparingly.

* In the real world, we risk less and also lose less if we ban and technically 
prevent them.

* The only alternative would be to come up with criteria for merges and 
actually enforce them (meaning, if you mess up the tree more than twice you 
lose your push access. Hello QA.).

-- 
Andreas K. Hüttel
Gentoo Linux developer (council, perl, libreoffice)
dilfridge@gentoo.org
http://www.akhuettel.de/

Re: [gentoo-dev] On banning merge commits

[M. J. Everitt]

[-- Attachment #1: Type: text/plain, Size: 498 bytes --]

On 08/05/16 12:13, Andreas K. Hüttel wrote:
> Am Sonntag, 8. Mai 2016, 07:09:31 schrieb Michał Górny:
>>> What is the correct course of action? I would very much like it to be
>>> worded in a document (GLEP and/or Wiki page) so that confusion is
>>> avoided and we all are on the same page on this topic.
>> You start by accepting my retirement.
> I think you should take a vacation for a while... Preferably somewhere 
> tropical, with no internet access and lots of beach... 
>
+5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: [gentoo-dev] On banning merge commits

[Rich Freeman]

On Sun, May 8, 2016 at 7:25 AM, Andreas K. Hüttel <dilfridge@gentoo.org> wrote:
>
> * However... as the past months have shown, when using merges it is much
> easier to accidentally mess up the entire tree than using rebases alone.
>

How does a merge make it any easier/harder to mess up the entire tree?
 I can see how they can make the history easier/harder to read but in
the end I believe the content of the tree itself ends up being
whatever was in the index when you made the commit.

>
> * The only alternative would be to come up with criteria for merges and
> actually enforce them (meaning, if you mess up the tree more than twice you
> lose your push access. Hello QA.).
>

Here is the other only alternative: use rebase commits when they're
most appropriate, and use merge commits when they're most appropriate,
and publish easy-to-understand guidelines for when each is the case.
I don't really see a need for hard rules unless we think devs can
actually comply with them.  Do we really want somebody to lose their
commit access because they pushed a string of rebased commits that
might have been more appropriate as a single merge commit?

Both types of commits have their place.  I think that bans are better
used for bad attitude than for mistakes.  I don't think you need a
100% rigid rule to enforce a ban either - this isn't a court of law
(and I think many of the failings of courts of law result from the
rigid application of rules, but that is a different matter).

-- 
Rich

Re: [gentoo-dev] On banning merge commits

[Michał Górny]

Dnia 8 maja 2016 12:30:15 CEST, Dirkjan Ochtman <djc@gentoo.org> napisał(a):
>On Sun, May 8, 2016 at 7:09 AM, Michał Górny <mgorny@gentoo.org> wrote:
>>> What is the correct course of action? I would very much like it to
>be worded in
>>> a document (GLEP and/or Wiki page) so that confusion is avoided and
>we all are
>>> on the same page on this topic.
>>
>> You start by accepting my retirement.
>
>Hey Michał,
>
>Patrice is just opening up a discussion on a mailing list. Clearly
>there are opposing viewpoints on whether we should use merge commits
>and/or how often. Making vague threats without any rationale/reasoning
>about why you would dislike any change in this direction is completely
>unproductive for the Gentoo community. I would be very interested in
>what your proposed way forward is, and why.

No, he didn't. He stated an imaginary fact ('we all seem to agree...'), and asked how to *enforce* that formally. That's not how you request differing opinions.

It's rather the common Gentoo reboot in which for Nth time you propose the same thing, pretend that all previous opposition didn't happen and hope people will be tired enough to let you have what you want.

My response is also a common Gentoo response to bad ideas, and it shouldn't come as a surprise to anyone.

-- 
Best regards,
Michał Górny (by phone)

Re: [gentoo-dev] On banning merge commits

[Kent Fredric]

On 8 May 2016 at 23:57, Rich Freeman <rich0@gentoo.org> wrote:
> How does a merge make it any easier/harder to mess up the entire tree?
>  I can see how they can make the history easier/harder to read but in
> the end I believe the content of the tree itself ends up being
> whatever was in the index when you made the commit.
>
>>
>> * The only alternative would be to come up with criteria for merges and
>> actually enforce them (meaning, if you mess up the tree more than twice you
>> lose your push access. Hello QA.).
>>
>
> Here is the other only alternative: use rebase commits when they're
> most appropriate, and use merge commits when they're most appropriate,
> and publish easy-to-understand guidelines for when each is the case.
> I don't really see a need for hard rules unless we think devs can
> actually comply with them.  Do we really want somebody to lose their
> commit access because they pushed a string of rebased commits that
> might have been more appropriate as a single merge commit?
>
> Both types of commits have their place.  I think that bans are better
> used for bad attitude than for mistakes.  I don't think you need a
> 100% rigid rule to enforce a ban either - this isn't a court of law
> (and I think many of the failings of courts of law result from the
> rigid application of rules, but that is a different matter).

I'd probably say any case where you have to resolve a conflict in the
merge commit is a clear indicator that the right branch needed
rebasing prior to merging, either way.

mostly, because it means some change on the right side becomes
transiently dependent on a change on the left side, and its obscured
by the resolution mid-merge.

( And such conflicts tend not to be even obvious that they've existed
when traversing a history with merge commits, because "merge diffs"
get suppressed by default ).

And its usually not clear what the "Solution" is when you're simply
comparing 2 heads.

With a rebase, you know exactly what the expected change is at the
commit that makes the change, and the rebase lets you see what the
existing state was at the time of applying the change.

Merges without conflicts however are much less contentious.

In short, if you get a merge conflict, then somebody needs to pay more
attention to what happened on one side of the changes, and one side of
that equation should be rebased to avoid that conflict.


-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] On banning merge commits

[Anthony G. Basile]

On 5/8/16 7:25 AM, Andreas K. Hüttel wrote:
> Am Sonntag, 8. Mai 2016, 01:52:22 schrieb Patrice Clement:
>>
>> What is the correct course of action? I would very much like it to be
>> worded in a document (GLEP and/or Wiki page) so that confusion is avoided
>> and we all are on the same page on this topic.
>>
> 
> * However... as the past months have shown, when using merges it is much 
> easier to accidentally mess up the entire tree than using rebases alone.
> 
> * So, in an ideal world we would use merges wisely and sparingly.

Correct.  I don't support outright banning merge commits, but they
should be reviewed carefully, like we do with other big commits to the
tree.  So maybe proceed as follows:

1. announce to gentoo-dev@ the intention to start a branch intending to
merge

2. hack hack hack

3. test the merge for any conflicts etc,

4. announce to the list a date/time to merge

5. if okay, ermge

> 
> * In the real world, we risk less and also lose less if we ban and technically 
> prevent them.
> 
> * The only alternative would be to come up with criteria for merges and 
> actually enforce them (meaning, if you mess up the tree more than twice you 
> lose your push access. Hello QA.).
> 


-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] On banning merge commits

[Kent Fredric]

On 9 May 2016 at 00:09, Anthony G. Basile <blueness@gentoo.org> wrote:
> 1. announce to gentoo-dev@ the intention to start a branch intending to
> merge
>
> 2. hack hack hack
>
> 3. test the merge for any conflicts etc,
>
> 4. announce to the list a date/time to merge
>
> 5. if okay, ermge


I think that's a bit excessive myself. I dislike merges, but I don't
hate them *quite* that much to justify such formality.

Because IMO, its not about forbidding merges, its about making sure
the use of merges is appropriate and effective.

For instance, stabilizing ~100 ebuilds in response to a single stable
request, it would make logical sense to group all those stabilizations
so that they appear on the left side as a single atomic change, while
still existing as a series on the right side of the branch for
historical accuracy
and for other practical reasons ( like simplified commit reverts )

Pretty much *anything* that does "bulk changes for single purpose" is
a good candidate for a merge.

For instance, there's ~100 commits sitting around somewhere in a
repository for introducing Perl 5.24 to the tree, which requires a
commit for each and every entry in virtual/perl-*

It would make much sense for this series to be visible on Master as a
"add Perl 5.24 to tree" commit, because all the changes are inherently
interdependent,
and it would make little sense to rewind to a specific point within
that series and use it as a portage tree.

But that's not significant enough to warrant a lot of formal fluffing around.

It for sure would be best if that 100 commit range was rebased before
merging, but it should still be a non-fast-forward merge just to keep
the history logical.



-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] On banning merge commits

[Dirkjan Ochtman]

On Sun, May 8, 2016 at 2:00 PM, Michał Górny <mgorny@gentoo.org> wrote:
> No, he didn't. He stated an imaginary fact ('we all seem to agree...'), and asked how to *enforce* that formally. That's not how you request differing opinions.

He used "seem to" to state that it was his perspective, and asked
"What is the correct course of action?", which to me does not at all
read as only being about formal enforcement.

> It's rather the common Gentoo reboot in which for Nth time you propose the same thing, pretend that all previous opposition didn't happen and hope people will be tired enough to let you have what you want.

Apparently the previous times it was discussed there was no
satisfactory resolution, or the topic (or understanding thereof) has
advanced since then. Perhaps we should GLEPs more to document such
discussions, so that we don't have to repeat them all the time.

> My response is also a common Gentoo response to bad ideas, and it shouldn't come as a surprise to anyone.

Well, even if it's common, I think it's much worse than Patrice's
initial email. If you think something is a bad idea, lay out why you
think so. If you don't want to repeat yourself, write it up in an
email list or blog post that you can link to after the fact. Your
response of posing a threat of quitting is not productive at all, and
seems kind of childish to me.

Cheers,

Dirkjan

Re: [gentoo-dev] On banning merge commits

[Rich Freeman]

On Sun, May 8, 2016 at 8:18 AM, Kent Fredric <kentfredric@gmail.com> wrote:
> On 9 May 2016 at 00:09, Anthony G. Basile <blueness@gentoo.org> wrote:
>> 1. announce to gentoo-dev@ the intention to start a branch intending to
>> merge
>>
>> 2. hack hack hack
>>
>> 3. test the merge for any conflicts etc,
>>
>> 4. announce to the list a date/time to merge
>>
>> 5. if okay, ermge
>
> It would make much sense for this series to be visible on Master as a
> "add Perl 5.24 to tree" commit, because all the changes are inherently
> interdependent,
> and it would make little sense to rewind to a specific point within
> that series and use it as a portage tree.
>
> But that's not significant enough to warrant a lot of formal fluffing around.
>
> It for sure would be best if that 100 commit range was rebased before
> merging, but it should still be a non-fast-forward merge just to keep
> the history logical.
>

++

merges shouldn't just be used for random pull-requests.  However, when
you're touching multiple packages/etc they should be considered.  They
should also be considered if for some reason you had a bazillion
commits to a single package that for some reason shouldn't be rebased.
I imagine that they'll be a small portion of commits as a result.
However, for the situations where they're appropriate they make a lot
of sense.

This was some of Duncan's point, but I will comment that we'll never
have as clean a history as the kernel simply because we don't have a
release-based workflow with the work cascading up various trees.  The
kernel is almost an ideal case for a merge-based workflow.  I imagine
that half of Gentoo's commit volume is random revbumps and keyword
changes and that is just going to be noise no matter what.  If we were
release-based we could do that stuff in its own branch and merge it
all in at once, but that just isn't us.

--
Rich

Re: [gentoo-dev] On banning merge commits

[Anthony G. Basile]

On 5/8/16 8:34 AM, Rich Freeman wrote:
> On Sun, May 8, 2016 at 8:18 AM, Kent Fredric <kentfredric@gmail.com> wrote:
>> On 9 May 2016 at 00:09, Anthony G. Basile <blueness@gentoo.org> wrote:
>>> 1. announce to gentoo-dev@ the intention to start a branch intending to
>>> merge
>>>
>>> 2. hack hack hack
>>>
>>> 3. test the merge for any conflicts etc,
>>>
>>> 4. announce to the list a date/time to merge
>>>
>>> 5. if okay, ermge
>>
>> It would make much sense for this series to be visible on Master as a
>> "add Perl 5.24 to tree" commit, because all the changes are inherently
>> interdependent,
>> and it would make little sense to rewind to a specific point within
>> that series and use it as a portage tree.
>>
>> But that's not significant enough to warrant a lot of formal fluffing around.
>>
>> It for sure would be best if that 100 commit range was rebased before
>> merging, but it should still be a non-fast-forward merge just to keep
>> the history logical.
>>
> 
> ++
> 
> merges shouldn't just be used for random pull-requests.  However, when
> you're touching multiple packages/etc they should be considered.  

i was actually thinking of sec-policy where i remember writing a script
back in the CVS days that did some 200 commits, one after another.  i
was migrating some work for Swift from an overlay to the main tree.


They
> should also be considered if for some reason you had a bazillion
> commits to a single package that for some reason shouldn't be rebased.

a bazillion commits to a category that almost no one touches and except
one person or team, like sec-policy or dev-perl etc.

so again, i'm against an outright banning of merge commits, but we need
to restrict them to reasonable cases, "reasonable" being judged by the
community.

> I imagine that they'll be a small portion of commits as a result.
> However, for the situations where they're appropriate they make a lot
> of sense.
> 
> This was some of Duncan's point, but I will comment that we'll never
> have as clean a history as the kernel simply because we don't have a
> release-based workflow with the work cascading up various trees.  The
> kernel is almost an ideal case for a merge-based workflow.  I imagine
> that half of Gentoo's commit volume is random revbumps and keyword
> changes and that is just going to be noise no matter what.  If we were
> release-based we could do that stuff in its own branch and merge it
> all in at once, but that just isn't us.
> 
> --
> Rich
> 


-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] On banning merge commits

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1044 bytes --]

On Sun, 08 May 2016 11:06:09 +0100
Amadeusz Żołnowski <aidecoe@gentoo.org> wrote:

> I am working at the moment on debundling ejabberd. It will come with
> ~30 packages and I will do "git merge --no-ff ejabberd-debundled"
> because it will actually look less messy.
> 
> Thanks,
> -- Amadeusz Żołnowski


Yes, this is exactly the type of merge commits that should be allowed.

Not the one or two user PR commits that might be based on a weeks old
tree, that a developer merges without rebasing.  It is these merge
commits which have caused all the grief we've experienced over merge
commits.

Merge commits should be used wisely and for larger branch merges like
the kde, gnome team's development overlay merges to the main tree or
similar larger one off project's like the one above.  It is these
larger commit branches that are much more difficult to "git pull
--rebase && git push --signed" successfully without some other pushes in
between causing a rejected non-fast forward push.
-- 
Brian Dolbec <dolsen>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-dev] On banning merge commits

[Jeroen Roovers]

On Sun, 8 May 2016 05:53:42 -0700
Brian Dolbec <dolsen@gentoo.org> wrote:

> It is these
> larger commit branches that are much more difficult to "git pull
> --rebase && git push --signed" successfully without some other pushes
> in between causing a rejected non-fast forward push.

You mean doing

until <git push>; do <git pull>; done

is not already as optimal as you could get?


     jer

Re: [gentoo-dev] On banning merge commits

[Alexis Ballier]

On Sun, 8 May 2016 01:52:22 +0200
Patrice Clement <monsieurp@gentoo.org> wrote:

> After yet another discussion about git in the #gentoo-dev channel
> tonight, the topic of merge commits came up for the umpteenth time.
> 
> We all seem to agree merge commits are really bad design, add clutter
> to the log graph after a while and should be banned altogether from
> reaching the central repository.

They don't clutter the log graph at all, if you admit it is a graph and
not a chain :)

 
> As of now, no policy is in place yet to keep developers from pushing
> merge commits.


I was under the impression that merging is needed in order to preserve
commit signatures when e.g. merging someone else's work.

Re: [gentoo-dev] On banning merge commits

[Kent Fredric]

On 9 May 2016 at 05:03, Alexis Ballier <aballier@gentoo.org> wrote:
> I was under the impression that merging is needed in order to preserve
> commit signatures when e.g. merging someone else's work.


Correct, but if the person applying the commits to tree is in fact
reviewing them as they go, then the fact they re-sign it with their
own signature
( and changing the commits "Committed by" in the process ) pretty much
means the chain of custody is preserved.

That is, the fact the original signature is lost is immaterial,
because we only need it as a signature that /somebody/ actually is
responsible for the commit, and the person performing the rebase takes
the essential responsibility in the process.

The original author metadata is however, not lost in this process,
only commit metadata changes. ( And the signature is commit metadata,
not author metadata )

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

[gentoo-dev] Re: On banning merge commits

[Duncan]

Rich Freeman posted on Sun, 08 May 2016 07:57:17 -0400 as excerpted:

> I think that bans are better used for bad attitude than for mistakes.

[Stepping back from the immediate discussion at hand...]

The above is wisdom, arguably, quotable sig-level wisdom!

Certainly wisdom enough to be worth emphasizing by calling it out in a 
quote as I'm doing here. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: On banning merge commits

[Duncan]

Rich Freeman posted on Sun, 08 May 2016 08:34:37 -0400 as excerpted:

> merges shouldn't just be used for random pull-requests.  However, when
> you're touching multiple packages/etc they should be considered.  They
> should also be considered if for some reason you had a bazillion commits
> to a single package that for some reason shouldn't be rebased.
> I imagine that they'll be a small portion of commits as a result.
> However, for the situations where they're appropriate they make a lot of
> sense.
> 
> This was some of Duncan's point, but I will comment that we'll never
> have as clean a history as the kernel simply because we don't have a
> release-based workflow with the work cascading up various trees.  The
> kernel is almost an ideal case for a merge-based workflow.  I imagine
> that half of Gentoo's commit volume is random revbumps and keyword
> changes and that is just going to be noise no matter what.  If we were
> release-based we could do that stuff in its own branch and merge it all
> in at once, but that just isn't us.

Recognized and agree.  Thanks.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] On banning merge commits

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 1621 bytes --]

On 05/08/2016 05:53 AM, Brian Dolbec wrote:
> On Sun, 08 May 2016 11:06:09 +0100
> Amadeusz Żołnowski <aidecoe@gentoo.org> wrote:
> 
>> I am working at the moment on debundling ejabberd. It will come with
>> ~30 packages and I will do "git merge --no-ff ejabberd-debundled"
>> because it will actually look less messy.
>>
>> Thanks,
>> -- Amadeusz Żołnowski
> 
> 
> Yes, this is exactly the type of merge commits that should be allowed.
> 
> Not the one or two user PR commits that might be based on a weeks old
> tree, that a developer merges without rebasing.  It is these merge
> commits which have caused all the grief we've experienced over merge
> commits.
> 
> Merge commits should be used wisely and for larger branch merges like
> the kde, gnome team's development overlay merges to the main tree or
> similar larger one off project's like the one above.  It is these
> larger commit branches that are much more difficult to "git pull
> --rebase && git push --signed" successfully without some other pushes in
> between causing a rejected non-fast forward push.
> 
I think you touched on the key phrase we should be taking from the
conversation: 'merges without rebasing'. Devs are encouraged -- if not
required -- to push commits after rebasing, to ensure we're pushing to
the latest HEAD and not something that's stale. If we hold merges to the
same standard, would the problems that some have with merge commits
disappear?

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] On banning merge commits

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 1069 bytes --]

On 05/08/2016 07:07 PM, Kent Fredric wrote:
> On 9 May 2016 at 05:03, Alexis Ballier <aballier@gentoo.org> wrote:
>> I was under the impression that merging is needed in order to preserve
>> commit signatures when e.g. merging someone else's work.
> 
> 
> Correct, but if the person applying the commits to tree is in fact
> reviewing them as they go, then the fact they re-sign it with their
> own signature
> ( and changing the commits "Committed by" in the process ) pretty much
> means the chain of custody is preserved.

And it is a requirement in particular in the case where the author is
not a gentoo dev as the certificate used for the signature otherwise
isn't recognized. The committing developer will need to have a local
framework in place for certificate validation to ensure that the author
is authentic, after that the committing author is responsible for all
behavior of the commit.

-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-dev] On banning merge commits

[Rich Freeman]

On Mon, May 9, 2016 at 7:27 AM, Kristian Fiskerstrand <k_f@gentoo.org> wrote:
> On 05/08/2016 07:07 PM, Kent Fredric wrote:
>> On 9 May 2016 at 05:03, Alexis Ballier <aballier@gentoo.org> wrote:
>>> I was under the impression that merging is needed in order to preserve
>>> commit signatures when e.g. merging someone else's work.
>>
>>
>> Correct, but if the person applying the commits to tree is in fact
>> reviewing them as they go, then the fact they re-sign it with their
>> own signature
>> ( and changing the commits "Committed by" in the process ) pretty much
>> means the chain of custody is preserved.
>
> And it is a requirement in particular in the case where the author is
> not a gentoo dev as the certificate used for the signature otherwise
> isn't recognized. The committing developer will need to have a local
> framework in place for certificate validation to ensure that the author
> is authentic, after that the committing author is responsible for all
> behavior of the commit.
>

Keep in mind that you can have both.  You can both preserve the
original commit with its signature, and introduce it as a merge commit
with a Gentoo signature.

I'm not saying we necessarily should do this, but certainly git makes
this possible, and it is a potential benefit of merge commits.

However, in this case it would not be possible to rebase the original
commit, which introduces some of the uncleanliness of non-rebased
merge commits.  In general I'm a fan of rebasing merge commits.

-- 
Rich

Re: [gentoo-dev] On banning merge commits

[Kent Fredric]

On 10 May 2016 at 00:23, Rich Freeman <rich0@gentoo.org> wrote:
> which introduces some of the uncleanliness of non-rebased
> merge commits.  In general I'm a fan of rebasing merge commits.


Non-rebased merge commits are worst when the merge involves  a
collision resolution.

While you can in theory rebase merge commits with rebase --preserve,
my experience has shown me that its very difficult to get right, and
the presence of merge collisions in the "preserved" rebase risks
getting the conflict resolution lost mid-rebase, which is not entirely
helpful.

If there was something we could feasibly put hard software policy
constraints against without any subjective "but but but" cases, it
would be these cases of "merge included conflicts".

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] On banning merge commits

[Rich Freeman]

On Mon, May 9, 2016 at 8:36 AM, Kent Fredric <kentfredric@gmail.com> wrote:
>
> While you can in theory rebase merge commits with rebase --preserve,
> my experience has shown me that its very difficult to get right, and
> the presence of merge collisions in the "preserved" rebase risks
> getting the conflict resolution lost mid-rebase, which is not entirely
> helpful.
>
> If there was something we could feasibly put hard software policy
> constraints against without any subjective "but but but" cases, it
> would be these cases of "merge included conflicts".
>

Is there any way to tell git to rebase everything but the conflicts
themselves, and then resolve those in the merge commit?  That might be
the cleanest solution as it filters out all the noise, and then has a
merge commit which clearly shows how the conflict was resolved.

Now, for non-trivial conflicts I think it is better to rebase or merge
into the branch, test, and then do a non-conflicting merge back into
the main tree.  How else will you know that you resolved the conflict
correctly?  By non-trivial I mean stuff other than keywords and
descriptions and such.

-- 
Rich

Re: [gentoo-dev] On banning merge commits

[Alexis Ballier]

On Mon, 9 May 2016 05:07:45 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> On 9 May 2016 at 05:03, Alexis Ballier <aballier@gentoo.org> wrote:
> > I was under the impression that merging is needed in order to
> > preserve commit signatures when e.g. merging someone else's work.  
> 
> 
> Correct, but if the person applying the commits to tree is in fact
> reviewing them as they go, then the fact they re-sign it with their
> own signature
> ( and changing the commits "Committed by" in the process ) pretty much
> means the chain of custody is preserved.


yeah, i think we have the same chain of custody with ssh push auth +
safe servers + ssl pull, we don't need signing for this.

> That is, the fact the original signature is lost is immaterial,
> because we only need it as a signature that /somebody/ actually is
> responsible for the commit, and the person performing the rebase takes
> the essential responsibility in the process.


well, then I can commit crap with --author mrp@gentoo.org and claim he
made me rebase it :)
I understand gpg signing of commits as a way to guarantee author is
correctly set and claims the commit.


Alexis.

Re: [gentoo-dev] On banning merge commits

[Kent Fredric]

On 11 May 2016 at 00:04, Alexis Ballier <aballier@gentoo.org> wrote:
> well, then I can commit crap with --author mrp@gentoo.org and claim he
> made me rebase it :)


Well, if you're going down that line ...

You don't rebase it, you just merge it, than then mrp claims obama
forced his hand to write the commit at gunpoint and sign it, and
that's why he is both --author and --committer

That's obviously silly talk :D

You put your name on it with your GPG key, then the responsibility
beyond that point is a social one, not a technical one.

The person who signed via GPG still holds the "Technical responsibility" :)

>I  understand gpg signing of commits as a way to guarantee author is
> correctly set and claims the commit.

No. GPG commit signing only guarantees "committer". That's why git
rebase re-writes committer as well as re-signing it.

The committer metadata itself is no real guarantee either, because you
can twiddle COMMIT env vars and change that on a whim, so I could
forge a commit authored by mrp and committed by aballier ... and
unless you checked the GPG sig, you'd never know that I made it.

But by design, the signature only indicates who the person was who
*committed* a commit, it can never indicate the true author.

For instance, a commit *could* in theory be authored by somebody who
has no access to a computer, and I could copy-paste that data and
upload it.

The true author would never be known /unless/ I forged author data,
but I sure was the person who committed it.

And "Commit responsibility" is what we're trying to regulate here.
"Author metadata" is just for attribution/credits sake, and a *weak*
responsibility.


-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] On banning merge commits

[Alexis Ballier]

On Wed, 11 May 2016 02:18:03 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> On 11 May 2016 at 00:04, Alexis Ballier <aballier@gentoo.org> wrote:
> > well, then I can commit crap with --author mrp@gentoo.org and claim
> > he made me rebase it :)  
> 
> 
> Well, if you're going down that line ...
> 
> You don't rebase it, you just merge it, than then mrp claims obama
> forced his hand to write the commit at gunpoint and sign it, and
> that's why he is both --author and --committer
> 
> That's obviously silly talk :D


yes, and it was meant to be :)


my point was more that if we want signed commits, then better have
author sign it, and thus use merge

if we just want committer signature, then I don't really see the
point in signing: it isnt more secure than pecker access + ldap
password, and infra could simply sign public git trees with a unique
gentoo key

[...]

Re: [gentoo-dev] On banning merge commits

[Kent Fredric]

On 11 May 2016 at 22:21, Alexis Ballier <aballier@gentoo.org> wrote:
>
> yes, and it was meant to be :)
>
>
> my point was more that if we want signed commits, then better have
> author sign it, and thus use merge

Eh, I see it more a "signed commits don't really add any value to this
discussion".

Both signing on master with rebase, and signing on a side branch by
the author and signed at the merge point "work", they both do what
they meant to achieve:

That essentially, there is always going to be a gentoo gatekeeper with
a *known* signature signing a critical part of the chain, ensuring
referential integrity of
the chain of custody.

That, imo, results in us discarding this argument, and folding back to
the "what are the benefits of rebasing/merging on their own merits
again?" , of which, there are many, even in the absence of signing.

> if we just want committer signature, then I don't really see the
> point in signing: it isnt more secure than pecker access + ldap
> password, and infra could simply sign public git trees with a unique
> gentoo key

There's an added security measure that exists /outside/ the gentoo
source control.

It means that you can still ascertain some degree of "authority"
without having to rely on gentoo's centralised control system.

For instance, that Perl branch in the overlay that was pending, you
can still validate who created it, and how, and who last touched each
and every commit,
despite the fact its not yet anywhere near gentoo infrastructure.

And being able to *independently* verify an arbitrary git commit was
created by the person who claims to have created it, is a good thing.

And that *independent* data can be used as a selection criteria by
maintainers pulling commits into gentoo.

But that said, none of this really reflects on wether we should use
merges or rebases.

Those situations are primarily driven by the nature of the work
itself, not our meta-level security concerns.



-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] On banning merge commits

[Rich Freeman]

On Wed, May 11, 2016 at 10:34 AM, Kent Fredric <kentfredric@gmail.com> wrote:
>
> There's an added security measure that exists /outside/ the gentoo
> source control.
>

It also fails differently.

If I find out that somebody compromised ssh in some way, doubt is cast
on any commit during the period in which the ssh server was
vulnerable, and that could go back quite a ways.

If I find out that somebody's gpg key was compromised, then at most
that one developer's commits are tainted.

Obviously an ssh breach could be limited to a single account as well,
but it has a server-wide component for which a parallel in git doesn't
really exist.

In any case, nobody has proposed getting rid of the requirement that a
known key be used to sign all direct commits to the tree.  That direct
commit could have a parent that is not signed with a key known to
Gentoo.

My sense is that most here would agree that most of our routine
commits should just be rebased, but merge commits have a legitimate
place, and it wouldn't hurt to better document what we consider best
practices (which seem to include rebasing merge commits when
practical).  I suspect that improvement is ultimately going to come
down to volunteer interest in creating that documentation, as with
much of the rest of our workflow.

-- 
Rich