[gentoo-dev] Developer branches on proj/gentoo

[gentoo-dev] Developer branches on proj/gentoo

[Patrice Clement]

Hi there

According to https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
"there may be developer-specific, task-specific, project-specific branches
etc". As far as I understand, it means I can go and create my own branch on the
main repository and push it and it gets spread all over the place. Is that
correct?

Could someone explain to me the rationale behind this decision?

Truth to be told, I kinda dislike the fact any developer can do this. 

proj/gentoo should be kept for "serious business" i.e. commits that affects the
tree. On the long run, if everybody goes down that road, we will see flourish
numerous branches (who said unmaintained?), all stalled at a different state of
the main repo, and it will only generate a lot of noise and confusion for
nothing. Further, since we've moved over to git, the main tree now gets
replicated to github and we all have github accounts here, don't we? Which makes
the whole forking and submitting PRs a cinch.

If a developer wants to tinker with the tree, he can fork it to its github
devspace, fiddle around, and later on send us a PR back with his changes to
merge.

Cheers,
Patrice

Re: [gentoo-dev] Developer branches on proj/gentoo

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 903 bytes --]

Dnia 2015-08-11, o godz. 15:52:16
Patrice Clement <monsieurp@gentoo.org> napisał(a):

> Hi there
> 
> According to https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
> "there may be developer-specific, task-specific, project-specific branches
> etc". As far as I understand, it means I can go and create my own branch on the
> main repository and push it and it gets spread all over the place. Is that
> correct?
> 
> Could someone explain to me the rationale behind this decision?
> 
> Truth to be told, I kinda dislike the fact any developer can do this. 

As long as it's used with caution, I don't see a problem. Of course it
would be bad if everyone pushed branches for any minor change. However,
if there is a long-term work going on a branch, I don't see a problem
with keeping it public.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] Developer branches on proj/gentoo

[Tobias Klausmann]

Hi! 

On Tue, 11 Aug 2015, Michał Górny wrote:
> Dnia 2015-08-11, o godz. 15:52:16
> Patrice Clement <monsieurp@gentoo.org> napisał(a):
> 
> > According to https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
> > "there may be developer-specific, task-specific, project-specific branches
> > etc". As far as I understand, it means I can go and create my own branch on the
> > main repository and push it and it gets spread all over the place. Is that
> > correct?
> > 
> > Could someone explain to me the rationale behind this decision?
> > 
> > Truth to be told, I kinda dislike the fact any developer can do this. 
> 
> As long as it's used with caution, I don't see a problem. Of course it
> would be bad if everyone pushed branches for any minor change. However,
> if there is a long-term work going on a branch, I don't see a problem
> with keeping it public.

I agree with monsierp here, person-level stuff should not be in
the official repo.

Also note that "not in the main repo" does not mean "non-public".
People (and projects) can always publish their version of the
tree somewhere else.

I personally even dislike the project branches, but those I am
more willing to accept.

Regards,
Tobias

PS: Call me a pessimist, but every time I see "if it's used with
caution", I think: yeah, but it won't.

-- 
Sent from aboard the Culture ship
	Fine Till You Came Along

Re: [gentoo-dev] Developer branches on proj/gentoo

[Alexis Ballier]

On Tue, 11 Aug 2015 16:01:05 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> Dnia 2015-08-11, o godz. 15:52:16
> Patrice Clement <monsieurp@gentoo.org> napisał(a):
> 
> > Hi there
> > 
> > According to
> > https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
> > "there may be developer-specific, task-specific, project-specific
> > branches etc". As far as I understand, it means I can go and create
> > my own branch on the main repository and push it and it gets spread
> > all over the place. Is that correct?
> > 
> > Could someone explain to me the rationale behind this decision?
> > 
> > Truth to be told, I kinda dislike the fact any developer can do
> > this. 
> 
> As long as it's used with caution, I don't see a problem.

Then we should define 'caution' I think :)

> Of course it
> would be bad if everyone pushed branches for any minor change.
> However, if there is a long-term work going on a branch, I don't see
> a problem with keeping it public.

Most, if not all, projects I've seen use forks for this. This doesn't
prevent being public but gives a clear definition of what 'caution' is.
Branches are usually reserved for releases maintainance.

Alexis.

Re: [gentoo-dev] Developer branches on proj/gentoo

[hasufell]

On 08/11/2015 04:10 PM, Alexis Ballier wrote:
> On Tue, 11 Aug 2015 16:01:05 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> 
>> Dnia 2015-08-11, o godz. 15:52:16
>> Patrice Clement <monsieurp@gentoo.org> napisał(a):
>>
>>> Hi there
>>>
>>> According to
>>> https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
>>> "there may be developer-specific, task-specific, project-specific
>>> branches etc". As far as I understand, it means I can go and create
>>> my own branch on the main repository and push it and it gets spread
>>> all over the place. Is that correct?
>>>
>>> Could someone explain to me the rationale behind this decision?
>>>
>>> Truth to be told, I kinda dislike the fact any developer can do
>>> this. 
>>
>> As long as it's used with caution, I don't see a problem.
> 
> Then we should define 'caution' I think :)
> 
>> Of course it
>> would be bad if everyone pushed branches for any minor change.
>> However, if there is a long-term work going on a branch, I don't see
>> a problem with keeping it public.
> 
> Most, if not all, projects I've seen use forks for this.

Blender does not, afaik. And I've seen a lot of projects doing that. The
difference between e.g. "myremote/featurebranch" and
"upstream/featurebranch" is just that someone has looked over
"upstream/featurebranch" and that it requires pull requests to get stuff
in (so the developer work happens in their developer forks, but the
results still get into the same upstream branch).

That would, for example, make sense for libressl. Since we basically
just overwrite a lot of ebuilds to be able to test them with libressl
patches. That is currently done with an overlay which always opens up
the problem that we lack behind the tree and undesired openssl ebuilds
leak in for the user, because of tree-overlay desync.

Re: [gentoo-dev] Developer branches on proj/gentoo

[Anthony G. Basile]

On 8/11/15 10:19 AM, hasufell wrote:
> On 08/11/2015 04:10 PM, Alexis Ballier wrote:
>> On Tue, 11 Aug 2015 16:01:05 +0200
>> Michał Górny <mgorny@gentoo.org> wrote:
>>
>>> Dnia 2015-08-11, o godz. 15:52:16
>>> Patrice Clement <monsieurp@gentoo.org> napisał(a):
>>>
>>>> Hi there
>>>>
>>>> According to
>>>> https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
>>>> "there may be developer-specific, task-specific, project-specific
>>>> branches etc". As far as I understand, it means I can go and create
>>>> my own branch on the main repository and push it and it gets spread
>>>> all over the place. Is that correct?
>>>>
>>>> Could someone explain to me the rationale behind this decision?
>>>>
>>>> Truth to be told, I kinda dislike the fact any developer can do
>>>> this.
>>> As long as it's used with caution, I don't see a problem.
>> Then we should define 'caution' I think :)
>>
>>
I would not say "caution" so much as good judgment.  The first example 
that came to mind was working with the profiles which crosses many 
directories and files.  In the past when I did restructuring to the 
hardened profiles, I tested by using a branch of the hardened-dev 
overlay.  It was annoying and I would do a bind mount over 
/usr/portage/profiles and had to rebase manually.  A test branch of the 
the main tree which could get rebased and eventually merged back would 
make the workflow so much better.  Another example was when we 
revitalized the selinux policies.  There were hundreds of commits to be 
done.  A branch here that got merged back would be ideal.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] Developer branches on proj/gentoo

[Rich Freeman]

On Tue, Aug 11, 2015 at 10:26 AM, Anthony G. Basile <blueness@gentoo.org> wrote:
> I would not say "caution" so much as good judgment.  The first example that
> came to mind was working with the profiles which crosses many directories
> and files.  In the past when I did restructuring to the hardened profiles, I
> tested by using a branch of the hardened-dev overlay.  It was annoying and I
> would do a bind mount over /usr/portage/profiles and had to rebase manually.
> A test branch of the the main tree which could get rebased and eventually
> merged back would make the workflow so much better.  Another example was
> when we revitalized the selinux policies.  There were hundreds of commits to
> be done.  A branch here that got merged back would be ideal.
>

Agree.  You could still do this with an outside repository that
everybody adds a remote to (a git repository can have many remotes).
You can merge/rebase from a branch outside of gentoo to one inside of
gentoo.

I think the preference should be that users doing their own work
should try to keep the interim stuff out of the main gentoo
repository.  On the other hand, when collaborating teams should be
welcome to use the gentoo repository or their own overlay as makes
sense, with the preference moving more to gentoo as the number of
impacted devs/testers/etc gets bigger.  It will always be a judgment
call.

In the end there isn't that big a difference in git between "git
checkout origin/proj/kde5" and "git checkout kde5overlay/master."
That is the beauty of git.

-- 
Rich

Re: [gentoo-dev] Developer branches on proj/gentoo

[Alexis Ballier]

On Tue, 11 Aug 2015 10:26:46 -0400
"Anthony G. Basile" <blueness@gentoo.org> wrote:

> On 8/11/15 10:19 AM, hasufell wrote:
> > On 08/11/2015 04:10 PM, Alexis Ballier wrote:
> >> On Tue, 11 Aug 2015 16:01:05 +0200
> >> Michał Górny <mgorny@gentoo.org> wrote:
> >>
> >>> Dnia 2015-08-11, o godz. 15:52:16
> >>> Patrice Clement <monsieurp@gentoo.org> napisał(a):
> >>>
> >>>> Hi there
> >>>>
> >>>> According to
> >>>> https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
> >>>> "there may be developer-specific, task-specific, project-specific
> >>>> branches etc". As far as I understand, it means I can go and
> >>>> create my own branch on the main repository and push it and it
> >>>> gets spread all over the place. Is that correct?
> >>>>
> >>>> Could someone explain to me the rationale behind this decision?
> >>>>
> >>>> Truth to be told, I kinda dislike the fact any developer can do
> >>>> this.
> >>> As long as it's used with caution, I don't see a problem.
> >> Then we should define 'caution' I think :)
> >>
> >>
> I would not say "caution" so much as good judgment.  The first
> example that came to mind was working with the profiles which crosses
> many directories and files.  In the past when I did restructuring to
> the hardened profiles, I tested by using a branch of the hardened-dev 
> overlay.  It was annoying and I would do a bind mount over 
> /usr/portage/profiles and had to rebase manually.  A test branch of
> the the main tree which could get rebased and eventually merged back
> would make the workflow so much better.  Another example was when we 
> revitalized the selinux policies.  There were hundreds of commits to
> be done.  A branch here that got merged back would be ideal.


you probably did this before it happened, but a solution in the last
months could have been to fork gentoo-portage-rsync-mirror, merge it
back (or better: rebase onto it) from time to time, and do a squashed PR
that you can merge with mgorny's scripts.

Re: [gentoo-dev] Developer branches on proj/gentoo

[Alexis Ballier]

On Tue, 11 Aug 2015 16:19:12 +0200
hasufell <hasufell@gentoo.org> wrote:

> On 08/11/2015 04:10 PM, Alexis Ballier wrote:
> > On Tue, 11 Aug 2015 16:01:05 +0200
> > Michał Górny <mgorny@gentoo.org> wrote:
> > 
> >> Dnia 2015-08-11, o godz. 15:52:16
> >> Patrice Clement <monsieurp@gentoo.org> napisał(a):
> >>
> >>> Hi there
> >>>
> >>> According to
> >>> https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
> >>> "there may be developer-specific, task-specific, project-specific
> >>> branches etc". As far as I understand, it means I can go and
> >>> create my own branch on the main repository and push it and it
> >>> gets spread all over the place. Is that correct?
> >>>
> >>> Could someone explain to me the rationale behind this decision?
> >>>
> >>> Truth to be told, I kinda dislike the fact any developer can do
> >>> this. 
> >>
> >> As long as it's used with caution, I don't see a problem.
> > 
> > Then we should define 'caution' I think :)
> > 
> >> Of course it
> >> would be bad if everyone pushed branches for any minor change.
> >> However, if there is a long-term work going on a branch, I don't
> >> see a problem with keeping it public.
> > 
> > Most, if not all, projects I've seen use forks for this.
> 
> Blender does not, afaik. And I've seen a lot of projects doing that.
> The difference between e.g. "myremote/featurebranch" and
> "upstream/featurebranch" is just that someone has looked over
> "upstream/featurebranch" and that it requires pull requests to get
> stuff in (so the developer work happens in their developer forks, but
> the results still get into the same upstream branch).

You can merge remote tracking branches just the same you merge
'upstream' branches. You can even rebase them which tends to give a
better history but is harder if you forbid non fast-forward pushes
to gentoo.git. Pull requests are only useful for pre-commit reviews.

> That would, for example, make sense for libressl. Since we basically
> just overwrite a lot of ebuilds to be able to test them with libressl
> patches. That is currently done with an overlay which always opens up
> the problem that we lack behind the tree and undesired openssl ebuilds
> leak in for the user, because of tree-overlay desync.

Branch or remote, this doesn't change anything since no commit to
master will automagically update your branch. The only thing you're
achieving is a fixed gentoo-x86 tree snapshot + an overlay in the same
repo, which you could already do anyway.

Re: [gentoo-dev] Developer branches on proj/gentoo

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 11/08/15 10:01 AM, Michał Górny wrote:
> Dnia 2015-08-11, o godz. 15:52:16 Patrice Clement
> <monsieurp@gentoo.org> napisał(a):
> 
>> Hi there
>> 
>> According to
>> https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
>>
>> 
"there may be developer-specific, task-specific, project-specific branch
es
>> etc". As far as I understand, it means I can go and create my own
>> branch on the main repository and push it and it gets spread all
>> over the place. Is that correct?
>> 
>> Could someone explain to me the rationale behind this decision?
>> 
>> Truth to be told, I kinda dislike the fact any developer can do
>> this.
> 
> As long as it's used with caution, I don't see a problem. Of course
> it would be bad if everyone pushed branches for any minor change.
> However, if there is a long-term work going on a branch, I don't
> see a problem with keeping it public.
> 

Examples in particular I can think of for something like this being
useful would be for, say, major EAPI-bump-related feature
implementations (ie, EAPI5 and slot-operators/subslots), or major
across-tree impementation changes like what we saw with the
multilib-eclass porting.

These are large projects touching most if not all ebuilds, that a
great many developers would or should be involved in.  Something like
this could be done in a separately hosted repo instead of in the main
gentoo repo, but then all developers would need to subscribe to this
other repo, while having it in a branch in the main one i think would
make it easier for everyone to get involved once they're ready, and
would still allow the changes to stay out of the master branch until
the project is ready to launch.






-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlXKEK8ACgkQAJxUfCtlWe35RwEAi2VkkJkCWXCh6tJhEKDbfmzY
fP3rh20RURm84+8K2ysA/2u3dcTukXlGcLHW2xRSR/bjx5be1X+IL8A48bsqgujr
=uppX
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Developer branches on proj/gentoo

[Alexis Ballier]

On Tue, 11 Aug 2015 11:11:43 -0400
Ian Stakenvicius <axs@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> On 11/08/15 10:01 AM, Michał Górny wrote:
> > Dnia 2015-08-11, o godz. 15:52:16 Patrice Clement
> > <monsieurp@gentoo.org> napisał(a):
> > 
> >> Hi there
> >> 
> >> According to
> >> https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
> >>
> >> 
> "there may be developer-specific, task-specific, project-specific
> branch es
> >> etc". As far as I understand, it means I can go and create my own
> >> branch on the main repository and push it and it gets spread all
> >> over the place. Is that correct?
> >> 
> >> Could someone explain to me the rationale behind this decision?
> >> 
> >> Truth to be told, I kinda dislike the fact any developer can do
> >> this.
> > 
> > As long as it's used with caution, I don't see a problem. Of course
> > it would be bad if everyone pushed branches for any minor change.
> > However, if there is a long-term work going on a branch, I don't
> > see a problem with keeping it public.
> > 
> 
> Examples in particular I can think of for something like this being
> useful would be for, say, major EAPI-bump-related feature
> implementations (ie, EAPI5 and slot-operators/subslots), or major
> across-tree impementation changes like what we saw with the
> multilib-eclass porting.
> 
> These are large projects touching most if not all ebuilds, that a
> great many developers would or should be involved in.  Something like
> this could be done in a separately hosted repo instead of in the main
> gentoo repo, but then all developers would need to subscribe to this
> other repo, while having it in a branch in the main one i think would
> make it easier for everyone to get involved once they're ready, and
> would still allow the changes to stay out of the master branch until
> the project is ready to launch.


For me, this is actually a reason to prohibit it :)

EAPI bumps should be done package by package, not at a major scale:
otherwise, let's just scrap EAPI entirely and update the API as we see
fit with p.masked package managers :)

multilib eclasses conversions should also be done one by one to be done
properly (otherwise using multilib-portage is probably a better idea)
and each conversion touches one package (two if you count emul-*).

Big changes that that go in feature branches and are merged in one pass
are, from my experience, way too much prone to errors. Did anyone ever
try to review a merge commit?

Re: [gentoo-dev] Developer branches on proj/gentoo

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 11/08/15 11:21 AM, Alexis Ballier wrote:
> On Tue, 11 Aug 2015 11:11:43 -0400 Ian Stakenvicius
> <axs@gentoo.org> wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>> 
>> On 11/08/15 10:01 AM, Michał Górny wrote:
>>> Dnia 2015-08-11, o godz. 15:52:16 Patrice Clement 
>>> <monsieurp@gentoo.org> napisał(a):
>>> 
>>>> Hi there
>>>> 
>>>> According to 
>>>> https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
>>>>
>>>>
>>
>>>> 
"there may be developer-specific, task-specific, project-specific
>> branch es
>>>> etc". As far as I understand, it means I can go and create
>>>> my own branch on the main repository and push it and it
>>>> gets spread all over the place. Is that correct?
>>>> 
>>>> Could someone explain to me the rationale behind this
>>>> decision?
>>>> 
>>>> Truth to be told, I kinda dislike the fact any developer
>>>> can do this.
>>> 
>>> As long as it's used with caution, I don't see a problem. Of
>>> course it would be bad if everyone pushed branches for any
>>> minor change. However, if there is a long-term work going on
>>> a branch, I don't see a problem with keeping it public.
>>> 
>> 
>> Examples in particular I can think of for something like this
>> being useful would be for, say, major EAPI-bump-related
>> feature implementations (ie, EAPI5 and
>> slot-operators/subslots), or major across-tree impementation
>> changes like what we saw with the multilib-eclass porting.
>> 
>> These are large projects touching most if not all ebuilds, that
>> a great many developers would or should be involved in.
>> Something like this could be done in a separately hosted repo
>> instead of in the main gentoo repo, but then all developers
>> would need to subscribe to this other repo, while having it in
>> a branch in the main one i think would make it easier for
>> everyone to get involved once they're ready, and would still
>> allow the changes to stay out of the master branch until the
>> project is ready to launch.
> 
> 
> For me, this is actually a reason to prohibit it :)
> 
> EAPI bumps should be done package by package, not at a major
> scale: otherwise, let's just scrap EAPI entirely and update the
> API as we see fit with p.masked package managers :)

Not EAPI bumps, but implementation of major new features as a result
of the new EAPI.  Most EAPI changes generally are beneficial to
particular ebuilds, but some (such as slot-operators and subslots)
really needed to be implemented across a great many packages (and
eclasses too at times).  We did it with overlays and patches via
b.g.o and slowly things migrated, but I think it would have gone a
lot faster if all developers had quick and easy access to a branch.

> multilib eclasses conversions should also be done one by one to
> be done properly (otherwise using multilib-portage is probably a
> better idea) and each conversion touches one package (two if you
> count emul-*).

But they don't just touch one package -- you pretty much needed to
do a full deptree at a time.  We worked around it with a rather
messy 'delete all files in emul-* that collide with the
multilib-built packages currently available' plus a convoluted set
of ||() deps wrapping each emul and the individual alternative
atoms.  And even then it was still a mess to try and actually use it
on end-user systems due to various conflicts.

> Big changes that that go in feature branches and are merged in
> one pass are, from my experience, way too much prone to errors.
> Did anyone ever try to review a merge commit?

This makes sense, yes; the merge back to the main tree could very
well be more difficult than it's worth, however if the branch is
available to all, then the migration into the main tree could be
done piecemeal in batches too rather than in one huge swash.

The point is, I think it'd be easier and faster to implement these
major treewide projects (and easier to verify too) if the work could
be done in a branch available to all, rather than in what would
effectively be an overlay someplace external.How we manage it
effectively, I can't say one way or the other but likely this is
something that we will need to learn from experience as much as
decree policy for.




-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlXKGfIACgkQAJxUfCtlWe11XgD/SvvIb9pcZ/k2WRH5OsrKG2G4
0uYC0godRRVytY7s78MA/0dMKUqAlVmqF/HntzPJYoLAqQxGCsrNassDB1iLBV6p
=/msL
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Developer branches on proj/gentoo

[Alexis Ballier]

On Tue, 11 Aug 2015 11:51:14 -0400
Ian Stakenvicius <axs@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> On 11/08/15 11:21 AM, Alexis Ballier wrote:
> > On Tue, 11 Aug 2015 11:11:43 -0400 Ian Stakenvicius
> > <axs@gentoo.org> wrote:
> > 
> >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
> >> 
> >> On 11/08/15 10:01 AM, Michał Górny wrote:
> >>> Dnia 2015-08-11, o godz. 15:52:16 Patrice Clement 
> >>> <monsieurp@gentoo.org> napisał(a):
> >>> 
> >>>> Hi there
> >>>> 
> >>>> According to 
> >>>> https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
> >>>>
> >>>>
> >>
> >>>> 
> "there may be developer-specific, task-specific, project-specific
> >> branch es
> >>>> etc". As far as I understand, it means I can go and create
> >>>> my own branch on the main repository and push it and it
> >>>> gets spread all over the place. Is that correct?
> >>>> 
> >>>> Could someone explain to me the rationale behind this
> >>>> decision?
> >>>> 
> >>>> Truth to be told, I kinda dislike the fact any developer
> >>>> can do this.
> >>> 
> >>> As long as it's used with caution, I don't see a problem. Of
> >>> course it would be bad if everyone pushed branches for any
> >>> minor change. However, if there is a long-term work going on
> >>> a branch, I don't see a problem with keeping it public.
> >>> 
> >> 
> >> Examples in particular I can think of for something like this
> >> being useful would be for, say, major EAPI-bump-related
> >> feature implementations (ie, EAPI5 and
> >> slot-operators/subslots), or major across-tree impementation
> >> changes like what we saw with the multilib-eclass porting.
> >> 
> >> These are large projects touching most if not all ebuilds, that
> >> a great many developers would or should be involved in.
> >> Something like this could be done in a separately hosted repo
> >> instead of in the main gentoo repo, but then all developers
> >> would need to subscribe to this other repo, while having it in
> >> a branch in the main one i think would make it easier for
> >> everyone to get involved once they're ready, and would still
> >> allow the changes to stay out of the master branch until the
> >> project is ready to launch.
> > 
> > 
> > For me, this is actually a reason to prohibit it :)
> > 
> > EAPI bumps should be done package by package, not at a major
> > scale: otherwise, let's just scrap EAPI entirely and update the
> > API as we see fit with p.masked package managers :)
> 
> Not EAPI bumps, but implementation of major new features as a result
> of the new EAPI.  Most EAPI changes generally are beneficial to
> particular ebuilds, but some (such as slot-operators and subslots)
> really needed to be implemented across a great many packages (and
> eclasses too at times).  We did it with overlays and patches via
> b.g.o and slowly things migrated, but I think it would have gone a
> lot faster if all developers had quick and easy access to a branch.


I'm glad this wasn't done that way actually. When subslots appeared,
there was a serious tendency to use them in a fashion that caused
useless rebuilds most of the time. With time, and discussions on some
specific cases, saner practices were adopted.


> > multilib eclasses conversions should also be done one by one to
> > be done properly (otherwise using multilib-portage is probably a
> > better idea) and each conversion touches one package (two if you
> > count emul-*).
> 
> But they don't just touch one package -- you pretty much needed to
> do a full deptree at a time.

Sorry, but no. There was an order in which you could do it, but that's
all.

>  We worked around it with a rather
> messy 'delete all files in emul-* that collide with the
> multilib-built packages currently available' plus a convoluted set
> of ||() deps wrapping each emul and the individual alternative
> atoms.  And even then it was still a mess to try and actually use it
> on end-user systems due to various conflicts.

We split up the huge task into small and manageable ones, yes. I see it
as a clear benefit and I'd even say this is one of the major reasons
multilib-portage didn't make it. This also allowed to iteratively
improve the multilib approach with the experience gained from
converting small parts while getting feedback from those interested.

I don't know what conflicts you're talking about, but I haven't
seen any that was not due to a poor conversion.

> > Big changes that that go in feature branches and are merged in
> > one pass are, from my experience, way too much prone to errors.
> > Did anyone ever try to review a merge commit?
> 
> This makes sense, yes; the merge back to the main tree could very
> well be more difficult than it's worth, however if the branch is
> available to all, then the migration into the main tree could be
> done piecemeal in batches too rather than in one huge swash.
> 
> The point is, I think it'd be easier and faster to implement these
> major treewide projects (and easier to verify too) if the work could
> be done in a branch available to all, rather than in what would
> effectively be an overlay someplace external.

You seem to like pull requests and pre-commit reviews :)
I believe it is optimistic to say that because there is a branch in
gentoo.git then suddenly a lot of people will use this outdated branch
to test the feature.

> How we manage it
> effectively, I can't say one way or the other but likely this is
> something that we will need to learn from experience as much as
> decree policy for.

yes; or discussions :)

Re: [gentoo-dev] Developer branches on proj/gentoo

[hasufell]

On 08/11/2015 05:21 PM, Alexis Ballier wrote:
> 
> Big changes that that go in feature branches and are merged in one pass
> are, from my experience, way too much prone to errors. Did anyone ever
> try to review a merge commit?
> 

You will run repoman (and probably other pkgcore based checks) before
you push that merge. That is for sure.

The only problem that can arise there is that we don't roll our versions
via branches, but via filenames. That means you may merge correctly, but
in master there was already a newer version of app-misc/foo which now
lacks the multilib migration (which isn't a tree breaker, since stuff
still repomanchecks).

We could probably come up with some magic git/bash lines that help with
that. As in: not just detect merge-conflicts, but also "soft conflicts"
in the sense that someone else touched the same ebuild-directory as you
in between.

NixOS for example has (probably not only for that reason) not any
version based filenames, but they roll release-channels via branches.

Re: [gentoo-dev] Developer branches on proj/gentoo

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 11/08/15 12:03 PM, hasufell wrote:
> On 08/11/2015 05:21 PM, Alexis Ballier wrote:
>> 
>> Big changes that that go in feature branches and are merged in
>> one pass are, from my experience, way too much prone to errors.
>> Did anyone ever try to review a merge commit?
>> 
> 
> You will run repoman (and probably other pkgcore based checks)
> before you push that merge. That is for sure.
> 
> The only problem that can arise there is that we don't roll our
> versions via branches, but via filenames. That means you may
> merge correctly, but in master there was already a newer version
> of app-misc/foo which now lacks the multilib migration (which
> isn't a tree breaker, since stuff still repomanchecks).
> 
> We could probably come up with some magic git/bash lines that
> help with that. As in: not just detect merge-conflicts, but also
> "soft conflicts" in the sense that someone else touched the same
> ebuild-directory as you in between.
> 
> NixOS for example has (probably not only for that reason) not
> any version based filenames, but they roll release-channels via
> branches.
> 

That sort of relates to the idea that was brought up last year, if
portage could be made to detect and do VDB-only merges and would
re-emerge ebuilds based on the fact that they were modified rather
than their ${PVR} being incremented, then we could get rid of
revision#'s entirely.  Not a true version removal but it would
reduce the number of distinct files we would be working with in
cases like the above.

But this isn't the place to discuss that tangent I don't think; that
needs a whole new thread and a whole lot of portage development and
possibly a PMS change?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlXKHf0ACgkQAJxUfCtlWe23RQEAuuHF7S5bKHl8ayGYgitGZFuh
ETcKKDxaKw76i2pVDwkA/RLwUKUpbZpId7mvl3j9c4obO9ZAxCaxW25UikU1ZtsV
=YBDy
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Developer branches on proj/gentoo

[Alexis Ballier]

On Tue, 11 Aug 2015 18:03:54 +0200
hasufell <hasufell@gentoo.org> wrote:

> On 08/11/2015 05:21 PM, Alexis Ballier wrote:
> > 
> > Big changes that that go in feature branches and are merged in one
> > pass are, from my experience, way too much prone to errors. Did
> > anyone ever try to review a merge commit?
> > 
> 
> You will run repoman (and probably other pkgcore based checks) before
> you push that merge. That is for sure.


passing repoman, or any static checker, is definitely a very small
subset of what is needed to be good for gentoo-x86

> The only problem that can arise there is that we don't roll our
> versions via branches, but via filenames. That means you may merge
> correctly, but in master there was already a newer version of
> app-misc/foo which now lacks the multilib migration (which isn't a
> tree breaker, since stuff still repomanchecks).
> 
> We could probably come up with some magic git/bash lines that help
> with that. As in: not just detect merge-conflicts, but also "soft
> conflicts" in the sense that someone else touched the same
> ebuild-directory as you in between.
> 
> NixOS for example has (probably not only for that reason) not any
> version based filenames, but they roll release-channels via branches.

After fixing all these sorts of conflicts, I hope you test things
thouroughly, again, after the merge. In the end, it's shorter and
simpler to stop and think at the beginning on how to split your work in
small commits without branching.

Re: [gentoo-dev] Developer branches on proj/gentoo

[hasufell]

On 08/11/2015 03:52 PM, Patrice Clement wrote:
> Hi there
> 
> According to https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
> "there may be developer-specific, task-specific, project-specific branches
> etc". As far as I understand, it means I can go and create my own branch on the
> main repository and push it and it gets spread all over the place. Is that
> correct?
> 
> Could someone explain to me the rationale behind this decision?
> 
> Truth to be told, I kinda dislike the fact any developer can do this. 
> 
> proj/gentoo should be kept for "serious business" i.e. commits that affects the
> tree. On the long run, if everybody goes down that road, we will see flourish
> numerous branches (who said unmaintained?), all stalled at a different state of
> the main repo, and it will only generate a lot of noise and confusion for
> nothing. Further, since we've moved over to git, the main tree now gets
> replicated to github and we all have github accounts here, don't we? Which makes
> the whole forking and submitting PRs a cinch.
> 
> If a developer wants to tinker with the tree, he can fork it to its github
> devspace, fiddle around, and later on send us a PR back with his changes to
> merge.
> 

Branches can still make sense, even if the model is that everyone has
his own fork, see
http://nvie.com/posts/a-successful-git-branching-model/
for an example.

I currently don't see a reason to limit the workflow to one master branch.

It doesn't necessarily generate noise or confusion and there are various
ways to only fetch specific branches if you really need to do so. Git's
main advantage _are_ branches and it has sufficient methods to deal with
a lot of them.

If they cause trouble, we can still prune them and enforce stricter
rules, but since we don't even know how they will be used, this point is
moot yet.

Re: [gentoo-dev] Developer branches on proj/gentoo

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 2521 bytes --]

On Tue, Aug 11, 2015 at 04:12:29PM +0200, hasufell wrote:
> On 08/11/2015 03:52 PM, Patrice Clement wrote:
> > Hi there
> > 
> > According to https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Branching_Model,
> > "there may be developer-specific, task-specific, project-specific branches
> > etc". As far as I understand, it means I can go and create my own branch on the
> > main repository and push it and it gets spread all over the place. Is that
> > correct?
> > 
> > Could someone explain to me the rationale behind this decision?
> > 
> > Truth to be told, I kinda dislike the fact any developer can do this. 
> > 
> > proj/gentoo should be kept for "serious business" i.e. commits that affects the
> > tree. On the long run, if everybody goes down that road, we will see flourish
> > numerous branches (who said unmaintained?), all stalled at a different state of
> > the main repo, and it will only generate a lot of noise and confusion for
> > nothing. Further, since we've moved over to git, the main tree now gets
> > replicated to github and we all have github accounts here, don't we? Which makes
> > the whole forking and submitting PRs a cinch.
> > 
> > If a developer wants to tinker with the tree, he can fork it to its github
> > devspace, fiddle around, and later on send us a PR back with his changes to
> > merge.
> > 
> 
> Branches can still make sense, even if the model is that everyone has
> his own fork, see
> http://nvie.com/posts/a-successful-git-branching-model/
> for an example.
> 
> I currently don't see a reason to limit the workflow to one master branch.
> 
> It doesn't necessarily generate noise or confusion and there are various
> ways to only fetch specific branches if you really need to do so. Git's
> main advantage _are_ branches and it has sufficient methods to deal with
> a lot of them.
> 
> If they cause trouble, we can still prune them and enforce stricter
> rules, but since we don't even know how they will be used, this point is
> moot yet.
 
 I'm with mgorny and hasufell on this; I'm not worried about regulating
 branches that much.

Also, since we have our own tree on g.g.o, the tree on github is a
mirror, so we should treat it as such, e.g. it could go down at any
point, and if it does, we can keep working based on our official tree.

There's even a way in git itself to do something like a github pull
request (see the git request-pull command), so we don't need to rely on
github for that.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]