[gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1400 bytes --]

Hello, everyone.

I have the pleasure to announce that the official rsync2git mirror is
up and running [1] thanks to Sven Wegener. It is updated from rsync
every 30 minutes, and can be used both to sync your local Gentoo
installs and to submit improvements via pull requests (see README [2]
for some details).

At the same time, I have established the 'Git Mirror' [3] project which
welcomes developers willing to help reviewing the pull requests
and helping those improvements reach package maintainers.

For users, this means that we now have a fairly efficient syncing
method and a pull request-based workflow for submitting fixes.
The auto-synced repository can also make proxy-maint workflow easier.

For developers, this either means:

a. if you want to help us, join the team, watch the pull requests.
CC maintainers when appropriate, review, even work towards merging
the changes with approval of the maintainers,

b. if you want to support git users, just wait till we CC you and then
review, help, merge :),

c. if you don't want to support git users, just ignore the repo. We'll
bother you directly after the changes are reviewed and ready :).

[1]:https://github.com/gentoo/gentoo-portage-rsync-mirror
[2]:https://github.com/gentoo/gentoo-portage-rsync-mirror#README
[3]:https://wiki.gentoo.org/wiki/Project:Git_mirror

-- 
Best regards,
Michał Górny

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[Seemant Kulleen]

[-- Attachment #1: Type: text/plain, Size: 1697 bytes --]

Congratulations!  This is huge!  Michal and Sven and everyone who did this
== awesome ppl

Cheers!
Seemant

seemantk empathic design
http://seemantk.com
On Jan 26, 2015 12:12 PM, "Michał Górny" <mgorny@gentoo.org> wrote:

> Hello, everyone.
>
> I have the pleasure to announce that the official rsync2git mirror is
> up and running [1] thanks to Sven Wegener. It is updated from rsync
> every 30 minutes, and can be used both to sync your local Gentoo
> installs and to submit improvements via pull requests (see README [2]
> for some details).
>
> At the same time, I have established the 'Git Mirror' [3] project which
> welcomes developers willing to help reviewing the pull requests
> and helping those improvements reach package maintainers.
>
> For users, this means that we now have a fairly efficient syncing
> method and a pull request-based workflow for submitting fixes.
> The auto-synced repository can also make proxy-maint workflow easier.
>
> For developers, this either means:
>
> a. if you want to help us, join the team, watch the pull requests.
> CC maintainers when appropriate, review, even work towards merging
> the changes with approval of the maintainers,
>
> b. if you want to support git users, just wait till we CC you and then
> review, help, merge :),
>
> c. if you don't want to support git users, just ignore the repo. We'll
> bother you directly after the changes are reviewed and ready :).
>
> [1]:https://github.com/gentoo/gentoo-portage-rsync-mirror
> [2]:https://github.com/gentoo/gentoo-portage-rsync-mirror#README
> [3]:https://wiki.gentoo.org/wiki/Project:Git_mirror
>
> --
> Best regards,
> Michał Górny
>

[-- Attachment #2: Type: text/html, Size: 2354 bytes --]

Re: [gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[Ben de Groot]

On 27 January 2015 at 04:12, Michał Górny <mgorny@gentoo.org> wrote:
> Hello, everyone.
>
> I have the pleasure to announce that the official rsync2git mirror is
> up and running [1] thanks to Sven Wegener. It is updated from rsync
> every 30 minutes, and can be used both to sync your local Gentoo
> installs and to submit improvements via pull requests (see README [2]
> for some details).
>
> At the same time, I have established the 'Git Mirror' [3] project which
> welcomes developers willing to help reviewing the pull requests
> and helping those improvements reach package maintainers.

Nice work!

I'm looking forward to working with the Git Mirror project.

-- 
Cheers,

Ben | yngwin
Gentoo developer

Re: [gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[Alexey Lapitsky]

Hi,

Looks awesome, great initiative!

I'm wondering if it makes sense to integrate travis-ci to run QA tests
automatically?

> Since the changes will be re-committed in CVS manually, we will end up updating the commit message anyway.

It should be possible to automate this step as well. Is there a reason
why not to do that?

--

Alexey


On Tue, Jan 27, 2015 at 4:19 PM, Ben de Groot <yngwin@gentoo.org> wrote:
> On 27 January 2015 at 04:12, Michał Górny <mgorny@gentoo.org> wrote:
>> Hello, everyone.
>>
>> I have the pleasure to announce that the official rsync2git mirror is
>> up and running [1] thanks to Sven Wegener. It is updated from rsync
>> every 30 minutes, and can be used both to sync your local Gentoo
>> installs and to submit improvements via pull requests (see README [2]
>> for some details).
>>
>> At the same time, I have established the 'Git Mirror' [3] project which
>> welcomes developers willing to help reviewing the pull requests
>> and helping those improvements reach package maintainers.
>
> Nice work!
>
> I'm looking forward to working with the Git Mirror project.
>
> --
> Cheers,
>
> Ben | yngwin
> Gentoo developer
>

Re: [gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[Rich Freeman]

On Wed, Jan 28, 2015 at 10:51 AM, Alexey Lapitsky <lex.public@gmail.com> wrote:
>
>> Since the changes will be re-committed in CVS manually, we will end up updating the commit message anyway.
>
> It should be possible to automate this step as well. Is there a reason
> why not to do that?
>

You're proposing auto-committing from the git mirror back into cvs?
Or just a tool to allow devs to cherry-pick commits from git and stage
them in cvs?  The first seems potentially fraught with peril.

At what point does this basically turn into Gentoo forking itself out
of frustration with the git migration?  :)

-- 
Rich

Re: [gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[Alexey Lapitsky]

Hi Rich,

I'm sorry if I don't understand the cherry-picking workflow very well,
I'm not a gentoo dev myself.
Correct me if I'm wrong, but in my opinion it's very valuable to save
maintainer time and I don't see a lot of value in doing manual
cherry-picking to cvs (given that QA tests pass automatically and
there is no conflicts).

Here's a naive example how it might work:

1. A user creates a PR
2. travis-ci automatically runs QA tests: repoman full on the PR,
reports the result back
3. Github hook is emitted, a "system" does some processing: for
example, create a bug in bugzilla, notify the maintainer, comment on
the PR
4. The relevant maintainer checks the PR, comments: "looks fine, merging"
5. Github hook is emitted to "the system", which checks the PR
comments, does some sanity checks, merges commit back to cvs, rejects
the PR

"The system" might be a simple script which has opt-in membership for
the subsystems / maintainers who wants to use it.

Does it sound sane?


On Wed, Jan 28, 2015 at 4:58 PM, Rich Freeman <rich0@gentoo.org> wrote:
> On Wed, Jan 28, 2015 at 10:51 AM, Alexey Lapitsky <lex.public@gmail.com> wrote:
>>
>>> Since the changes will be re-committed in CVS manually, we will end up updating the commit message anyway.
>>
>> It should be possible to automate this step as well. Is there a reason
>> why not to do that?
>>
>
> You're proposing auto-committing from the git mirror back into cvs?
> Or just a tool to allow devs to cherry-pick commits from git and stage
> them in cvs?  The first seems potentially fraught with peril.
>
> At what point does this basically turn into Gentoo forking itself out
> of frustration with the git migration?  :)
>
> --
> Rich
>

Re: [gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1018 bytes --]

On Wed, 28 Jan 2015 17:22:21 +0100
Alexey Lapitsky <lex.public@gmail.com> wrote:
> Here's a naive example how it might work:
> 
> 1. A user creates a PR
> 2. travis-ci automatically runs QA tests: repoman full on the PR,
> reports the result back
> 3. Github hook is emitted, a "system" does some processing: for
> example, create a bug in bugzilla, notify the maintainer, comment on
> the PR
> 4. The relevant maintainer checks the PR, comments: "looks fine,
> merging" 5. Github hook is emitted to "the system", which checks the
> PR comments, does some sanity checks, merges commit back to cvs,
> rejects the PR
> 
> "The system" might be a simple script which has opt-in membership for
> the subsystems / maintainers who wants to use it.
> 
> Does it sound sane?

Exherbo has been doing something along those lines for ages, and not
just for repoman: there's also full build testing (from a clean system,
to check dependencies are correct). It's not very hard to do.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[Rich Freeman]

On Wed, Jan 28, 2015 at 12:56 PM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Wed, 28 Jan 2015 17:22:21 +0100
> Alexey Lapitsky <lex.public@gmail.com> wrote:
>> Here's a naive example how it might work:
>>
>> 1. A user creates a PR
>> 2. travis-ci automatically runs QA tests: repoman full on the PR,
>> reports the result back
>> 3. Github hook is emitted, a "system" does some processing: for
>> example, create a bug in bugzilla, notify the maintainer, comment on
>> the PR
>> 4. The relevant maintainer checks the PR, comments: "looks fine,
>> merging" 5. Github hook is emitted to "the system", which checks the
>> PR comments, does some sanity checks, merges commit back to cvs,
>> rejects the PR
>>
>> "The system" might be a simple script which has opt-in membership for
>> the subsystems / maintainers who wants to use it.
>>
>> Does it sound sane?
>
> Exherbo has been doing something along those lines for ages, and not
> just for repoman: there's also full build testing (from a clean system,
> to check dependencies are correct). It's not very hard to do.
>

Including having two repositories both being committed to, with a
delayed propagation of commits from each to the other?  Somehow I
doubt that is what you're doing with exherbo.

The CI isn't the part I'm picking on here.  The idea of still working
in cvs while having a separate CI and commit path in git that applies
back to cvs is the part that sounds rather prone to breakage.  That
seems like a massive hack.  The real solution is to get the heck off
of cvs.

-- 
Rich

[gentoo-project] Re: Official rsync->git mirror & 'Git mirror' project announcement

[NP-Hardass]

Alexey Lapitsky <lex.public <at> gmail.com> writes:

> 
> Hi,
> 
> Looks awesome, great initiative!
> 
> I'm wondering if it makes sense to integrate travis-ci to run QA tests
> automatically?
> 
> > Since the changes will be re-committed in CVS manually, we will end up 
updating the commit message anyway.
> 
> It should be possible to automate this step as well. Is there a reason
> why not to do that?
> 
> --
> 
> Alexey

Looks like they opted to roll their own system to use travis with repoman. 
(https://github.com/gentoo/gentoo-portage-rsync-
mirror/blob/master/.travis.yml)
Just a FYI, there is also repoman-travis (https://github.com/mrueg/repoman-
travis) from mrueg, though it doesn't make sense to use repoman-travis in 
this situation, since repoman-travis assumes that you don't have the gentoo 
tree set  up, and imports a copy of it, prior to running repoman.  I use 
repoman-travis in all of my overlays.

Re: [gentoo-project] Re: Official rsync->git mirror & 'Git mirror' project announcement

[Michał Górny]

NP-Hardass <np.hardass@gmail.com> napisał:

>Alexey Lapitsky <lex.public <at> gmail.com> writes:
>
>> 
>> Hi,
>> 
>> Looks awesome, great initiative!
>> 
>> I'm wondering if it makes sense to integrate travis-ci to run QA
>tests
>> automatically?
>> 
>> > Since the changes will be re-committed in CVS manually, we will end
>up 
>updating the commit message anyway.
>> 
>> It should be possible to automate this step as well. Is there a
>reason
>> why not to do that?
>> 
>> --
>> 
>> Alexey
>
>Looks like they opted to roll their own system to use travis with
>repoman. 
>(https://github.com/gentoo/gentoo-portage-rsync-
>mirror/blob/master/.travis.yml)
>Just a FYI, there is also repoman-travis
>(https://github.com/mrueg/repoman-
>travis) from mrueg, though it doesn't make sense to use repoman-travis
>in 
>this situation, since repoman-travis assumes that you don't have the
>gentoo 
>tree set  up, and imports a copy of it, prior to running repoman.  I
>use 
>repoman-travis in all of my overlays.

We know, mrueg helped us a bit. However, we needed a dedicated script because of repository size and independency :).

Right now i'm working on replacing it with pcheck. We still have a few minor bugs to fix before deploying though. Then we should be able to perform the most important checks in less than 1 hour! More like 5 minutes, i suspect.

-- 
Michał Górny

Re: [gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[Markos Chandras]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 01/26/2015 08:12 PM, Michał Górny wrote:
> Hello, everyone.
> 
> I have the pleasure to announce that the official rsync2git mirror
> is up and running [1] thanks to Sven Wegener. It is updated from
> rsync every 30 minutes, and can be used both to sync your local
> Gentoo installs and to submit improvements via pull requests (see
> README [2] for some details).
> 
> At the same time, I have established the 'Git Mirror' [3] project
> which welcomes developers willing to help reviewing the pull
> requests and helping those improvements reach package maintainers.
> 
> For users, this means that we now have a fairly efficient syncing 
> method and a pull request-based workflow for submitting fixes. The
> auto-synced repository can also make proxy-maint workflow easier.
> 
> For developers, this either means:
> 
> a. if you want to help us, join the team, watch the pull requests. 
> CC maintainers when appropriate, review, even work towards merging 
> the changes with approval of the maintainers,
> 
> b. if you want to support git users, just wait till we CC you and
> then review, help, merge :),
> 
> c. if you don't want to support git users, just ignore the repo.
> We'll bother you directly after the changes are reviewed and ready
> :).
> 
> [1]:https://github.com/gentoo/gentoo-portage-rsync-mirror 
> [2]:https://github.com/gentoo/gentoo-portage-rsync-mirror#README 
> [3]:https://wiki.gentoo.org/wiki/Project:Git_mirror
> 
Hi,

First of all let me express my gratitude on what you did! It's a
really nice effort towards the right direction. However, I feel there
is a bit of overlap with the proxy-maintainers[1] repository. On the
readme file you suggest to not commit anything to this git repository
(not even merge pull requests). So what's the purpose of having the
pull requests in the first place? I believe, in order to avoid
confusion, users should use the proxy-maintainers repository for pull
requests for the packages they maintain. Otherwise, just stick to the
bugzilla for ebuilds that have developers as maintainers. I believe we
are running into the risk of having too many places for user
contributions, and that will make maintenance a nightmare because you
will have devs taking care of proxy-maint pull requests, and then
others taking care of the rsync-git mirror pull requests. I believe
you need to rethink the proposed workflow.

[1] https://github.com/gentoo/proxy-maintainers

- -- 
Regards,
Markos Chandras
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQF8BAEBCgBmBQJUyVk1XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGRDlGMzA4MUI2MzBDODQ4RDBGOEYxMjQx
RjEwRUQ0QjgxREVCRjE5AAoJEB8Q7UuB3r8ZHQ4H/1uY5QZ2G0O2B3kXb4iRxIRG
/juGzRhedVHj0Y14T4BBsEVJlZK8um+JBBDNTGRIlRwWmF2FnsCq+lbd3cnxLTUP
MmG9XMxO+a1rRlksfCkWJw4y3+/rvB5Uj3em8AsRMKL2H+p29EIeZDuvXeqfAo+6
BUZ287VYK1TcT0PEfUhdlhhfu/uR3KwQMLDiMKYLHZFxRwqezGCL9HiDxZQH/kUA
bb/lBOCKm3vUofDQic1frcSGys5Dk+iz5wdMUOvPQBFAfsg+kzr5/YJBkXAeOmme
zFW+uecuHBdjDhauq8KqiSx65YPIWT6Uz0wQ6GjwRWQ/Nhr6u3CbeG06zjN1xrY=
=nWJ4
-----END PGP SIGNATURE-----

Re: [gentoo-project] Official rsync->git mirror & 'Git mirror' project announcement

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4006 bytes --]

Dnia 2015-01-28, o godz. 21:48:37
Markos Chandras <hwoarang@gentoo.org> napisał(a):

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> On 01/26/2015 08:12 PM, Michał Górny wrote:
> > Hello, everyone.
> > 
> > I have the pleasure to announce that the official rsync2git mirror
> > is up and running [1] thanks to Sven Wegener. It is updated from
> > rsync every 30 minutes, and can be used both to sync your local
> > Gentoo installs and to submit improvements via pull requests (see
> > README [2] for some details).
> > 
> > At the same time, I have established the 'Git Mirror' [3] project
> > which welcomes developers willing to help reviewing the pull
> > requests and helping those improvements reach package maintainers.
> > 
> > For users, this means that we now have a fairly efficient syncing 
> > method and a pull request-based workflow for submitting fixes. The
> > auto-synced repository can also make proxy-maint workflow easier.
> > 
> > For developers, this either means:
> > 
> > a. if you want to help us, join the team, watch the pull requests. 
> > CC maintainers when appropriate, review, even work towards merging 
> > the changes with approval of the maintainers,
> > 
> > b. if you want to support git users, just wait till we CC you and
> > then review, help, merge :),
> > 
> > c. if you don't want to support git users, just ignore the repo.
> > We'll bother you directly after the changes are reviewed and ready
> > :).
> > 
> > [1]:https://github.com/gentoo/gentoo-portage-rsync-mirror 
> > [2]:https://github.com/gentoo/gentoo-portage-rsync-mirror#README 
> > [3]:https://wiki.gentoo.org/wiki/Project:Git_mirror
> > 
> Hi,
> 
> First of all let me express my gratitude on what you did! It's a
> really nice effort towards the right direction. However, I feel there
> is a bit of overlap with the proxy-maintainers[1] repository. On the
> readme file you suggest to not commit anything to this git repository
> (not even merge pull requests). So what's the purpose of having the
> pull requests in the first place? I believe, in order to avoid
> confusion, users should use the proxy-maintainers repository for pull
> requests for the packages they maintain. Otherwise, just stick to the
> bugzilla for ebuilds that have developers as maintainers. I believe we
> are running into the risk of having too many places for user
> contributions, and that will make maintenance a nightmare because you
> will have devs taking care of proxy-maint pull requests, and then
> others taking care of the rsync-git mirror pull requests. I believe
> you need to rethink the proposed workflow.
> 
> [1] https://github.com/gentoo/proxy-maintainers

I have talked to some of the team members and they expressed interest
in using this repository instead of proxy-maint one. The main advantage
is that the repository is synced to gx86, so users don't have to
copy/sync ebuilds manually to work on them.

The pull request workflow still has advantages, even if you don't use
the github's built-in merge mechanism. Developers can easily pull
the changes to their local repo and test them, and then copy over to
CVS. Plus github provides a simple review workflow with comments
and updates.

As I see it, it's both better for users and developers. More points of
contribution can be a bit confusing but also go towards a more
distributed workflow. If user can help me by contributing to Gentoo, I
don't think we should require him to match our fancy tools but let him
use the tools he finds helpful. And a lot of people find github this
way.

That said, I still think the workflow is easier. Cloning and creating
the pull request is more console-friendly than opening a bug
and attaching few files. Reviews can be more dynamic, and updating
the pull request is much easier than deprecating and attaching new
files. Not to mention downloading and testing those files...

-- 
Best regards,
Michał Górny

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]