* [gentoo-scm] repoman patch for git support (from drobbins) @ 2008-11-07 1:12 Jeremy Olexa 2008-11-07 0:55 ` Robin H. Johnson 0 siblings, 1 reply; 20+ messages in thread From: Jeremy Olexa @ 2008-11-07 1:12 UTC (permalink / raw To: gentoo-scm Hello list, I happened to be in #gentoo-portage and found drobbins hacking on repoman. I asked for permission to post this and he agreed. I am merely the messenger and hosting the patch. I *think* he has sent this to zmedico already. Anyway, hack on.. http://www.funtoo.org/repoman (full code) http://dev.gentoo.org/~darkside/repoman-git-drobbins.patch (against 2.2_rc12 but applies to rc13 as well) -Jeremy ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-07 1:12 [gentoo-scm] repoman patch for git support (from drobbins) Jeremy Olexa @ 2008-11-07 0:55 ` Robin H. Johnson 2008-11-11 10:22 ` Fabian Groffen 0 siblings, 1 reply; 20+ messages in thread From: Robin H. Johnson @ 2008-11-07 0:55 UTC (permalink / raw To: gentoo-scm [-- Attachment #1: Type: text/plain, Size: 971 bytes --] On Thu, Nov 06, 2008 at 07:12:13PM -0600, Jeremy Olexa wrote: > Hello list, > > I happened to be in #gentoo-portage and found drobbins hacking on > repoman. I asked for permission to post this and he agreed. I am > merely the messenger and hosting the patch. I *think* he has sent this > to zmedico already. Anyway, hack on.. > > http://www.funtoo.org/repoman (full code) > http://dev.gentoo.org/~darkside/repoman-git-drobbins.patch (against > 2.2_rc12 but applies to rc13 as well) Bug: @@ -1949,8 +1975,10 @@ .. + elif vcs == "git": + print "(svn -q commit -F commitmessagefile)" Other notes: 'git ls-files' and most of the other git actions will always go to the top of the tree first, and run recursively from there. Some care is needed if your tree is dirty because of this. -- Robin Hugh Johnson Gentoo Linux Developer & Infra Guy E-Mail : robbat2@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 [-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-07 0:55 ` Robin H. Johnson @ 2008-11-11 10:22 ` Fabian Groffen 2008-11-11 9:36 ` Robin H. Johnson 2008-11-12 1:55 ` Zac Medico 0 siblings, 2 replies; 20+ messages in thread From: Fabian Groffen @ 2008-11-11 10:22 UTC (permalink / raw To: gentoo-scm FYI: zmedico just committed a modified version of Daniel's git patch to Portage's trunk. On 06-11-2008 16:55:15 -0800, Robin H. Johnson wrote: > On Thu, Nov 06, 2008 at 07:12:13PM -0600, Jeremy Olexa wrote: > > Hello list, > > > > I happened to be in #gentoo-portage and found drobbins hacking on > > repoman. I asked for permission to post this and he agreed. I am > > merely the messenger and hosting the patch. I *think* he has sent this > > to zmedico already. Anyway, hack on.. > > > > http://www.funtoo.org/repoman (full code) > > http://dev.gentoo.org/~darkside/repoman-git-drobbins.patch (against > > 2.2_rc12 but applies to rc13 as well) > Bug: > @@ -1949,8 +1975,10 @@ > .. > + elif vcs == "git": > + print "(svn -q commit -F commitmessagefile)" > > Other notes: > 'git ls-files' and most of the other git actions will always go to the top of > the tree first, and run recursively from there. Some care is needed if your > tree is dirty because of this. -- Fabian Groffen Gentoo on a different level ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-11 10:22 ` Fabian Groffen @ 2008-11-11 9:36 ` Robin H. Johnson 2008-11-12 1:55 ` Zac Medico 1 sibling, 0 replies; 20+ messages in thread From: Robin H. Johnson @ 2008-11-11 9:36 UTC (permalink / raw To: gentoo-scm [-- Attachment #1: Type: text/plain, Size: 422 bytes --] On Tue, Nov 11, 2008 at 11:22:57AM +0100, Fabian Groffen wrote: > FYI: > zmedico just committed a modified version of Daniel's git patch to > Portage's trunk. Yup, I saw it, and it's got the minor bug I mentioned here fixed. P.S. Please don't top-post. -- Robin Hugh Johnson Gentoo Linux Developer & Infra Guy E-Mail : robbat2@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 [-- Attachment #2: Type: application/pgp-signature, Size: 329 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-11 10:22 ` Fabian Groffen 2008-11-11 9:36 ` Robin H. Johnson @ 2008-11-12 1:55 ` Zac Medico 2008-11-15 16:52 ` Mike Auty 1 sibling, 1 reply; 20+ messages in thread From: Zac Medico @ 2008-11-12 1:55 UTC (permalink / raw To: gentoo-scm -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Fabian Groffen wrote: > FYI: > > zmedico just committed a modified version of Daniel's git patch to > Portage's trunk. It's released in portage-2.2_rc14, so please go ahead and try it if you'd like. - -- Thanks, Zac -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkkaN5QACgkQ/ejvha5XGaN4QQCeJjLXa+7mAbqBb4sh84dUbxwj VIkAoMCAImjsC0YXcW5MKVNPKMkPVs2D =Zfj8 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-12 1:55 ` Zac Medico @ 2008-11-15 16:52 ` Mike Auty 2008-11-15 22:32 ` Zac Medico 2008-11-15 22:44 ` Alec Warner 0 siblings, 2 replies; 20+ messages in thread From: Mike Auty @ 2008-11-15 16:52 UTC (permalink / raw To: gentoo-scm -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Zac Medico wrote: > It's released in portage-2.2_rc14, so please go ahead and try it if > you'd like. So, can we test this on the gentoo-x86.git in overlays.g.o, or are we trying to keep that as a clean conversion? If not, will this work on normal, small git overlays or do we need a duplicate of gentoo-x86.git that we can have access to and try commits out on? Mike 5:) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkke/l0ACgkQu7rWomwgFXpA0wCfd8F1uHSZQjFEAMe3WOaDe9Qq iiwAn3/fi/gnNL2NTqs1gCiGo6y4v7hp =U3gv -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-15 16:52 ` Mike Auty @ 2008-11-15 22:32 ` Zac Medico 2008-11-16 5:04 ` Donnie Berkholz 2008-11-15 22:44 ` Alec Warner 1 sibling, 1 reply; 20+ messages in thread From: Zac Medico @ 2008-11-15 22:32 UTC (permalink / raw To: gentoo-scm -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mike Auty wrote: > So, can we test this on the gentoo-x86.git in overlays.g.o, or are we > trying to keep that as a clean conversion? I'm not sure what the rules are for that. > If not, will this work on normal, small git overlays or do we need a > duplicate of gentoo-x86.git that we can have access to and try commits > out on? It should work fine on a normal git overlay. - -- Thanks, Zac -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkkfTfgACgkQ/ejvha5XGaPjNQCdGYX+A+wPP5OamHKs/VqYDfjX lqgAoMKWm/5f4NJbhso1awBVk78QrviN =FxFg -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-15 22:32 ` Zac Medico @ 2008-11-16 5:04 ` Donnie Berkholz 0 siblings, 0 replies; 20+ messages in thread From: Donnie Berkholz @ 2008-11-16 5:04 UTC (permalink / raw To: Zac Medico; +Cc: gentoo-scm [-- Attachment #1: Type: text/plain, Size: 614 bytes --] On 14:32 Sat 15 Nov , Zac Medico wrote: > Mike Auty wrote: > > So, can we test this on the gentoo-x86.git in overlays.g.o, or are we > > trying to keep that as a clean conversion? > > I'm not sure what the rules are for that. cvs2git doesn't do incremental conversion, so it doesn't really matter what happens with that repository. But you can do just as much with a local clone (or two, if you really want to use `git push`), so I don't see any real incentive in actually pushing to it. -- Thanks, Donnie Donnie Berkholz Developer, Gentoo Linux Blog: http://dberkholz.wordpress.com [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-15 16:52 ` Mike Auty 2008-11-15 22:32 ` Zac Medico @ 2008-11-15 22:44 ` Alec Warner 2008-11-15 23:07 ` Mike Auty 1 sibling, 1 reply; 20+ messages in thread From: Alec Warner @ 2008-11-15 22:44 UTC (permalink / raw To: Mike Auty; +Cc: gentoo-scm On Sat, Nov 15, 2008 at 8:52 AM, Mike Auty <ikelos@gentoo.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Zac Medico wrote: >> It's released in portage-2.2_rc14, so please go ahead and try it if >> you'd like. > > So, can we test this on the gentoo-x86.git in overlays.g.o, or are we > trying to keep that as a clean conversion? I would imagine you are free to clone that and do whatever you want with it wrt to testing. Pushing the changes back onto o.g.o may not be a good idea at this time; depending on how well the patches work (it is nice to get people using them for a while). > > If not, will this work on normal, small git overlays or do we need a > duplicate of gentoo-x86.git that we can have access to and try commits > out on? > > Mike 5:) > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.9 (GNU/Linux) > > iEYEARECAAYFAkke/l0ACgkQu7rWomwgFXpA0wCfd8F1uHSZQjFEAMe3WOaDe9Qq > iiwAn3/fi/gnNL2NTqs1gCiGo6y4v7hp > =U3gv > -----END PGP SIGNATURE----- > > ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-15 22:44 ` Alec Warner @ 2008-11-15 23:07 ` Mike Auty 2008-11-16 4:53 ` Donnie Berkholz 0 siblings, 1 reply; 20+ messages in thread From: Mike Auty @ 2008-11-15 23:07 UTC (permalink / raw To: gentoo-scm -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Alec Warner wrote: >> I would imagine you are free to clone that and do whatever you want >> with it wrt to testing. Pushing the changes back onto o.g.o may not >> be a good idea at this time; depending on how well the patches work >> (it is nice to get people using them for a while). The idea was to test an as-close-to-the-real-thing version of git portage as possible. Would repoman commit to the local repository and then push back to the main tree, or would repoman do the push to the main tree directly? I could clone my clone to do local testing, but I thought the idea would be to try out some test/real commits to a large remote copy of the tree, and see how all the patches hold up? Mike 5:) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkkfVjcACgkQu7rWomwgFXoxGACglr8otN45871UbA9cnsjYGWiI DWMAniQd1/Ma+bgAAq4E2CZ2N7G177tX =xZMY -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-15 23:07 ` Mike Auty @ 2008-11-16 4:53 ` Donnie Berkholz 2008-11-16 6:41 ` Zac Medico ` (2 more replies) 0 siblings, 3 replies; 20+ messages in thread From: Donnie Berkholz @ 2008-11-16 4:53 UTC (permalink / raw To: Mike Auty; +Cc: gentoo-scm [-- Attachment #1: Type: text/plain, Size: 1161 bytes --] On 23:07 Sat 15 Nov , Mike Auty wrote: > The idea was to test an as-close-to-the-real-thing version of git > portage as possible. Would repoman commit to the local repository and > then push back to the main tree, or would repoman do the push to the > main tree directly? > > I could clone my clone to do local testing, but I thought the idea would > be to try out some test/real commits to a large remote copy of the tree, > and see how all the patches hold up? The patch in repoman just commits locally, it does not push. I suppose an operation or option could be added that treats that as if it were a single combined step, but I definitely do not want that as the only choice. There's still a lot of unresolved questions about the precise repository setup and work flow, so simulating it isn't entirely possible yet. One major problem with repoman+git is that a major feature of git is fast commits, but repoman is pretty slow (try a scan in x11-base/xorg-server). Anyone got a good idea for how to deal with this? -- Thanks, Donnie Donnie Berkholz Developer, Gentoo Linux Blog: http://dberkholz.wordpress.com [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-16 4:53 ` Donnie Berkholz @ 2008-11-16 6:41 ` Zac Medico 2008-11-16 6:47 ` Donnie Berkholz 2008-11-16 9:34 ` Alec Warner 2008-11-16 11:29 ` Mike Auty 2 siblings, 1 reply; 20+ messages in thread From: Zac Medico @ 2008-11-16 6:41 UTC (permalink / raw To: Donnie Berkholz; +Cc: gentoo-scm -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Donnie Berkholz wrote: > One major problem with repoman+git is that a major feature of git is > fast commits, but repoman is pretty slow (try a scan in > x11-base/xorg-server). Anyone got a good idea for how to deal with this? The main performance bottleneck is the dependency checks, and the time consumed by those is related to the number of profiles checked. # egrep -v "(#.*|^$)" /usr/portage/profiles/profiles.desc | wc -l 122 It seems excessive to check 122 profiles. If we could reduce that number significantly then we might see a significant decrease in the time consumed by repoman. - -- Thanks, Zac -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkkfwHwACgkQ/ejvha5XGaNzkQCguStWOec8fV2z8O48N5It11gi 3I0AnjXKgwIEZzkYM6dXXjAwsCdQqrsr =IhuY -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-16 6:41 ` Zac Medico @ 2008-11-16 6:47 ` Donnie Berkholz 2008-11-16 7:10 ` Zac Medico 0 siblings, 1 reply; 20+ messages in thread From: Donnie Berkholz @ 2008-11-16 6:47 UTC (permalink / raw To: Zac Medico; +Cc: gentoo-scm [-- Attachment #1: Type: text/plain, Size: 862 bytes --] On 22:41 Sat 15 Nov , Zac Medico wrote: > Donnie Berkholz wrote: > > One major problem with repoman+git is that a major feature of git is > > fast commits, but repoman is pretty slow (try a scan in > > x11-base/xorg-server). Anyone got a good idea for how to deal with this? > > The main performance bottleneck is the dependency checks, and the > time consumed by those is related to the number of profiles checked. > > # egrep -v "(#.*|^$)" /usr/portage/profiles/profiles.desc | wc -l > 122 > > It seems excessive to check 122 profiles. If we could reduce that > number significantly then we might see a significant decrease in the > time consumed by repoman. You could cut it in half by not checking 'dev' profiles by default. -- Thanks, Donnie Donnie Berkholz Developer, Gentoo Linux Blog: http://dberkholz.wordpress.com [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-16 6:47 ` Donnie Berkholz @ 2008-11-16 7:10 ` Zac Medico 0 siblings, 0 replies; 20+ messages in thread From: Zac Medico @ 2008-11-16 7:10 UTC (permalink / raw To: Donnie Berkholz; +Cc: gentoo-scm -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Donnie Berkholz wrote: > On 22:41 Sat 15 Nov , Zac Medico wrote: >> Donnie Berkholz wrote: >>> One major problem with repoman+git is that a major feature of git is >>> fast commits, but repoman is pretty slow (try a scan in >>> x11-base/xorg-server). Anyone got a good idea for how to deal with this? >> The main performance bottleneck is the dependency checks, and the >> time consumed by those is related to the number of profiles checked. >> >> # egrep -v "(#.*|^$)" /usr/portage/profiles/profiles.desc | wc -l >> 122 >> >> It seems excessive to check 122 profiles. If we could reduce that >> number significantly then we might see a significant decrease in the >> time consumed by repoman. > > You could cut it in half by not checking 'dev' profiles by default. > That seems reasonable, since broken dependencies for 'dev' profiles aren't fatal anyway. It's also worth noting that we could do much more extensive dependency checks if we moved them to the server side. For example, we could check for problems in reverse dependencies such as those encountered by some people commenting on bug 244511 about missing mit-krb5 or heimdal keywords when e2fsprogs-libs was first stabilized. [1] http://bugs.gentoo.org/show_bug.cgi?id=244511 - -- Thanks, Zac -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkkfx3YACgkQ/ejvha5XGaMIewCeL/srgIBZRpd2SS5nKaEJTUQM n3EAn1JbZPXTLwWkof1LMWn52TcJDX0Z =pUS7 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-16 4:53 ` Donnie Berkholz 2008-11-16 6:41 ` Zac Medico @ 2008-11-16 9:34 ` Alec Warner 2008-11-16 17:20 ` Donnie Berkholz 2008-11-16 18:05 ` Zac Medico 2008-11-16 11:29 ` Mike Auty 2 siblings, 2 replies; 20+ messages in thread From: Alec Warner @ 2008-11-16 9:34 UTC (permalink / raw To: Donnie Berkholz; +Cc: Mike Auty, gentoo-scm On Sat, Nov 15, 2008 at 8:53 PM, Donnie Berkholz <dberkholz@gentoo.org> wrote: > On 23:07 Sat 15 Nov , Mike Auty wrote: >> The idea was to test an as-close-to-the-real-thing version of git >> portage as possible. Would repoman commit to the local repository and >> then push back to the main tree, or would repoman do the push to the >> main tree directly? >> >> I could clone my clone to do local testing, but I thought the idea would >> be to try out some test/real commits to a large remote copy of the tree, >> and see how all the patches hold up? > > The patch in repoman just commits locally, it does not push. I suppose > an operation or option could be added that treats that as if it were a > single combined step, but I definitely do not want that as the only > choice. > > There's still a lot of unresolved questions about the precise repository > setup and work flow, so simulating it isn't entirely possible yet. > > One major problem with repoman+git is that a major feature of git is > fast commits, but repoman is pretty slow (try a scan in > x11-base/xorg-server). Anyone got a good idea for how to deal with this? Change the workflow[1]. Right now you pay the cost of running tree-wide tests every time you commit. This incentivizes developers to commit less often (to avoid paying the tax of tree-wide tests). In CVS commiting less is a problem: 1. Make Changes to a number of files. 2. Script your commits. 3. Run Script 4....N: Script commits one file at a time. 4.5: A race condition between changes you have commited to CVS versus uncommited changes occurs when CVS is synced to Osprey. This race condition can often cause tree oddness. 5. All Changes commited. 5.5: All changes synced to Osprey. I am unsure if repoman category and repoman tree-wide commits avoid this race condition. A new scheme would be: 1. Make Changes to a number of files. 2. Category or Treewide Repoman commit. 3. Run taxing tree-wide tests. 4. git commit -a (?) 5. Done! Tell me if I'm horribly wrong or missing something, it is late here. -Alec [1] I fully support server-side tests but I have low expectations for their implementation or deployment. Also it stands to reason that distributed tests scale better in the long run and it is likely that we could make our local tests run much faster if we dedicated some developer time to it. I am sensing a possible SOC project here? > > -- > Thanks, > Donnie > > Donnie Berkholz > Developer, Gentoo Linux > Blog: http://dberkholz.wordpress.com > ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-16 9:34 ` Alec Warner @ 2008-11-16 17:20 ` Donnie Berkholz 2008-11-16 21:49 ` Alec Warner 2008-11-16 18:05 ` Zac Medico 1 sibling, 1 reply; 20+ messages in thread From: Donnie Berkholz @ 2008-11-16 17:20 UTC (permalink / raw To: Alec Warner; +Cc: Mike Auty, gentoo-scm [-- Attachment #1: Type: text/plain, Size: 1429 bytes --] On 01:34 Sun 16 Nov , Alec Warner wrote: > Change the workflow[1]. > > Right now you pay the cost of running tree-wide tests every time you > commit. This incentivizes developers to commit less often (to avoid > paying the tax of tree-wide tests). > > In CVS commiting less is a problem: > > 1. Make Changes to a number of files. > 2. Script your commits. > 3. Run Script > 4....N: Script commits one file at a time. > 4.5: A race condition between changes you have commited to CVS versus > uncommited changes occurs when CVS is synced to Osprey. This race > condition can often cause tree oddness. > 5. All Changes commited. > 5.5: All changes synced to Osprey. > > I am unsure if repoman category and repoman tree-wide commits avoid > this race condition. > > A new scheme would be: > > 1. Make Changes to a number of files. > 2. Category or Treewide Repoman commit. > 3. Run taxing tree-wide tests. > 4. git commit -a (?) > 5. Done! > > Tell me if I'm horribly wrong or missing something, it is late here. Are you saying we could use the index here? Another option (that you may or may not be suggesting) could be to make a new 'repoman push' operation that actually does the hard checks, and make 'repoman commit' just do a local commit and only very quick checks. -- Thanks, Donnie Donnie Berkholz Developer, Gentoo Linux Blog: http://dberkholz.wordpress.com [-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-16 17:20 ` Donnie Berkholz @ 2008-11-16 21:49 ` Alec Warner 0 siblings, 0 replies; 20+ messages in thread From: Alec Warner @ 2008-11-16 21:49 UTC (permalink / raw To: Donnie Berkholz; +Cc: Mike Auty, gentoo-scm On Sun, Nov 16, 2008 at 9:20 AM, Donnie Berkholz <dberkholz@gentoo.org> wrote: > On 01:34 Sun 16 Nov , Alec Warner wrote: >> Change the workflow[1]. >> >> Right now you pay the cost of running tree-wide tests every time you >> commit. This incentivizes developers to commit less often (to avoid >> paying the tax of tree-wide tests). >> >> In CVS commiting less is a problem: >> >> 1. Make Changes to a number of files. >> 2. Script your commits. >> 3. Run Script >> 4....N: Script commits one file at a time. >> 4.5: A race condition between changes you have commited to CVS versus >> uncommited changes occurs when CVS is synced to Osprey. This race >> condition can often cause tree oddness. >> 5. All Changes commited. >> 5.5: All changes synced to Osprey. >> >> I am unsure if repoman category and repoman tree-wide commits avoid >> this race condition. >> >> A new scheme would be: >> >> 1. Make Changes to a number of files. >> 2. Category or Treewide Repoman commit. >> 3. Run taxing tree-wide tests. >> 4. git commit -a (?) >> 5. Done! >> >> Tell me if I'm horribly wrong or missing something, it is late here. > > Are you saying we could use the index here? Another option (that you may > or may not be suggesting) could be to make a new 'repoman push' > operation that actually does the hard checks, and make 'repoman commit' > just do a local commit and only very quick checks. That was what I was thinking. > > -- > Thanks, > Donnie > > Donnie Berkholz > Developer, Gentoo Linux > Blog: http://dberkholz.wordpress.com > ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-16 9:34 ` Alec Warner 2008-11-16 17:20 ` Donnie Berkholz @ 2008-11-16 18:05 ` Zac Medico 2008-11-16 21:55 ` Alec Warner 1 sibling, 1 reply; 20+ messages in thread From: Zac Medico @ 2008-11-16 18:05 UTC (permalink / raw To: Alec Warner; +Cc: Donnie Berkholz, Mike Auty, gentoo-scm -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Alec Warner wrote: > [1] I fully support server-side tests but I have low expectations for > their implementation or deployment. Why so low? I'm pretty optimistic. ;) > Also it stands to reason that > distributed tests scale better in the long run and it is likely that > we could make our local tests run much faster if we dedicated some > developer time to it. I am sensing a possible SOC project here? The problem with running the tests locally is that our dataset is pretty large and the types of QA tests that it needs are pretty expensive. Given these constraints, it makes more sense to do the tests on a dedicated server than to do it on a client which may have limited resources. - -- Thanks, Zac -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkkgYPMACgkQ/ejvha5XGaONpQCeJRZY1JHxreTbvQpi8R9PIlDh 6EYAoKEbAfmp5r5XplG1JhU+zMMK+jyp =R44q -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-16 18:05 ` Zac Medico @ 2008-11-16 21:55 ` Alec Warner 0 siblings, 0 replies; 20+ messages in thread From: Alec Warner @ 2008-11-16 21:55 UTC (permalink / raw To: Zac Medico; +Cc: Donnie Berkholz, Mike Auty, gentoo-scm On Sun, Nov 16, 2008 at 10:05 AM, Zac Medico <zmedico@gentoo.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Alec Warner wrote: >> [1] I fully support server-side tests but I have low expectations for >> their implementation or deployment. > > Why so low? I'm pretty optimistic. ;) Because the only person willing to work on them is you, and you are a single point of failure; + busy. > >> Also it stands to reason that >> distributed tests scale better in the long run and it is likely that >> we could make our local tests run much faster if we dedicated some >> developer time to it. I am sensing a possible SOC project here? > > The problem with running the tests locally is that our dataset is > pretty large and the types of QA tests that it needs are pretty > expensive. Given these constraints, it makes more sense to do the > tests on a dedicated server than to do it on a client which may have > limited resources. Well if you run the expensive tests on our large datasets every time someone commits you may soon destroy the scm server too without some modifications, either to the tests (a daemon that caches tree state?) or to our commit processes (pushing less often means tests run less often means server doesn't die) or some combination of these things. If the entire git tree is in a memory-backed FS; I'm not sure how fast the checks are (if we eschew the memory-based daemon thing). > - -- > Thanks, > Zac > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.9 (GNU/Linux) > > iEYEARECAAYFAkkgYPMACgkQ/ejvha5XGaONpQCeJRZY1JHxreTbvQpi8R9PIlDh > 6EYAoKEbAfmp5r5XplG1JhU+zMMK+jyp > =R44q > -----END PGP SIGNATURE----- > ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [gentoo-scm] repoman patch for git support (from drobbins) 2008-11-16 4:53 ` Donnie Berkholz 2008-11-16 6:41 ` Zac Medico 2008-11-16 9:34 ` Alec Warner @ 2008-11-16 11:29 ` Mike Auty 2 siblings, 0 replies; 20+ messages in thread From: Mike Auty @ 2008-11-16 11:29 UTC (permalink / raw To: gentoo-scm -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Donnie Berkholz wrote: > The patch in repoman just commits locally, it does not push. I suppose > an operation or option could be added that treats that as if it were a > single combined step, but I definitely do not want that as the only > choice. The main reason for wanting a push as well is to make the experience as close to previous repoman usage as possible. Changing from "repoman commit" to "repoman commit; git push" admittedly isn't much of a change, but it seems odd to expose the scm step to devs who previously could have got away with only using repoman. I'm all for a option to allow for manual pushing. However, I'd also try to keep the normal repoman commands working as similarly to the current system as possible. Just from overlay usage I'm sure there'll be cases of fixes getting committed but not pushed. What do people think? Since we're making the change, should we get devs into good habits with distributed SCM and know how it all works? Would it be better to push by default to maintain consistency with old versions? Mike 5:) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkkgBBAACgkQu7rWomwgFXpXegCcCnH8cSvJLCU4lRfwYemmohSP PJoAn0+vwm9ukQskX+pm5qjw9xyc4rh9 =ZaRl -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2008-11-16 21:56 UTC | newest] Thread overview: 20+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-11-07 1:12 [gentoo-scm] repoman patch for git support (from drobbins) Jeremy Olexa 2008-11-07 0:55 ` Robin H. Johnson 2008-11-11 10:22 ` Fabian Groffen 2008-11-11 9:36 ` Robin H. Johnson 2008-11-12 1:55 ` Zac Medico 2008-11-15 16:52 ` Mike Auty 2008-11-15 22:32 ` Zac Medico 2008-11-16 5:04 ` Donnie Berkholz 2008-11-15 22:44 ` Alec Warner 2008-11-15 23:07 ` Mike Auty 2008-11-16 4:53 ` Donnie Berkholz 2008-11-16 6:41 ` Zac Medico 2008-11-16 6:47 ` Donnie Berkholz 2008-11-16 7:10 ` Zac Medico 2008-11-16 9:34 ` Alec Warner 2008-11-16 17:20 ` Donnie Berkholz 2008-11-16 21:49 ` Alec Warner 2008-11-16 18:05 ` Zac Medico 2008-11-16 21:55 ` Alec Warner 2008-11-16 11:29 ` Mike Auty
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox