[gentoo-dev] News for Gentoo CI and pull request CI

[gentoo-dev] News for Gentoo CI and pull request CI

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2062 bytes --]

Hi, everyone.

I'd like to make a short announcement that in the last few days I've
been using my little free time to hack on my CI scripts, and they had
a few major improvements. Most notably:

1. pkgcheck now checks for outdated Manifests. In other words,
gentoo-ci will most likely complain about missing or extraneous
Manifest entries before Infra does, and pull request checks should
complain loudly about them before the PRs are merged.

2. The scripts use XML output of pkgcheck and group issues by packages,
rather than grepping text output for failure indications
and considering successive lines a single failure. Which means better
output, better permalinks, better navigation and a completely new ugly
UI.

3. Most of the persisting issues (like deprecations) are skipped now,
and all the output fits on one page. And that page is usually easier on
the browser than each of the splits were.

4. gentoo-ci compares previous check results correctly, and splits
the output into three groups: new issues, previous issues and fixes
issues.

5. gentoo-ci bisects on new issues, pinpoints the exact commit
introducing the failure and CCs both the author and the committer.
In other words, does the hard work for me and does it more efficiently.
It also has nice bisect caching support which makes bisecting a number
of issues caused by the same commit very fast.

6. pull-request checks now distinguish between issues introduced by PR
and carried over from ::gentoo. Not that you should ignore the latter
entirely since they may hide additional issues from PR.

Most of this fun stuff came from a simple obvious thing that only
recently occurred to me: I don't have to run a full pkgcheck for
bisecting! I just need to check the packages that are failing
in the most recent commit!

Of course, the whole thing is open source:

1. https://bitbucket.org/mgorny/repo-mirror-ci

2. https://bitbucket.org/mgorny/pkgcheck-result-parser

Enjoy!

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] News for Gentoo CI and pull request CI

[Pacho Ramos]

El vie, 11-12-2015 a las 23:13 +0100, Michał Górny escribió:
> Hi, everyone.
> 
> I'd like to make a short announcement that in the last few days I've
> been using my little free time to hack on my CI scripts, and they had
> a few major improvements. Most notably:
> 
> 1. pkgcheck now checks for outdated Manifests. In other words,
> gentoo-ci will most likely complain about missing or extraneous
> Manifest entries before Infra does, and pull request checks should
> complain loudly about them before the PRs are merged.
> 
> 2. The scripts use XML output of pkgcheck and group issues by
> packages,
> rather than grepping text output for failure indications
> and considering successive lines a single failure. Which means better
> output, better permalinks, better navigation and a completely new
> ugly
> UI.
> 
> 3. Most of the persisting issues (like deprecations) are skipped now,
> and all the output fits on one page. And that page is usually easier
> on
> the browser than each of the splits were.
> 
> 4. gentoo-ci compares previous check results correctly, and splits
> the output into three groups: new issues, previous issues and fixes
> issues.
> 
> 5. gentoo-ci bisects on new issues, pinpoints the exact commit
> introducing the failure and CCs both the author and the committer.
> In other words, does the hard work for me and does it more
> efficiently.
> It also has nice bisect caching support which makes bisecting a
> number
> of issues caused by the same commit very fast.
> 
> 6. pull-request checks now distinguish between issues introduced by
> PR
> and carried over from ::gentoo. Not that you should ignore the latter
> entirely since they may hide additional issues from PR.
> 
> Most of this fun stuff came from a simple obvious thing that only
> recently occurred to me: I don't have to run a full pkgcheck for
> bisecting! I just need to check the packages that are failing
> in the most recent commit!
> 
> Of course, the whole thing is open source:
> 
> 1. https://bitbucket.org/mgorny/repo-mirror-ci
> 
> 2. https://bitbucket.org/mgorny/pkgcheck-result-parser
> 
> Enjoy!
> 

Thanks a lot!

Re: [gentoo-dev] News for Gentoo CI and pull request CI

[Patrice Clement]

[-- Attachment #1: Type: text/plain, Size: 2325 bytes --]

Friday 11 Dec 2015 23:13:33, Michał Górny wrote :
> Hi, everyone.
> 
> I'd like to make a short announcement that in the last few days I've
> been using my little free time to hack on my CI scripts, and they had
> a few major improvements. Most notably:
> 
> 1. pkgcheck now checks for outdated Manifests. In other words,
> gentoo-ci will most likely complain about missing or extraneous
> Manifest entries before Infra does, and pull request checks should
> complain loudly about them before the PRs are merged.
> 
> 2. The scripts use XML output of pkgcheck and group issues by packages,
> rather than grepping text output for failure indications
> and considering successive lines a single failure. Which means better
> output, better permalinks, better navigation and a completely new ugly
> UI.
> 
> 3. Most of the persisting issues (like deprecations) are skipped now,
> and all the output fits on one page. And that page is usually easier on
> the browser than each of the splits were.
> 
> 4. gentoo-ci compares previous check results correctly, and splits
> the output into three groups: new issues, previous issues and fixes
> issues.
> 
> 5. gentoo-ci bisects on new issues, pinpoints the exact commit
> introducing the failure and CCs both the author and the committer.
> In other words, does the hard work for me and does it more efficiently.
> It also has nice bisect caching support which makes bisecting a number
> of issues caused by the same commit very fast.
> 
> 6. pull-request checks now distinguish between issues introduced by PR
> and carried over from ::gentoo. Not that you should ignore the latter
> entirely since they may hide additional issues from PR.
> 
> Most of this fun stuff came from a simple obvious thing that only
> recently occurred to me: I don't have to run a full pkgcheck for
> bisecting! I just need to check the packages that are failing
> in the most recent commit!
> 
> Of course, the whole thing is open source:
> 
> 1. https://bitbucket.org/mgorny/repo-mirror-ci
> 
> 2. https://bitbucket.org/mgorny/pkgcheck-result-parser
> 
> Enjoy!
> 
> -- 
> Best regards,
> Michał Górny
> <http://dev.gentoo.org/~mgorny/>


Awesome Michal, thanks a lot!

-- 
Patrice Clement
Gentoo Linux developer
http://www.gentoo.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] News for Gentoo CI and pull request CI

[Daniel Campbell]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 12/11/2015 02:13 PM, Michał Górny wrote:
> Hi, everyone.
> 
> I'd like to make a short announcement that in the last few days
> I've been using my little free time to hack on my CI scripts, and
> they had a few major improvements. Most notably:
> 
> 1. pkgcheck now checks for outdated Manifests. In other words, 
> gentoo-ci will most likely complain about missing or extraneous 
> Manifest entries before Infra does, and pull request checks should 
> complain loudly about them before the PRs are merged.
> 
> 2. The scripts use XML output of pkgcheck and group issues by
> packages, rather than grepping text output for failure indications 
> and considering successive lines a single failure. Which means
> better output, better permalinks, better navigation and a
> completely new ugly UI.
> 
> 3. Most of the persisting issues (like deprecations) are skipped
> now, and all the output fits on one page. And that page is usually
> easier on the browser than each of the splits were.
> 
> 4. gentoo-ci compares previous check results correctly, and splits 
> the output into three groups: new issues, previous issues and
> fixes issues.
> 
> 5. gentoo-ci bisects on new issues, pinpoints the exact commit 
> introducing the failure and CCs both the author and the committer. 
> In other words, does the hard work for me and does it more
> efficiently. It also has nice bisect caching support which makes
> bisecting a number of issues caused by the same commit very fast.
> 
> 6. pull-request checks now distinguish between issues introduced by
> PR and carried over from ::gentoo. Not that you should ignore the
> latter entirely since they may hide additional issues from PR.
> 
> Most of this fun stuff came from a simple obvious thing that only 
> recently occurred to me: I don't have to run a full pkgcheck for 
> bisecting! I just need to check the packages that are failing in
> the most recent commit!
> 
> Of course, the whole thing is open source:
> 
> 1. https://bitbucket.org/mgorny/repo-mirror-ci
> 
> 2. https://bitbucket.org/mgorny/pkgcheck-result-parser
> 
> Enjoy!
> 

Looks like you have a lot going on! That's cool. A quick question: is
this all automatic, or is there something besides repoman that
developers should be adding to their workflow to reduce the odds of
issues? I care about building a better tree and writing better builds,
but I haven't found much in the way of a "Gentoo QA Guide" that would
help a developer like me not make mistakes like that one issue with
apulse we dealt with a while back.

Any suggestions?

- -- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWbnyHAAoJEAEkDpRQOeFwY3UQANI3+WtewqvBkWqJcwMuEmUS
2Fiepdgnl9pjQZuEf9Ufmo7dPttdJELopfSmmXPKcNtqNrt44mgmf3KzwqFm9jpw
wlKL2Byeg6Ho4v0hAU6owSsK+U4iggZEpsEl21U0wWUhCVuoZgxMexTGB/lNtxv5
UwrmNPzgHyGCdzPj2bK72CiX3Ws0TPqa0u4JXZ5+wMoOkc/9+7nNsTp9F7hqyWDP
8EE06Way5RKhzLdNHwiZ6euGJbiKuOQDp9Nm/3MDC9413MU/8ztp+DUyfWa47JUB
F1HIfEao2TOESqQvQ6uarvpbUFFSN9E+nEA6HC8zXRrgk4DOKiPOZm63titFN1t2
9pzQMSJCI1ntloimEjzFnMl8jbvBSdgSGMm09MulSsJegp3uqng7W4vzhtJNCFkb
NhXkXgTSGObPqo/a8BoaW2eqZeQLNY6+iynrXd6bvXLmRZOd23eIMLN7uvWwnJs5
hm7wcHPSWJMlFm12X5XuNX/Ow7k1LbIsjsnQmS7kMvzk2x6Ezf/WmOPBQK7Czs/4
/PRWKm0ejr/pSdf57DJhm1mDSj1CjQ8kTAyk1AxjAx2pcUNPpnGNJrnYFJ8oBf9m
NdeGSXvjxukf2j/SFwJ0IGheqK8nrEZ2doLY67vbvmNBHtDqhAg8A3hqBSUzx+IP
eXDuw9fLnphOCovOjeG9
=+0LZ
-----END PGP SIGNATURE-----

Re: [gentoo-dev] News for Gentoo CI and pull request CI

[Michał Górny]

Dnia 14 grudnia 2015 09:23:40 CET, Daniel Campbell <zlg@gentoo.org> napisał(a):
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>
>On 12/11/2015 02:13 PM, Michał Górny wrote:
>> Hi, everyone.
>> 
>> I'd like to make a short announcement that in the last few days
>> I've been using my little free time to hack on my CI scripts, and
>> they had a few major improvements. Most notably:
>> 
>> 1. pkgcheck now checks for outdated Manifests. In other words, 
>> gentoo-ci will most likely complain about missing or extraneous 
>> Manifest entries before Infra does, and pull request checks should 
>> complain loudly about them before the PRs are merged.
>> 
>> 2. The scripts use XML output of pkgcheck and group issues by
>> packages, rather than grepping text output for failure indications 
>> and considering successive lines a single failure. Which means
>> better output, better permalinks, better navigation and a
>> completely new ugly UI.
>> 
>> 3. Most of the persisting issues (like deprecations) are skipped
>> now, and all the output fits on one page. And that page is usually
>> easier on the browser than each of the splits were.
>> 
>> 4. gentoo-ci compares previous check results correctly, and splits 
>> the output into three groups: new issues, previous issues and
>> fixes issues.
>> 
>> 5. gentoo-ci bisects on new issues, pinpoints the exact commit 
>> introducing the failure and CCs both the author and the committer. 
>> In other words, does the hard work for me and does it more
>> efficiently. It also has nice bisect caching support which makes
>> bisecting a number of issues caused by the same commit very fast.
>> 
>> 6. pull-request checks now distinguish between issues introduced by
>> PR and carried over from ::gentoo. Not that you should ignore the
>> latter entirely since they may hide additional issues from PR.
>> 
>> Most of this fun stuff came from a simple obvious thing that only 
>> recently occurred to me: I don't have to run a full pkgcheck for 
>> bisecting! I just need to check the packages that are failing in
>> the most recent commit!
>> 
>> Of course, the whole thing is open source:
>> 
>> 1. https://bitbucket.org/mgorny/repo-mirror-ci
>> 
>> 2. https://bitbucket.org/mgorny/pkgcheck-result-parser
>> 
>> Enjoy!
>> 
>
>Looks like you have a lot going on! That's cool. A quick question: is
>this all automatic, or is there something besides repoman that
>developers should be adding to their workflow to reduce the odds of
>issues? I care about building a better tree and writing better builds,
>but I haven't found much in the way of a "Gentoo QA Guide" that would
>help a developer like me not make mistakes like that one issue with
>apulse we dealt with a while back.
>
>Any suggestions?

It's all automatic and pretty much a subset of what repoman does. So, no, you don't need to change anything in your workflow. Just remember to run repoman.

However, if you're planning some potentially breaking change, you can file a PR to get free full-tree scan with pkgcheck without having to implement it yourself. Though i should probably provide a script making full-tree pkgcheck scans easier.

>
>- -- 
>Daniel Campbell - Gentoo Developer
>OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
>fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v2
>
>iQIcBAEBCAAGBQJWbnyHAAoJEAEkDpRQOeFwY3UQANI3+WtewqvBkWqJcwMuEmUS
>2Fiepdgnl9pjQZuEf9Ufmo7dPttdJELopfSmmXPKcNtqNrt44mgmf3KzwqFm9jpw
>wlKL2Byeg6Ho4v0hAU6owSsK+U4iggZEpsEl21U0wWUhCVuoZgxMexTGB/lNtxv5
>UwrmNPzgHyGCdzPj2bK72CiX3Ws0TPqa0u4JXZ5+wMoOkc/9+7nNsTp9F7hqyWDP
>8EE06Way5RKhzLdNHwiZ6euGJbiKuOQDp9Nm/3MDC9413MU/8ztp+DUyfWa47JUB
>F1HIfEao2TOESqQvQ6uarvpbUFFSN9E+nEA6HC8zXRrgk4DOKiPOZm63titFN1t2
>9pzQMSJCI1ntloimEjzFnMl8jbvBSdgSGMm09MulSsJegp3uqng7W4vzhtJNCFkb
>NhXkXgTSGObPqo/a8BoaW2eqZeQLNY6+iynrXd6bvXLmRZOd23eIMLN7uvWwnJs5
>hm7wcHPSWJMlFm12X5XuNX/Ow7k1LbIsjsnQmS7kMvzk2x6Ezf/WmOPBQK7Czs/4
>/PRWKm0ejr/pSdf57DJhm1mDSj1CjQ8kTAyk1AxjAx2pcUNPpnGNJrnYFJ8oBf9m
>NdeGSXvjxukf2j/SFwJ0IGheqK8nrEZ2doLY67vbvmNBHtDqhAg8A3hqBSUzx+IP
>eXDuw9fLnphOCovOjeG9
>=+0LZ
>-----END PGP SIGNATURE-----

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.