[gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: net-fs/samba/, net-fs/samba/files/, net-fs/samba/files/4.4/

[gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: net-fs/samba/, net-fs/samba/files/, net-fs/samba/files/4.4/

[Robin H. Johnson]

Your commit is missing a DIST entry for:
samba-disable-python-patches-4.4.0.tar.xz
As such, is breaking git->rsync export.
Please fix ASAP.

Was this a partial repoman used, because the commit log shows a Portage
line, yet the DIST is still missing.

On Tue, Feb 09, 2016 at 10:06:42PM +0000, Ian Stakenvicius wrote:
> commit:     bedb32a1fbb6675c639446b39d25d4f13e2d4b67
> Author:     Ian Stakenvicius <axs <AT> gentoo <DOT> org>
> AuthorDate: Tue Feb  9 20:37:31 2016 +0000
> Commit:     Ian Stakenvicius <axs <AT> gentoo <DOT> org>
> CommitDate: Tue Feb  9 22:06:20 2016 +0000
> URL:        https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=bedb32a1
> 
> net-fs/samba: add ebuild for 4.4.0rc2
> 
> Package-Manager: portage-2.2.26
> 
>  net-fs/samba/Manifest                    |   1 +
>  net-fs/samba/files/4.4/samba4.confd      |  38 +++++
>  net-fs/samba/files/4.4/samba4.initd-r1   |  56 ++++++++
>  net-fs/samba/files/samba-4.4.0-pam.patch |  29 ++++
>  net-fs/samba/samba-4.4.0_rc2.ebuild      | 233 +++++++++++++++++++++++++++++++
>  5 files changed, 357 insertions(+)
> 
> diff --git a/net-fs/samba/Manifest b/net-fs/samba/Manifest
> index e318a0a..5faacd4 100644
> --- a/net-fs/samba/Manifest
> +++ b/net-fs/samba/Manifest
> @@ -4,6 +4,7 @@ DIST samba-3.6.25.tar.gz 34121828 SHA256 8f2c8a7f2bd89b0dfd228ed917815852f7c625b
>  DIST samba-4.2.7.tar.gz 20741971 SHA256 f586ab3166ce4c663360f15b1de24ef083816a5471856e3ad49bc26b35f0104a SHA512 74314083c04689696f0423bc990947bfafad679edcac97e6c137e99c17de1e262a4d8450b57de733a70c86c746300c7c5a1365b56c0e353ce79b05e0baf8eb9a WHIRLPOOL 84e7d2f3a60701ee929198caf86371c9e1694be6def47a4f0f12d4d221b995209505c23564c304fbdd95ab5ae528f941946bd361ec6e388f7ba4db08792ff3ba
>  DIST samba-4.2.8.tar.gz 20745527 SHA256 d2c0ca97ab415ede829d15ddad411d76e4f7b6a82e280bf7fbc9910c30fa4593 SHA512 cdee04ebc2303c1cadf2c0a45530909b6c97838e611378498faaaa6fcade8850746253d51ae71fd872c741f54ec2d3a9d452651291355e20001ca443fae9054a WHIRLPOOL 84b3f78b41da98eaa463f9b1c467e3c82268d31ac3d3e48d75b0a4dc04f479d12f2387c045281b0caa3a841c351587d0eabec403163ff479d8c700f0b638e5f4
>  DIST samba-4.3.4.tar.gz 20434434 SHA256 5d0eb52e842832af922f7d57716eacff23192906ec3bdf6727e18ca24f1419d9 SHA512 021351534a70cd351934d7f8bfc3c4e9ed9ea3f11f778f6f9d076b3368103f7f478ff1745cb257de0bf2ee38ae76ecba58e01a4db6cbcacbd8a4876e8e1b30f2 WHIRLPOOL 328721951ed932c5813d6157ca2933e22adb793d5cd6667577e40151bcdae8dcddf5ca4e053cd6494e0f82f5801ae480716520c625dd9c337557abc168e00dec
> +DIST samba-4.4.0rc2.tar.gz 20569387 SHA256 931c6241f239621244fc170f9a5b188c024fe578279c28494dc40e696e5572e3 SHA512 5258eb588f0e553e4d8742440ea9da1e91a4bbdc33fafd2c070e58a1b473bcf2d4a1c4db787856535b3404b08002a639f1de4c182f662284d0c441617ca74977 WHIRLPOOL d0ab40694bee7330fd06f9fd8f461b890ccac36e3366247d020cf9ae284ccd80dc5f1c98e029a5cd38bca65cd092cb4094c6d2c9644212d2e212f2a015c3b720
>  DIST samba-disable-python-patches-4.2.7.tar.xz 6296 SHA256 06a1b9aeb91b622d3c2a02a86edfc26e26f10303699c8b2badbd21ce68b10ec0 SHA512 ff746c2969b254d9ccad1440699fccd5958222eea8284a8e068b96df377d6cea8551ec3c6be7103cebf227b0b9038a5b06d3b06d9b247e181403e9fe1ad7eedf WHIRLPOOL 8ea9f34c5f011624b43c0f3f27601574c27e00c5a728d9af5b1cece090da362d51f93ef6cecd37f1204bbd4e608ed58027f52ef5c3d700a1cfdbbb0e5355c3b8
>  DIST samba-disable-python-patches-4.3.3.tar.xz 6016 SHA256 00debe6c5cc57b87150ded67db8dc54e5ec487f6ed610c96e8fa393743c47f66 SHA512 775abcee86690605e156f4c560f25d762f5cc2e72177a55003ad5124ed643322f2c84514342ed0eadad2c8e1ea97006bc6ce7d504ca8a29c27a201666ce4bdf6 WHIRLPOOL 86c40669e706f6c3b955e6fb892931532e241dd92cae2e7b5986e78f6b5fe50c42c019b97650942de81c8c4989568bcb93e49a7bcb2f9fd300d189da5fa08fe4
>  DIST smb_traffic_analyzer_v2.diff.bz2 12226 SHA256 1bae7eafbe8ac2382313d5ab9d43d73ba64b63a714f0f588516952d476fb868d SHA512 aa0e457a0dd282e61e6dfcd5705c29b319832dca9711b1b5baf8373e2f079991399c3537c050219ccb861a93f86353ebff677a5c625d2e3f1f3a13ee5c4087d0 WHIRLPOOL 85ee72a360f67ebe71be5cd400ecd635280a0d7c64ebb8b94656a5ef1a94f74a987de86408af00ce1b81cc8363b1b3cf14726860d29b72ee610d4bab73d6b139
...
> 
> --- /dev/null
> +++ b/net-fs/samba/samba-4.4.0_rc2.ebuild
...
> +SRC_URI="mirror://samba/${SRC_PATH}/${MY_P}.tar.gz
> +	https://dev.gentoo.org/~axs/distfiles/samba-disable-python-patches-4.4.0.tar.xz"
...

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: net-fs/samba/, net-fs/samba/files/, net-fs/samba/files/4.4/

[Tim Harder]

[-- Attachment #1: Type: text/plain, Size: 669 bytes --]

On 2016-02-09 18:50, Robin H. Johnson wrote:
>Your commit is missing a DIST entry for:
>samba-disable-python-patches-4.4.0.tar.xz
>As such, is breaking git->rsync export.
>Please fix ASAP.
>
>Was this a partial repoman used, because the commit log shows a Portage
>line, yet the DIST is still missing.

Just to note, devs that make these mistakes should already get emails from the
CI test setup [1]. If those emails need to be more specific about the severity
I'm sure that could be changed so you don't need to act like a CI bot if you
don't want to. ;)

Thanks,
Tim

[1]: https://archives.gentoo.org/gentoo-automated-testing/message/79a7b4fe2afaa71de9ef1189464f8324

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: net-fs/samba/, net-fs/samba/files/, net-fs/samba/files/4.4/

[Robin H. Johnson]

On Tue, Feb 09, 2016 at 07:03:28PM -0500, Tim Harder wrote:
> Just to note, devs that make these mistakes should already get emails from the
> CI test setup [1]. If those emails need to be more specific about the severity
> I'm sure that could be changed so you don't need to act like a CI bot if you
> don't want to. ;)
> [1]: https://archives.gentoo.org/gentoo-automated-testing/message/79a7b4fe2afaa71de9ef1189464f8324
Well then raise the severity, because it's still blocking rsync as
nobody fixed it.

I have reverted it now, so that rsync continues to propagate (it would
be nice if repoman manifest-check could be non-fatal about missing DIST
entries, but strict about everything else).

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: net-fs/samba/, net-fs/samba/files/, net-fs/samba/files/4.4/

[Tim Harder]

[-- Attachment #1: Type: text/plain, Size: 1166 bytes --]

On 2016-02-09 19:59, Robin H. Johnson wrote:
>On Tue, Feb 09, 2016 at 07:03:28PM -0500, Tim Harder wrote:
>> Just to note, devs that make these mistakes should already get emails from the
>> CI test setup [1]. If those emails need to be more specific about the severity
>> I'm sure that could be changed so you don't need to act like a CI bot if you
>> don't want to. ;)
>> [1]: https://archives.gentoo.org/gentoo-automated-testing/message/79a7b4fe2afaa71de9ef1189464f8324
>Well then raise the severity, because it's still blocking rsync as
>nobody fixed it.
>
>I have reverted it now, so that rsync continues to propagate (it would
>be nice if repoman manifest-check could be non-fatal about missing DIST
>entries, but strict about everything else).

I imagine in the end it would be better to put some of the faster checks
into the git hook pipeline itself. For example, reject commits at push
time that are missing DIST entries. 

Since I'm not familiar with the infra setup all I can do is provide
support if it's feasible and someone wants to implement this using
pkgcore/pkgcheck similar to what the CI stuff already does.

Thanks,
Tim

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: net-fs/samba/, net-fs/samba/files/, net-fs/samba/files/4.4/

[Robin H. Johnson]

On Tue, Feb 09, 2016 at 08:08:33PM -0500, Tim Harder wrote:
> I imagine in the end it would be better to put some of the faster checks
> into the git hook pipeline itself. For example, reject commits at push
> time that are missing DIST entries. 
> 
> Since I'm not familiar with the infra setup all I can do is provide
> support if it's feasible and someone wants to implement this using
> pkgcore/pkgcheck similar to what the CI stuff already does.
I will happily take a pre-receive hook, regardless of it using
pkgcore/portage. Note that it should operate WITHOUT having a checkout,
needs to be able to take new ebuilds & Manifest on stdin.

Trying to checkout the tree for each update I don't think is a good
idea, as it's 5+ seconds of overhead; and somebody else could try to
push a change in the meantime.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: net-fs/samba/, net-fs/samba/files/, net-fs/samba/files/4.4/

[Robin H. Johnson]

On Wed, Feb 10, 2016 at 01:24:16AM +0000, Robin H. Johnson wrote:
> I will happily take a pre-receive hook, regardless of it using
> pkgcore/portage. Note that it should operate WITHOUT having a checkout,
> needs to be able to take new ebuilds & Manifest on stdin.
> 
> Trying to checkout the tree for each update I don't think is a good
> idea, as it's 5+ seconds of overhead; and somebody else could try to
> push a change in the meantime.
Slight clarification here:
I am willing to keep a stateful checkout as a post-receive hook, and the
pre-receive hook CAN use that checkout to read other contents. It just
might be more than one commit behind sometimes.

Eg:
T=1 push P1 comes in, triggers pre-receive, passes, triggers post-receive hook (async)
T=1..5 working tree in the process of post-receive, state is dirty [1]
T=2 push P2 comes in, triggers pre-receive. What should happen?

What should the pre-receive do in this case?
A) fail outright? (terrible choice, included for completeness)
B) block on the active post-receive completing? (while !done; sleep)
C) validate against a changing tree? (might fail, not in a way that's
easy to reproduce)
D) validate against a previous CLEAN tree? (if this push depends on a
change in push P1, it will fail when it should have passed)

[1] actually a really good question here, does git prepare changes
somewhere in .git and atomically rename into place, even so, multiple
moves cannot be atomic together.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: net-fs/samba/, net-fs/samba/files/, net-fs/samba/files/4.4/

[Jason Zaman]

On Wed, Feb 10, 2016 at 05:11:20AM +0000, Robin H. Johnson wrote:
> On Wed, Feb 10, 2016 at 01:24:16AM +0000, Robin H. Johnson wrote:
> > I will happily take a pre-receive hook, regardless of it using
> > pkgcore/portage. Note that it should operate WITHOUT having a checkout,
> > needs to be able to take new ebuilds & Manifest on stdin.
> > 
> > Trying to checkout the tree for each update I don't think is a good
> > idea, as it's 5+ seconds of overhead; and somebody else could try to
> > push a change in the meantime.
> Slight clarification here:
> I am willing to keep a stateful checkout as a post-receive hook, and the
> pre-receive hook CAN use that checkout to read other contents. It just
> might be more than one commit behind sometimes.
> 
> Eg:
> T=1 push P1 comes in, triggers pre-receive, passes, triggers post-receive hook (async)
> T=1..5 working tree in the process of post-receive, state is dirty [1]
> T=2 push P2 comes in, triggers pre-receive. What should happen?
> 
> What should the pre-receive do in this case?
> A) fail outright? (terrible choice, included for completeness)
> B) block on the active post-receive completing? (while !done; sleep)
> C) validate against a changing tree? (might fail, not in a way that's
> easy to reproduce)
> D) validate against a previous CLEAN tree? (if this push depends on a
> change in push P1, it will fail when it should have passed)
> 
> [1] actually a really good question here, does git prepare changes
> somewhere in .git and atomically rename into place, even so, multiple
> moves cannot be atomic together.
> 
> -- 
> Robin Hugh Johnson
> Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
> E-Mail     : robbat2@gentoo.org
> GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
> 

We do not need the whole manifest checking, we just need to have a list
of SRC_URI that is being added in any ebuilds and then grep Manifest if
those filenames exist. The receive hook checking the hashes of the files
is probably asking too much. This would miss any files added by eclasses
but that is rare and changes less often so we should probably be okay.

Is there any sane way to spit out condensed SRC_URI for all use-flags?

-- Jason

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: net-fs/samba/, net-fs/samba/files/, net-fs/samba/files/4.4/

[Robin H. Johnson]

On Wed, Feb 10, 2016 at 01:22:22PM +0800, Jason Zaman wrote:
> We do not need the whole manifest checking, we just need to have a list
> of SRC_URI that is being added in any ebuilds and then grep Manifest if
> those filenames exist. The receive hook checking the hashes of the files
> is probably asking too much. This would miss any files added by eclasses
> but that is rare and changes less often so we should probably be okay.
> 
> Is there any sane way to spit out condensed SRC_URI for all use-flags?
eclasses add a LOT. 
Of the ~40k ebuilds in the tree, ~11k do not match the regex /^\w*SRC_URI/.

This means that you need an update to date copy of eclasses at the very
least. If Portage/pkgcore could work on a tree that had only a partial
Git checkout of: eclasses/, metadata/, cat/pn/
Then this probably COULD run much faster.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85