[gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 365 bytes --]

All,

I understood the usefulness of this line to some when we were using CVS
since it expanded into the ebuild revision, date, etc.

This expansion doesn't take place under git, so now I don't understand
the usefulness of this line. If I have missed something, can someone
fill me in, or if it isn't useful any more can we consider removing it?

Thanks,

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Mikle Kolyada]

13.08.2015 18:36, William Hubbs пишет:
> All,
>
> I understood the usefulness of this line to some when we were using CVS
> since it expanded into the ebuild revision, date, etc.
>
> This expansion doesn't take place under git, so now I don't understand
> the usefulness of this line. If I have missed something, can someone
> fill me in, or if it isn't useful any more can we consider removing it?
>
> Thanks,
>
> William
>
Hmm, the same question here. As far as i remember git does not depend on
any ebuild's header.

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Matthias Maier]

[-- Attachment #1: Type: text/plain, Size: 433 bytes --]

> This expansion doesn't take place under git, so now I don't understand
> the usefulness of this line. If I have missed something, can someone
> fill me in, or if it isn't useful any more can we consider removing it?

It is possible to define custom keyword expansions for $Id$ with
.gitattributes (man gitattributes /ident).

This might be of some use when converting a given git checkout to a tree
snapshot, etc.

Best,
Matthias


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 820 bytes --]

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 434 bytes --]

On 13 Aug 2015 10:36, William Hubbs wrote:
> I understood the usefulness of this line to some when we were using CVS
> since it expanded into the ebuild revision, date, etc.
> 
> This expansion doesn't take place under git, so now I don't understand
> the usefulness of this line. If I have missed something, can someone
> fill me in, or if it isn't useful any more can we consider removing it?

delete it and be done
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Tyler Pohl]

[-- Attachment #1: Type: text/plain, Size: 542 bytes --]

Can someone please tell me how to get off this mailing list?  Please please
please
On Aug 13, 2015 8:37 AM, "William Hubbs" <williamh@gentoo.org> wrote:

> All,
>
> I understood the usefulness of this line to some when we were using CVS
> since it expanded into the ebuild revision, date, etc.
>
> This expansion doesn't take place under git, so now I don't understand
> the usefulness of this line. If I have missed something, can someone
> fill me in, or if it isn't useful any more can we consider removing it?
>
> Thanks,
>
> William
>
>

[-- Attachment #2: Type: text/html, Size: 841 bytes --]

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Matt Turner]

On Thu, Aug 13, 2015 at 9:18 AM, Tyler Pohl <tylerapohl@gmail.com> wrote:
> Can someone please tell me how to get off this mailing list?  Please please
> please

Well first you don't reply to an unrelated topic...

But the instructions are here:
https://www.gentoo.org/get-involved/mailing-lists/instructions.html

(the thing you find by googling something like "gentoo unsubscribe")

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Rich Freeman]

On Thu, Aug 13, 2015 at 12:12 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> On 13 Aug 2015 10:36, William Hubbs wrote:
>> I understood the usefulness of this line to some when we were using CVS
>> since it expanded into the ebuild revision, date, etc.
>>
>> This expansion doesn't take place under git, so now I don't understand
>> the usefulness of this line. If I have missed something, can someone
>> fill me in, or if it isn't useful any more can we consider removing it?
>
> delete it and be done

When I asked a few days ago the arugment was made that it will be
expanded when the ebuilds hit rsync, and then users can reference
these when submitting bugs so that devs know what revision they're
using/etc.  It was stated that this was a highly-requested feature.

I'm still personally leaning towards delete it and be done for a few reasons:

1.  If the user syncs from git and not from rsync (which I suspect
will be come steadily more popular - git is very efficient at syncing
though it takes more space), then they won't get the hash, though they
can of course just ask git for it.

2.  Users don't routinely attach ebuilds to bugs, so we'll still be
pestering them for this info.

3.  That hash doesn't provide any kind of at-a-glance info anyway.
You're going to to have to ask git what it is.  It is a unique ID for
the file though.

I think IDs embedded in files make more sense for workflows that
envision a lot of out-of-git work, since they tie back into the git
tree.  If you stay in git all the time, they're redundant.

But, the biggest reason IMHO is that it is in-band signaling and I can
forsee all the issues we ran into with cvs keywords.  The git
validation and migration work constantly ran into this as probably our
#1 unfixable problem, and Robin's go-live emails indicated that we
still had tons of patches that had keyword expansion issues.  The job
of an scm should be to store files and regurgitate them on request.
If you ask it to also mangle their contents you're inevitably asking
it to mangle their contents in ways you didn't forsee.

-- 
Rich

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

>>>>> On Thu, 13 Aug 2015, Rich Freeman wrote:

> On Thu, Aug 13, 2015 at 12:12 PM, Mike Frysinger <vapier@gentoo.org> wrote:
>> delete it and be done

+1

> When I asked a few days ago the arugment was made that it will be
> expanded when the ebuilds hit rsync, and then users can reference
> these when submitting bugs so that devs know what revision they're
> using/etc.  It was stated that this was a highly-requested feature.

If this is "a highly-requested feature" then some discussion or other
reference to it should exist in mailing lists. I can neither remember
nor can I find such a discussion.

So, could someone please provide a pointer to when and where keeping
the $Id$ line was requested?

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1053 bytes --]

Dnia 2015-08-13, o godz. 19:13:06
Ulrich Mueller <ulm@gentoo.org> napisał(a):

> >>>>> On Thu, 13 Aug 2015, Rich Freeman wrote:
> 
> > On Thu, Aug 13, 2015 at 12:12 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> >> delete it and be done
> 
> +1
> 
> > When I asked a few days ago the arugment was made that it will be
> > expanded when the ebuilds hit rsync, and then users can reference
> > these when submitting bugs so that devs know what revision they're
> > using/etc.  It was stated that this was a highly-requested feature.
> 
> If this is "a highly-requested feature" then some discussion or other
> reference to it should exist in mailing lists. I can neither remember
> nor can I find such a discussion.
> 
> So, could someone please provide a pointer to when and where keeping
> the $Id$ line was requested?

Just to be clear, Infra did it and replaced all $Header$ with $Id$.
Removing it right now without discussion with them would be wrong.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Mike Gilbert]

On Thu, Aug 13, 2015 at 1:13 PM, Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>> On Thu, 13 Aug 2015, Rich Freeman wrote:
>
>> On Thu, Aug 13, 2015 at 12:12 PM, Mike Frysinger <vapier@gentoo.org> wrote:
>>> delete it and be done
>
> +1
>
>> When I asked a few days ago the arugment was made that it will be
>> expanded when the ebuilds hit rsync, and then users can reference
>> these when submitting bugs so that devs know what revision they're
>> using/etc.  It was stated that this was a highly-requested feature.
>
> If this is "a highly-requested feature" then some discussion or other
> reference to it should exist in mailing lists. I can neither remember
> nor can I find such a discussion.

FYI, there has been one bug report about it since the git migration happened.

https://bugs.gentoo.org/show_bug.cgi?id=557386

To quote the reporter:

"Whilst many users may not care, this makes it much more difficult to
tell whether a change has been made to an ebuild without a
version-bump - and in my specific case, I maintain an overlay repo
which attempts to track upstream, and this has removed my ability to
(again, easily) tell if a minor update has been made to an ebuild I've
overlaid."

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 749 bytes --]

>>>>> On Thu, 13 Aug 2015, Michał Górny wrote:

> Dnia 2015-08-13, o godz. 19:13:06
> Ulrich Mueller <ulm@gentoo.org> napisał(a):

>> If this is "a highly-requested feature" then some discussion or
>> other reference to it should exist in mailing lists. I can neither
>> remember nor can I find such a discussion.
>> 
>> So, could someone please provide a pointer to when and where
>> keeping the $Id$ line was requested?

> Just to be clear, Infra did it and replaced all $Header$ with $Id$.
> Removing it right now without discussion with them would be wrong.

I guess nobody suggests to remove it without discussion.

Still, I'd like to see some evidence for the above claim that keeping
the Id was highly requested.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Is the $Id$ line in our ebuilds still useful?

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 353 bytes --]

On Thu, 13 Aug 2015 19:13:06 +0200 Ulrich Mueller wrote:
> >>>>> On Thu, 13 Aug 2015, Rich Freeman wrote:
> 
> > On Thu, Aug 13, 2015 at 12:12 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> >> delete it and be done
> 
> +1

+1 to remove this CVS remnant. It was needed before, it is not
really needed now.
 
Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-dev] Infra plans regarding $Id$ - official answer...

[Robin H. Johnson]

On Thu, Aug 13, 2015 at 10:36:16AM -0500, William Hubbs wrote:
> I understood the usefulness of this line to some when we were using CVS
> since it expanded into the ebuild revision, date, etc.
> 
> This expansion doesn't take place under git, so now I don't understand
> the usefulness of this line. If I have missed something, can someone
> fill me in, or if it isn't useful any more can we consider removing it?
The following is the official answer of Infra, regarding the $Id$
expansion.

The intent is that the ONLY place the keywords are expanded, will be in
the rsync export. FUTURE tense, it's not ready yet.

If there is demand (and I think the consensus is
actually the OPPOSITE), we could also have it expand on your local
checkouts.

It expands to the hash of the blob of that file; and from that, you can
identify which commits the blob exists in.

The primary use case of it is to allow users to easily see what version
of a given ebuild they are using.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Rich Freeman]

On Thu, Aug 13, 2015 at 3:43 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
>
> The intent is that the ONLY place the keywords are expanded, will be in
> the rsync export. FUTURE tense, it's not ready yet.
>

Will that include any case where the string "$Id$" appears in a patch file?

That is the main source of problems here.

Really, anytime what the dev tests and commits is different from what
the users see is a potential source of problems.

-- 
Rich

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Robin H. Johnson]

On Thu, Aug 13, 2015 at 03:59:37PM -0400, Rich Freeman wrote:
> On Thu, Aug 13, 2015 at 3:43 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> > The intent is that the ONLY place the keywords are expanded, will be in
> > the rsync export. FUTURE tense, it's not ready yet.
> Will that include any case where the string "$Id$" appears in a patch file?
Those should be ripped out with extreme prejudice, ditto other
identifiers.

I'm thinking of having an explicit whitelist for expansion, but need to
figure out more consistency for some parts to do it.

> That is the main source of problems here.
> 
> Really, anytime what the dev tests and commits is different from what
> the users see is a potential source of problems.
If you want that, then you can expand it on your local systems.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Rich Freeman]

On Thu, Aug 13, 2015 at 7:28 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> On Thu, Aug 13, 2015 at 03:59:37PM -0400, Rich Freeman wrote:
>> On Thu, Aug 13, 2015 at 3:43 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
>> > The intent is that the ONLY place the keywords are expanded, will be in
>> > the rsync export. FUTURE tense, it's not ready yet.
>> Will that include any case where the string "$Id$" appears in a patch file?
> Those should be ripped out with extreme prejudice, ditto other
> identifiers.
>

Seems like this should be a repoman warning at the very least,
assuming we keep them at all.

>
>> That is the main source of problems here.
>>
>> Really, anytime what the dev tests and commits is different from what
>> the users see is a potential source of problems.
> If you want that, then you can expand it on your local systems.
>

The problem with this is that it depends on every dev doing something.

The original purpose of this was so that people could tell if a file
changes without it being revbumped?  If so, rather than parsing an
sha1 out of the file, why not just hash the file yourself?

-- 
Rich

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Kent Fredric]

On 14 August 2015 at 07:59, Rich Freeman <rich0@gentoo.org> wrote:
> Will that include any case where the string "$Id$" appears in a patch file?


Surely that can be avoided by using a git attributes specification
that only applies to  */*/*.ebuild ?

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 514 bytes --]

Dnia 2015-08-14, o godz. 13:55:47
Kent Fredric <kentfredric@gmail.com> napisał(a):

> On 14 August 2015 at 07:59, Rich Freeman <rich0@gentoo.org> wrote:
> > Will that include any case where the string "$Id$" appears in a patch file?
> 
> 
> Surely that can be avoided by using a git attributes specification
> that only applies to  */*/*.ebuild ?

Don't force the implicit expansion on all developers and users forking
the repo.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Kent Fredric]

On 14 August 2015 at 17:45, Michał Górny <mgorny@gentoo.org> wrote:
> Don't force the implicit expansion on all developers and users forking
> the repo.


You wouldn't, I thought we were talking about $Id$ only being expanded
on the git->rsync transition, and people worrying that $Id$ would be
expanded in patches, where the sensible thing would be to carefully
filter which files $Id$ expansion happens instead of relying on all
files in all folders to not have patterns that just happen to match
$Id(: .*)$


-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Daniel Campbell (zlg)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 08/13/2015 12:43 PM, Robin H. Johnson wrote:
> On Thu, Aug 13, 2015 at 10:36:16AM -0500, William Hubbs wrote:
>> I understood the usefulness of this line to some when we were
>> using CVS since it expanded into the ebuild revision, date, etc.
>> 
>> This expansion doesn't take place under git, so now I don't
>> understand the usefulness of this line. If I have missed
>> something, can someone fill me in, or if it isn't useful any more
>> can we consider removing it?
> The following is the official answer of Infra, regarding the $Id$ 
> expansion.
> 
> The intent is that the ONLY place the keywords are expanded, will
> be in the rsync export. FUTURE tense, it's not ready yet.
> 
> If there is demand (and I think the consensus is actually the
> OPPOSITE), we could also have it expand on your local checkouts.
> 
> It expands to the hash of the blob of that file; and from that, you
> can identify which commits the blob exists in.
> 
> The primary use case of it is to allow users to easily see what
> version of a given ebuild they are using.
> 

I honestly don't see the point of this when `git log` or even `git
diff` or standard `diff` will tell you if what's in your overlay
differs from the source. With some bash magic it could even be
automated. The point of that 'feature' is to see what, if anything,
has changed between one's overlay and Gentoo's running tree. A diff
would not only be able to tell you *if* anything changed, but also
*what*, without adding around 5-7 extra bytes per ebuild. Sure, it's
only bytes, but when multiplied against the number of ebuilds we have,
it can make a few hundred KB difference. When expanded, that number
multiplies. Is it worth adding this extra bloat to something that a
standard utility can expose better than a hash?

Just my two cents.

- -- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJVzbCnAAoJEAEkDpRQOeFwlvkP/1KVnAH09LmHlO7kFPdFhEvE
IhoscaZc/Pve4QcLMwnWAq2T3Uq4EzqYW2hICyuOAvp6bvca1ybpv7U6k+FYws8V
lSarmFidfd0LKRqwPzrEjZb6kVxkKefzsAyqAZK9JcU5AhkI6cjIjMRihbxSVAJv
BphWmbZBVuU4ZmoRiUcGR0Qzhcd4D4K0qjk6R4r5yCKaU5ACXj+ul5FOiD4GmsKc
288YgkLO8l+MQhLAQ5Ie6lL5E3tfVzgJ2U0F6R7xIG0uT8kXU5p7OuBN3ATEP96E
P92T6QIOGug4fjjpdInjQPMaY+NnF3x6LshHgRQXHO1lWNIceJKTX+em4VqU4JEH
UCsWgh+QPppPXrXTNE0J94BMK0DLP2iiHxYFtKT8u7ABfhAOvuTbA6B/H2ujjIaT
919htZGbSt3JkLCo5/gxqrJbntmqG0hehYr25C/XTHXJ+c5B5reaWKwuYW1Smpg7
whVLUcEHVFiU32bMFiETKfNVIGa3mxNUfO9wHpqBD4lSUNr3eJao5R25t5NMDJkL
/znRsFb5h49uZEASjBWTICIsKjfjqaydRS5oQ9VZZcxdo3azZboqxLeuEAUnSSrn
H8/F6HNbSJ5MVetRG2DrLpWFCS94LSjhF4DYQk0xzK3lvMXHMrdhZI0G94ZTx9PW
X2I5+Y9cj1mVLzWrxHyu
=W4Zd
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1089 bytes --]

On Fri, 14 Aug 2015 02:11:09 -0700 Daniel Campbell (zlg) wrote:
> I honestly don't see the point of this when `git log` or even `git
> diff` or standard `diff` will tell you if what's in your overlay
> differs from the source. With some bash magic it could even be
> automated. The point of that 'feature' is to see what, if anything,
> has changed between one's overlay and Gentoo's running tree. A diff
> would not only be able to tell you *if* anything changed, but also
> *what*, without adding around 5-7 extra bytes per ebuild. Sure, it's
> only bytes, but when multiplied against the number of ebuilds we have,
> it can make a few hundred KB difference. When expanded, that number
> multiplies. Is it worth adding this extra bloat to something that a
> standard utility can expose better than a hash?

Agree here. Also I don't like the idea of post-modifying content of
signed commits: files developers committed to the tree should be
the same users get. As a side effect this will simplify tree
consistency checks and forensics.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[hasufell]

On 08/14/2015 01:10 PM, Andrew Savchenko wrote:
> On Fri, 14 Aug 2015 02:11:09 -0700 Daniel Campbell (zlg) wrote:
>> I honestly don't see the point of this when `git log` or even `git
>> diff` or standard `diff` will tell you if what's in your overlay
>> differs from the source. With some bash magic it could even be
>> automated. The point of that 'feature' is to see what, if anything,
>> has changed between one's overlay and Gentoo's running tree. A diff
>> would not only be able to tell you *if* anything changed, but also
>> *what*, without adding around 5-7 extra bytes per ebuild. Sure, it's
>> only bytes, but when multiplied against the number of ebuilds we have,
>> it can make a few hundred KB difference. When expanded, that number
>> multiplies. Is it worth adding this extra bloat to something that a
>> standard utility can expose better than a hash?
> 
> Agree here. Also I don't like the idea of post-modifying content of
> signed commits: files developers committed to the tree should be
> the same users get. As a side effect this will simplify tree
> consistency checks and forensics.
> 

The files are already modified (e.g. Manifest) for rsync, so this
arguments becomes a moot point.

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1681 bytes --]

On Fri, 14 Aug 2015 13:16:47 +0200 hasufell wrote:
> On 08/14/2015 01:10 PM, Andrew Savchenko wrote:
> > On Fri, 14 Aug 2015 02:11:09 -0700 Daniel Campbell (zlg) wrote:
> >> I honestly don't see the point of this when `git log` or even `git
> >> diff` or standard `diff` will tell you if what's in your overlay
> >> differs from the source. With some bash magic it could even be
> >> automated. The point of that 'feature' is to see what, if anything,
> >> has changed between one's overlay and Gentoo's running tree. A diff
> >> would not only be able to tell you *if* anything changed, but also
> >> *what*, without adding around 5-7 extra bytes per ebuild. Sure, it's
> >> only bytes, but when multiplied against the number of ebuilds we have,
> >> it can make a few hundred KB difference. When expanded, that number
> >> multiplies. Is it worth adding this extra bloat to something that a
> >> standard utility can expose better than a hash?
> > 
> > Agree here. Also I don't like the idea of post-modifying content of
> > signed commits: files developers committed to the tree should be
> > the same users get. As a side effect this will simplify tree
> > consistency checks and forensics.
> > 
> 
> The files are already modified (e.g. Manifest) for rsync, so this
> arguments becomes a moot point.

No.

1. Modified ebuild != modified manifest.

2. The question is why manifests are modified for rsync. In git
manifests are thin (only distfiles are there), in rsync they also
contain checksums for ebuilds and files dir content. Do we really
need this? These manifests are not signed now, so of little use.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Kristian Fiskerstrand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 08/14/2015 01:56 PM, Andrew Savchenko wrote:

..

> 
> 2. The question is why manifests are modified for rsync. In git 
> manifests are thin (only distfiles are there), in rsync they also 
> contain checksums for ebuilds and files dir content. Do we really 
> need this? These manifests are not signed now, so of little use.

They will be OpenPGP signed by a releng key during thickening and
portage will auto-verify it using gkeys once things are in place. As
such checksum for ebuilds and other files certainly needs to be part
of the manifest, otherwise it can open up for malicious alterations of
these files.

- -- 
Kristian Fiskerstrand
Public PGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
-----BEGIN PGP SIGNATURE-----

iQEcBAEBCgAGBQJVzeLTAAoJECULev7WN52F9z8H/1Es0XTZP2eBmVyMSfVf65T7
MVO2v+0r91kjBekwkmKMNbLM/ZubAq1af20xSUW5Q9kBANJ3GIieU/6CpcVS3BCP
bgjSCSOj2cydCgWO3i6eydrB9yEpLVPzi4rezbNVSaLsG3WYEl07z/knXYU5mJJW
ViXNeOBPyCDpJiwgccGDmIbFvIghI9bPFOCrLRvmH5v+Velk0QNdK/PZd9pvd792
FIyoPcE2hq8NYpeH7o/WWwLcsczERg5HhcAnTmTZYZ0DpLhQzEfHrLlkD46JbR0j
JT7rn7PtmtsQNoXTQesmA4hrGLu26fUVljqSbIwJt/33ijis7VSxZVedCp7wGyc=
=c5IU
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 14/08/15 07:16 AM, hasufell wrote:
> On 08/14/2015 01:10 PM, Andrew Savchenko wrote:
>> On Fri, 14 Aug 2015 02:11:09 -0700 Daniel Campbell (zlg)
>> wrote:
>>> I honestly don't see the point of this when `git log` or even
>>> `git diff` or standard `diff` will tell you if what's in your
>>> overlay differs from the source. With some bash magic it
>>> could even be automated. The point of that 'feature' is to
>>> see what, if anything, has changed between one's overlay and
>>> Gentoo's running tree. A diff would not only be able to tell
>>> you *if* anything changed, but also *what*, without adding
>>> around 5-7 extra bytes per ebuild. Sure, it's only bytes, but
>>> when multiplied against the number of ebuilds we have, it can
>>> make a few hundred KB difference. When expanded, that number 
>>> multiplies. Is it worth adding this extra bloat to something
>>> that a standard utility can expose better than a hash?
>> 
>> Agree here. Also I don't like the idea of post-modifying
>> content of signed commits: files developers committed to the
>> tree should be the same users get. As a side effect this will
>> simplify tree consistency checks and forensics.
>> 
> 
> The files are already modified (e.g. Manifest) for rsync, so
> this arguments becomes a moot point.
> 
> 


I think it'd also be handy, in terms of debugging rsync's gone awry
(especially in the case of unofficial rsync mirrors) if the ebuilds
contained the commit-id; it'd provide a more simple way to check
where in git history this particular ebuild was committed.  I'm
thinking especially in cases where for whatever reason an rsync
doesn't synchronize the entire repo.

That said, since $Id$ is apparently not going to be a commit-id that
can be checked quickly by 'git show' or searched for in 'git log',
I'm not sure if that makes it as easy to check as I'm imagining...

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlXN/DUACgkQAJxUfCtlWe0TugEAr+eKeiFcPrNLfbR2vrN06U7E
cldl+tb+rjJZT7NMZ3UBANV4x8b1fj4NQu+DO38bYKCtZ7NZdNWALUXiDxYSf9mO
=c00L
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Matthias Maier]

[-- Attachment #1: Type: text/plain, Size: 433 bytes --]


> They will be OpenPGP signed by a releng key during thickening and
> portage will auto-verify it using gkeys once things are in place. As
> such checksum for ebuilds and other files certainly needs to be part
> of the manifest, otherwise it can open up for malicious alterations of
> these files.

And we switch portage in the near future to enforce signature checking
on rsync'ed repositories? (e.g. controlled via repos.d/*) :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 820 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 14/08/15 07:56 AM, Andrew Savchenko wrote:
> 2. The question is why manifests are modified for rsync. In git 
> manifests are thin (only distfiles are there), in rsync they
> also contain checksums for ebuilds and files dir content. Do we
> really need this? These manifests are not signed now, so of
> little use.

There's still plenty of cases where there can be mis-matches that
the checksums will catch; just because it's not a be-all-and-end-all
security solution doesn't mean it's not valuable for data integrity
in general.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlXN/NsACgkQAJxUfCtlWe3BBQD/blYWTRa7WuF+GdGlQ8grxvlk
Rdx67cc5Bfvt5qTvuVwBAOf6Ef5f7QUX8jI0vLM6Sn7Gy+CPopxFanqIcgLvMjfr
=dea0
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Rich Freeman]

On Fri, Aug 14, 2015 at 8:45 AM, Kristian Fiskerstrand <k_f@gentoo.org> wrote:
>
>>
>> 2. The question is why manifests are modified for rsync. In git
>> manifests are thin (only distfiles are there), in rsync they also
>> contain checksums for ebuilds and files dir content. Do we really
>> need this? These manifests are not signed now, so of little use.
>
> They will be OpenPGP signed by a releng key during thickening and
> portage will auto-verify it using gkeys once things are in place. As
> such checksum for ebuilds and other files certainly needs to be part
> of the manifest, otherwise it can open up for malicious alterations of
> these files.
>

As much as I'd love to see it all folded into git, the reality is also
that git signatures are only bound to files by a series of sha1
hashes, and sha1 is not a strong hash function.  Git really ought to
move to sha256 at some point, preferably in a manner that makes it
expandable in the future to other hash functions.  But, this isn't a
high-priority for upstream.

The same limitation is true of any git gpg signature, including tag
signatures.  It is all held together by sha1.  The manifest system is
much stronger.

-- 
Rich

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Kristian Fiskerstrand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 08/14/2015 04:54 PM, Rich Freeman wrote:
> On Fri, Aug 14, 2015 at 8:45 AM, Kristian Fiskerstrand
> <k_f@gentoo.org> wrote:
>> 
>>> 
>>> 2. The question is why manifests are modified for rsync. In
>>> git manifests are thin (only distfiles are there), in rsync
>>> they also contain checksums for ebuilds and files dir content.
>>> Do we really need this? These manifests are not signed now, so
>>> of little use.
>> 
>> They will be OpenPGP signed by a releng key during thickening
>> and portage will auto-verify it using gkeys once things are in
>> place. As such checksum for ebuilds and other files certainly
>> needs to be part of the manifest, otherwise it can open up for
>> malicious alterations of these files.
>> 
> 
> As much as I'd love to see it all folded into git, the reality is
> also that git signatures are only bound to files by a series of
> sha1 hashes, and sha1 is not a strong hash function.  Git really
> ought to move to sha256 at some point, preferably in a manner that
> makes it expandable in the future to other hash functions.  But,
> this isn't a high-priority for upstream.

I'm not really too worried about second preimage attacks on sha1 at
the present time, so can understand that priority.

> 
> The same limitation is true of any git gpg signature, including
> tag signatures.  It is all held together by sha1.  The manifest
> system is much stronger.
> 

Well, it is only as good as the input it gets, so if the git
infrastructure (if sha1 truly turns out to be an issue, presuming that
it is verified at point of staging) or the staging area for rsync
mirror is compromised (since the Manifests are signed when thickened,
a compromise here can override everything else) it will replicate to
users, so these points needs to be properly protected.

- -- 
Kristian Fiskerstrand
Public PGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
-----BEGIN PGP SIGNATURE-----

iQEcBAEBCgAGBQJVzgP7AAoJECULev7WN52F0QoIAMWD3crryd+J5wt4xYfTTRHl
6t4Jqhg5f4yIbC/9L7ldpqRpg/rNeO1kl7/vqHGTPQIuZXsbw+40LksFHhR9R6U+
lyt9d8pzDE2jjzKieLRYAXLmz0SWKB7HxBcnueaizYOFjSxJS4qcgCoj6u3X0t4B
TTt1VOHP83t4WZGPSbGBhaqlHIFVbVf/NmaXEXvOqO7LmuLuR0CUNj5L0mZxNhIM
W/ey0YzU/mwLpbDf/Xx0MGW8xFe5oVbLxruydYIWr6OVPSWwunn3vnU2fOWpN4Xx
siJzTo2lLgJ7ypGwbvYpAmh3bH3pbOPqCvk7UD75Au+kHQkT7oqwlp2B1PErmQU=
=+CcW
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1601 bytes --]

Hi,

On Fri, 14 Aug 2015 10:54:57 -0400 Rich Freeman wrote:
> On Fri, Aug 14, 2015 at 8:45 AM, Kristian Fiskerstrand <k_f@gentoo.org> wrote:
> > They will be OpenPGP signed by a releng key during thickening and
> > portage will auto-verify it using gkeys once things are in place. As
> > such checksum for ebuilds and other files certainly needs to be part
> > of the manifest, otherwise it can open up for malicious alterations of
> > these files.
> >
> 
> As much as I'd love to see it all folded into git, the reality is also
> that git signatures are only bound to files by a series of sha1
> hashes, and sha1 is not a strong hash function.  Git really ought to
> move to sha256 at some point, preferably in a manner that makes it
> expandable in the future to other hash functions.  But, this isn't a
> high-priority for upstream.
> 
> The same limitation is true of any git gpg signature, including tag
> signatures.  It is all held together by sha1.  The manifest system is
> much stronger.
 
OK, if manifests are that important, why not generate full manifest
during repoman commit? If we do not tamper with $Id$, the only file
outside of this manifest will be ChangeLog generated during rsync
propagation. Then we have following options:
- do not sing ChangeLog: even if it will be tampered, little harm
can be done, since it doesn't affect live system or build process;
- sign ChangeLog with releng key;
- sign developer-signed manifest + ChangeLog with releng key. Thus
we'll have double signature for most important files.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1927 bytes --]

Dnia 2015-08-15, o godz. 10:50:02
Andrew Savchenko <bircoph@gentoo.org> napisał(a):

> Hi,
> 
> On Fri, 14 Aug 2015 10:54:57 -0400 Rich Freeman wrote:
> > On Fri, Aug 14, 2015 at 8:45 AM, Kristian Fiskerstrand <k_f@gentoo.org> wrote:
> > > They will be OpenPGP signed by a releng key during thickening and
> > > portage will auto-verify it using gkeys once things are in place. As
> > > such checksum for ebuilds and other files certainly needs to be part
> > > of the manifest, otherwise it can open up for malicious alterations of
> > > these files.
> > >
> > 
> > As much as I'd love to see it all folded into git, the reality is also
> > that git signatures are only bound to files by a series of sha1
> > hashes, and sha1 is not a strong hash function.  Git really ought to
> > move to sha256 at some point, preferably in a manner that makes it
> > expandable in the future to other hash functions.  But, this isn't a
> > high-priority for upstream.
> > 
> > The same limitation is true of any git gpg signature, including tag
> > signatures.  It is all held together by sha1.  The manifest system is
> > much stronger.
>  
> OK, if manifests are that important, why not generate full manifest
> during repoman commit? If we do not tamper with $Id$, the only file
> outside of this manifest will be ChangeLog generated during rsync
> propagation. Then we have following options:
> - do not sing ChangeLog: even if it will be tampered, little harm
> can be done, since it doesn't affect live system or build process;
> - sign ChangeLog with releng key;
> - sign developer-signed manifest + ChangeLog with releng key. Thus
> we'll have double signature for most important files.

How about we switch back to CVS if we're going to kill git anyway? It'd
at least save our time wasted by these pointless discussions.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 2252 bytes --]

On Sat, 15 Aug 2015 09:53:37 +0200 Michał Górny wrote:
> Dnia 2015-08-15, o godz. 10:50:02
> Andrew Savchenko <bircoph@gentoo.org> napisał(a):
> 
> > Hi,
> > 
> > On Fri, 14 Aug 2015 10:54:57 -0400 Rich Freeman wrote:
> > > On Fri, Aug 14, 2015 at 8:45 AM, Kristian Fiskerstrand <k_f@gentoo.org> wrote:
> > > > They will be OpenPGP signed by a releng key during thickening and
> > > > portage will auto-verify it using gkeys once things are in place. As
> > > > such checksum for ebuilds and other files certainly needs to be part
> > > > of the manifest, otherwise it can open up for malicious alterations of
> > > > these files.
> > > >
> > > 
> > > As much as I'd love to see it all folded into git, the reality is also
> > > that git signatures are only bound to files by a series of sha1
> > > hashes, and sha1 is not a strong hash function.  Git really ought to
> > > move to sha256 at some point, preferably in a manner that makes it
> > > expandable in the future to other hash functions.  But, this isn't a
> > > high-priority for upstream.
> > > 
> > > The same limitation is true of any git gpg signature, including tag
> > > signatures.  It is all held together by sha1.  The manifest system is
> > > much stronger.
> >  
> > OK, if manifests are that important, why not generate full manifest
> > during repoman commit? If we do not tamper with $Id$, the only file
> > outside of this manifest will be ChangeLog generated during rsync
> > propagation. Then we have following options:
> > - do not sing ChangeLog: even if it will be tampered, little harm
> > can be done, since it doesn't affect live system or build process;
> > - sign ChangeLog with releng key;
> > - sign developer-signed manifest + ChangeLog with releng key. Thus
> > we'll have double signature for most important files.
> 
> How about we switch back to CVS if we're going to kill git anyway? It'd
> at least save our time wasted by these pointless discussions.

I don't understand your point. Please explain.

I see nobody here talking about killing git. I see people concerned
that git is not cryptographically secure enough, thus looking for
gpg-signed manifests or other solutions.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2543 bytes --]

Dnia 2015-08-15, o godz. 11:51:01
Andrew Savchenko <bircoph@gentoo.org> napisał(a):

> On Sat, 15 Aug 2015 09:53:37 +0200 Michał Górny wrote:
> > Dnia 2015-08-15, o godz. 10:50:02
> > Andrew Savchenko <bircoph@gentoo.org> napisał(a):
> > 
> > > Hi,
> > > 
> > > On Fri, 14 Aug 2015 10:54:57 -0400 Rich Freeman wrote:
> > > > On Fri, Aug 14, 2015 at 8:45 AM, Kristian Fiskerstrand <k_f@gentoo.org> wrote:
> > > > > They will be OpenPGP signed by a releng key during thickening and
> > > > > portage will auto-verify it using gkeys once things are in place. As
> > > > > such checksum for ebuilds and other files certainly needs to be part
> > > > > of the manifest, otherwise it can open up for malicious alterations of
> > > > > these files.
> > > > >
> > > > 
> > > > As much as I'd love to see it all folded into git, the reality is also
> > > > that git signatures are only bound to files by a series of sha1
> > > > hashes, and sha1 is not a strong hash function.  Git really ought to
> > > > move to sha256 at some point, preferably in a manner that makes it
> > > > expandable in the future to other hash functions.  But, this isn't a
> > > > high-priority for upstream.
> > > > 
> > > > The same limitation is true of any git gpg signature, including tag
> > > > signatures.  It is all held together by sha1.  The manifest system is
> > > > much stronger.
> > >  
> > > OK, if manifests are that important, why not generate full manifest
> > > during repoman commit? If we do not tamper with $Id$, the only file
> > > outside of this manifest will be ChangeLog generated during rsync
> > > propagation. Then we have following options:
> > > - do not sing ChangeLog: even if it will be tampered, little harm
> > > can be done, since it doesn't affect live system or build process;
> > > - sign ChangeLog with releng key;
> > > - sign developer-signed manifest + ChangeLog with releng key. Thus
> > > we'll have double signature for most important files.
> > 
> > How about we switch back to CVS if we're going to kill git anyway? It'd
> > at least save our time wasted by these pointless discussions.
> 
> I don't understand your point. Please explain.
> 
> I see nobody here talking about killing git. I see people concerned
> that git is not cryptographically secure enough, thus looking for
> gpg-signed manifests or other solutions.

I see you talking about introducing whole new bucket of merge
conflicts.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1463 bytes --]

On Sat, 15 Aug 2015 11:02:19 +0200 Michał Górny wrote:
> > > > OK, if manifests are that important, why not generate full manifest
> > > > during repoman commit? If we do not tamper with $Id$, the only file
> > > > outside of this manifest will be ChangeLog generated during rsync
> > > > propagation. Then we have following options:
> > > > - do not sing ChangeLog: even if it will be tampered, little harm
> > > > can be done, since it doesn't affect live system or build process;
> > > > - sign ChangeLog with releng key;
> > > > - sign developer-signed manifest + ChangeLog with releng key. Thus
> > > > we'll have double signature for most important files.
> > > 
> > > How about we switch back to CVS if we're going to kill git anyway? It'd
> > > at least save our time wasted by these pointless discussions.
> > 
> > I don't understand your point. Please explain.
> > 
> > I see nobody here talking about killing git. I see people concerned
> > that git is not cryptographically secure enough, thus looking for
> > gpg-signed manifests or other solutions.
> 
> I see you talking about introducing whole new bucket of merge
> conflicts.
 
Where? The only case where such conflict may occur is when several
developers are working on the same package at the same time. This
is quite rare occasion. And even with current thin-manifest
workflow there may be conflict if they touch the same files.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Kent Fredric]

On 15 August 2015 at 21:56, Andrew Savchenko <bircoph@gentoo.org> wrote:
> And even with current thin-manifest
> workflow there may be conflict if they touch the same files.


They'll be single-line conflicts though, which will mean assuming
different developers touch different files, git will be able to
trivially merge them, because the lines impacted on either sides are
unique.

Signing both sides of the conflict means you have ~20+ extra lines of
conflict that git has to make you choose between, .... where *neither*
side is correct, and you have to dispose of the conflicted data
manually, and regenerate it from scratch in the middle of the merge.


-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[hasufell]

On 08/15/2015 11:56 AM, Andrew Savchenko wrote:
> On Sat, 15 Aug 2015 11:02:19 +0200 Michał Górny wrote:
>>>>> OK, if manifests are that important, why not generate full manifest
>>>>> during repoman commit? If we do not tamper with $Id$, the only file
>>>>> outside of this manifest will be ChangeLog generated during rsync
>>>>> propagation. Then we have following options:
>>>>> - do not sing ChangeLog: even if it will be tampered, little harm
>>>>> can be done, since it doesn't affect live system or build process;
>>>>> - sign ChangeLog with releng key;
>>>>> - sign developer-signed manifest + ChangeLog with releng key. Thus
>>>>> we'll have double signature for most important files.
>>>>
>>>> How about we switch back to CVS if we're going to kill git anyway? It'd
>>>> at least save our time wasted by these pointless discussions.
>>>
>>> I don't understand your point. Please explain.
>>>
>>> I see nobody here talking about killing git. I see people concerned
>>> that git is not cryptographically secure enough, thus looking for
>>> gpg-signed manifests or other solutions.
>>
>> I see you talking about introducing whole new bucket of merge
>> conflicts.
>  
> Where? The only case where such conflict may occur is when several
> developers are working on the same package at the same time. This
> is quite rare occasion. And even with current thin-manifest
> workflow there may be conflict if they touch the same files.
> 

-1

No one has proven that git is cryptographically insecure. Everyone
claiming that probably refers to
https://www.schneier.com/blog/archives/2012/10/when_will_we_se.html and
the fact that we don't sign blob objects.

While that is something git upstream has to fix, all known SHA1
"attacks" are NOT "preimage attacks". So the whole point is utterly and
mathematically moot for us in practice. This is wasting our time.

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Peter Stuge]

Hi and happy Git days! :)


Robin H. Johnson wrote:
> It expands to the hash of the blob of that file; and from that, you can
> identify which commits the blob exists in.

$ git ls-tree HEAD README
100644 blob 08ae16956b8944da2fef75fee892dcba457cf4f0    README
$ 

$ (stat --printf='blob %s\0' README; cat README) | sha1sum
08ae16956b8944da2fef75fee892dcba457cf4f0  -
$ 

This is so simple to generate that it doesn't really need a
placeholder in every ebuild in the repository.


> The primary use case of it is to allow users to easily see what
> version of a given ebuild they are using.

I guess the merits of this use case are up for discussion also
outside of infra?

I think a single tree-wide identifier (let's it a snapshot identifier?)
would be far more useful.


Kind regards

//Peter

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Jason Zaman]

[-- Attachment #1: Type: text/plain, Size: 952 bytes --]

On 15 Aug 2015 20:45, "Peter Stuge" <peter@stuge.se> wrote:
>
> Hi and happy Git days! :)
>
>
> Robin H. Johnson wrote:
> > It expands to the hash of the blob of that file; and from that, you can
> > identify which commits the blob exists in.
>
> $ git ls-tree HEAD README
> 100644 blob 08ae16956b8944da2fef75fee892dcba457cf4f0    README
> $
>
> $ (stat --printf='blob %s\0' README; cat README) | sha1sum
> 08ae16956b8944da2fef75fee892dcba457cf4f0  -
> $
>
> This is so simple to generate that it doesn't really need a
> placeholder in every ebuild in the repository.
>
>
> > The primary use case of it is to allow users to easily see what
> > version of a given ebuild they are using.
>
> I guess the merits of this use case are up for discussion also
> outside of infra?
>
> I think a single tree-wide identifier (let's it a snapshot identifier?)
> would be far more useful.

I already filed a bug for this:

https://bugs.gentoo.org/557366

-- Jason

[-- Attachment #2: Type: text/html, Size: 1379 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Rich Freeman]

On Sat, Aug 15, 2015 at 7:24 AM, hasufell <hasufell@gentoo.org> wrote:
>
> No one has proven that git is cryptographically insecure. Everyone
> claiming that probably refers to
> https://www.schneier.com/blog/archives/2012/10/when_will_we_se.html and
> the fact that we don't sign blob objects.
>
> While that is something git upstream has to fix, all known SHA1
> "attacks" are NOT "preimage attacks". So the whole point is utterly and
> mathematically moot for us in practice. This is wasting our time.
>

I'd take a few issues with your wording, but not with your argument as
it applies to Gentoo.

Whether a collision attack or a preimage attack is necessary to make
something insecure depends on how the hash is being used.  In the case
of Gentoo the author is the signer, so a collision attack doesn't
really mean much.  If the signer wanted to sign something different,
they would just do it.  In another workflow where the where the signer
is somebody other than the author, then a collision attack might be
meaningful, even in the absence of a preimage attack.  That is, as the
author I can use a collision attack to generate two git trees with the
same hash.  One is a tree you would be willing to sign off on, and the
other is one that you would be unwilling to sign off on.  After
obtaining your signature I could keep your commit object and swap out
the tree resulting in a repo that has your signature but what you
signed off on is not what is in the repo.  You could see how in some
code-review scenarios that might be relevant (only if the submitted
tree is signed unmodified - any rebasing could break this, and even a
merge could break it[1]).  On the other hand, in the author-is-signer
workflow of Gentoo it isn't very relevant, since if I want to stick
something nasty in the tree I can just do it.

Overall, though, sha1's days are numbered.  I don't think that is a
reason to make an inconvenient change in our workflow just now.
However, it really is the sort of thing the git maintainers should be
thinking about and at least planning for.

This also brings up the issue of unstated assumptions made by
development teams.  They use a tool which has some cryptographic
weakness, based on the argument that the weakness is not a problem for
their intended use.  Then when somebody else comes along and uses the
application in a manner not originally intended or conceived of by the
authors, the weakness compromises the security of the application,
perhaps in a way the users aren't aware of.  Then when the problem is
exploited and publicized everybody gets to argue about whose fault it
is.

Footnotes:
1 - ...though if you could be reasonably sure that other commits being
rebased/merged wouldn't touch the part of the tree you're messing with
you could do a collision on a tree object below the root and not on
the root itself.  As long as no other commits before the merge mess
with the part of the tree you're playing games with you could swap out
just that part of the tree if you could do a collision attack and then
both the merge and your own commit will inherit the changes.

-- 
Rich

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Rich Freeman]

On Sat, Aug 15, 2015 at 8:44 AM, Peter Stuge <peter@stuge.se> wrote:
>
> $ git ls-tree HEAD README
> 100644 blob 08ae16956b8944da2fef75fee892dcba457cf4f0    README
> $
>
> $ (stat --printf='blob %s\0' README; cat README) | sha1sum
> 08ae16956b8944da2fef75fee892dcba457cf4f0  -
> $
>
> This is so simple to generate that it doesn't really need a
> placeholder in every ebuild in the repository.
>

Additionally, sha1sum tells you what was actually used, not what the
hash was the last time it was seen by git.  If a file is modified
after the fact sha1sum will catch that.

Maybe you care where it originally came from, but I'm not sure that it
really matters - if the user isn't using the actual ebuild out of the
tree they should be attaching it to the bug anyway.  And if you care
that much where the ebuild came from you should be storing it in git
in the first place.

-- 
Rich

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1004 bytes --]

On Sat, Aug 15, 2015 at 02:44:47PM +0200, Peter Stuge wrote:
> Hi and happy Git days! :)
> 
> 
> Robin H. Johnson wrote:
> > It expands to the hash of the blob of that file; and from that, you can
> > identify which commits the blob exists in.
> 
> $ git ls-tree HEAD README
> 100644 blob 08ae16956b8944da2fef75fee892dcba457cf4f0    README
> $ 
> 
> $ (stat --printf='blob %s\0' README; cat README) | sha1sum
> 08ae16956b8944da2fef75fee892dcba457cf4f0  -
> $ 
> 
> This is so simple to generate that it doesn't really need a
> placeholder in every ebuild in the repository.

If this is this simple to regenerate, consider my response here a very
strong +1; let's kill $id$ out of the git repository and generate this
some how at the rsync level if users need it or give them a tool that
will generate it.

I don't object to someone having a way to get this information if it
is useful to them, but I don't think we need a placeholder for it in the
git repository.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Robin H. Johnson]

On Sun, Aug 16, 2015 at 12:33:05PM -0500, William Hubbs wrote:
> On Sat, Aug 15, 2015 at 02:44:47PM +0200, Peter Stuge wrote:
> > Hi and happy Git days! :)
> > 
> > 
> > Robin H. Johnson wrote:
> > > It expands to the hash of the blob of that file; and from that, you can
> > > identify which commits the blob exists in.
> > 
> > $ git ls-tree HEAD README
> > 100644 blob 08ae16956b8944da2fef75fee892dcba457cf4f0    README
> > $ 
> > 
> > $ (stat --printf='blob %s\0' README; cat README) | sha1sum
> > 08ae16956b8944da2fef75fee892dcba457cf4f0  -
> > $ 
> > 
> > This is so simple to generate that it doesn't really need a
> > placeholder in every ebuild in the repository.
It's ONLY applicable if the file has never been modified. If the file
has been modified, then unless you know exactly what the modifications
were (or have a backup before modifications), you cannot figure out the
exact source.

> If this is this simple to regenerate, consider my response here a very
> strong +1; let's kill $id$ out of the git repository and generate this
> some how at the rsync level if users need it or give them a tool that
> will generate it.
Ebuilds are NOT the only place where it's relevant, but they have a
similar use case to the other places.

Ebuild:
- User copies .ebuild with expanded $Id$ to /usr/local/portage/...
- User modifies local ebuild copy
- User submits changed ebuild to bugzilla
- Developer uses Header/Id to figure out what to base a diff on for
  merging.

Non-ebuild
- Package installs /etc/init.d/foobar, with some bug
- User reports it is buggy
- User modifies it to temporarily work around the bug
- User submits modified version back
- Developer asks what the original version was if it didn't have some
  useful marker in the file.

Example of old marker:
/etc/init.d/nfs:# $Header: /var/cvsroot/gentoo-x86/net-fs/nfs-utils/files/nfs.initd,v 1.26 2011/09/18 01:51:51 vapier Exp $

> I don't object to someone having a way to get this information if it
> is useful to them, but I don't think we need a placeholder for it in the
> git repository.
If we don't have a placeholder for where to inject it, how do we know
where it's safe to inject?

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-dev] Infra plans regarding $Id$ - official answer...

[Rich Freeman]

On Sun, Aug 16, 2015 at 2:12 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
>
> Ebuild:
> - User copies .ebuild with expanded $Id$ to /usr/local/portage/...
> - User modifies local ebuild copy
> - User submits changed ebuild to bugzilla
> - Developer uses Header/Id to figure out what to base a diff on for
>   merging.

I guess I could see it being useful for quickly spotting what the user
changed and applying it to the latest release, rather than carefully
checking for backdoors or trying to find the right changes in a
mismatched diff.

On the other hand, we could try to encourage users to use git
format-patch, which accomplishes the same thing in a nicer way (if
they're using git).

I do see the potential use of it.  I am not really sure how much use
it will get though.

>
> Non-ebuild
> - Package installs /etc/init.d/foobar, with some bug
> - User reports it is buggy
> - User modifies it to temporarily work around the bug
> - User submits modified version back
> - Developer asks what the original version was if it didn't have some
>   useful marker in the file.

I thought we were only expanding this on ebuilds.  We need to be
really cafeful about this because we've already been bitten numerous
times with cvs keyword expansion issues, because...

> If we don't have a placeholder for where to inject it, how do we know
> where it's safe to inject?

... if you're not limiting this to */*/*.ebuild then how do you know
where it is safe to expand?

-- 
Rich