[gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1733 bytes --]

Hello,

I would like to propose a new attempt at Manifest signatures. Instead
of using a single per-Manifest signature, we would keep separate
signatures for each of the files, as an additional (optional) hash
type.


Motivation
----------
The current signing approach gives all the responsibility for Manifest
signature to the developer who committed last update to the ebuild
directory regardless of the actual commit significance.

Consider the following: Dev A is the primary package maintainer. He/she
reviewed all the ebuilds and committed a signed Manifest. Then Dev B
performs a slight cleanup of the ebuild directory. He/she modifies
metadata.xml a little and/or removes an old ebuild.

The actual ebuilds weren't modified -- yet Dev B has to sign all
of them once again. Moreover, if Dev B doesn't use Manifest signing,
the signature from Dev A is lost.


The solution
------------
As a solution for this I suggest making the GPG signatures per-file,
simply creating an additional hash type for them. For example,
a single Manifest line would look like:

EBUILD foo-1.ebuild 1000 RMD160 ... SHA1 ... SHA256 ... GPG ...

Where the GPG signature will be an explicit signature done by the dev
modifying (or reviewing) a particular file. Then, if another dev
modifies a single file, the signatures for other files would be
untouched.


Potential issues
----------------
This signing model does not provide a mechanism for signing file
removals. In other words, if a dev does remove files only, he/she won't
leave any signature changes at all. If there's a reason to do that, we
can consider using a complete Manifest file signature in parallel.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[Arun Raghavan]

On 3 October 2010 13:28, Michał Górny <mgorny@gentoo.org> wrote:
> Hello,
>
> I would like to propose a new attempt at Manifest signatures. Instead
> of using a single per-Manifest signature, we would keep separate
> signatures for each of the files, as an additional (optional) hash
> type.
>
>
> Motivation
> ----------
> The current signing approach gives all the responsibility for Manifest
> signature to the developer who committed last update to the ebuild
> directory regardless of the actual commit significance.
>
> Consider the following: Dev A is the primary package maintainer. He/she
> reviewed all the ebuilds and committed a signed Manifest. Then Dev B
> performs a slight cleanup of the ebuild directory. He/she modifies
> metadata.xml a little and/or removes an old ebuild.
>
> The actual ebuilds weren't modified -- yet Dev B has to sign all
> of them once again. Moreover, if Dev B doesn't use Manifest signing,
> the signature from Dev A is lost.

If we make the GPG signatures mandatory at some point of time, that
addresses the second of your concerns. I do not understand why the
first a problem - could you clarify?

Cheers,
-- 
Arun Raghavan
http://arunraghavan.net/
(Ford_Prefect | Gentoo) & (arunsr | GNOME)

Re: [gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 3508 bytes --]

On Sun, Oct 03, 2010 at 09:58:48AM +0200, Micha?? G??rny wrote:
> The current signing approach gives all the responsibility for Manifest
> signature to the developer who committed last update to the ebuild
> directory regardless of the actual commit significance.
> 
> Consider the following: Dev A is the primary package maintainer. He/she
> reviewed all the ebuilds and committed a signed Manifest. Then Dev B
> performs a slight cleanup of the ebuild directory. He/she modifies
> metadata.xml a little and/or removes an old ebuild.
> 
> The actual ebuilds weren't modified -- yet Dev B has to sign all
> of them once again. Moreover, if Dev B doesn't use Manifest signing,
> the signature from Dev A is lost.
This exact problem was raised in 2003, as some of the early tree-signing
discussion. If you haven't read my endnote to GLEP57, I strongly
encourage you to do so.

The end question from many of those discussions, on the matter of
internal signature in the Manifest was:
Why are VCS annotations on the unchanged files not enough?
$ cvs annotate Manifest
That can clearly show you the lines that have not changed, regardless of
how many times the Manifest is signed and resigned.

The best (and simultaneously worse) solution that turned up in the prior
versions was the proposed appending of new data to Manifests, not
replacing. The new data had to be signed however, otherwise it opened up
a new way to exploit the security. And if you're signing the new data,
there is no problem to just resign all of it.

> The solution
> ------------
I would like to thank you for a new insight into the problem.
Your solution is sufficiently novel that none of the prior proposals are
anything like it.

> As a solution for this I suggest making the GPG signatures per-file,
> simply creating an additional hash type for them. For example,
> a single Manifest line would look like:
> 
> EBUILD foo-1.ebuild 1000 RMD160 ... SHA1 ... SHA256 ... GPG ...
I'm assuming you are proposing taking the ascii-armoured detached
signature here, stripping the newlines (except the last one, which needs
to be kept as a separate field), and using that.

Alternatively base64 encode the non-armoured binary detached signature
ourselves. Very little difference in the two ultimately however.

> Where the GPG signature will be an explicit signature done by the dev
> modifying (or reviewing) a particular file. Then, if another dev
> modifies a single file, the signatures for other files would be
> untouched.

> Potential issues
> ----------------
> This signing model does not provide a mechanism for signing file
> removals. In other words, if a dev does remove files only, he/she won't
> leave any signature changes at all. If there's a reason to do that, we
> can consider using a complete Manifest file signature in parallel.
Some more issues for you:
1. Increases the size of the Manifest by a minimum of 710 bytes _per_
   file. (4 bytes for 'GPG ', 700-900 for the hash, 1 for the field space, 5-12 bytes for the
   trailer).
1.1. 55907 Manifest2 entries need this signing, so that's a ~38MiB
     increase in the tree size.
2. Impossible to validate without Portage itself, or at least another
   tool to convert the signature back into a form readable by GnuPG.


-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 330 bytes --]

Re: [gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[James Cloos]

>>>>> "RHJ" == Robin H Johnson <robbat2@gentoo.org> writes:

RHJ> Some more issues for you:
RHJ> 1. Increases the size of the Manifest by a minimum of 710 bytes _per_
RHJ>    file. (4 bytes for 'GPG ', 700-900 for the hash, 1 for the field space, 5-12 bytes for the
RHJ>    trailer).
RHJ> 1.1. 55907 Manifest2 entries need this signing, so that's a ~38MiB
RHJ>      increase in the tree size.
RHJ> 2. Impossible to validate without Portage itself, or at least another
RHJ>    tool to convert the signature back into a form readable by GnuPG.

From the standpoint of someone using Gentoo to Get Work Done:

RMD160 and SHA1 just waste space.  SHA2 is sufficient non-encrypted
hashing.

Put distfile sigs in $DISTDIR or $FILESDIR.  They are just too large
for a line-per-entry file.

Include the signing keyid in the filename to support both allowing
multiple devs to sign a file and an easy indication of who signed it.

Have portage note in the ebuild log what was signed, by what key, and
whether the sigs were true.

Make failing on a bad sig optional (per overlay?) and make sure that
even when portage /is/ configured to fail on a bad sig that it only
fails that one package and anything in the current set which depends
on that version of the failed package.  Don't stop everything just
because /one/ package has a problem.

And think about a way to sign Changelog entries.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 2077 bytes --]

On Tue, Oct 05, 2010 at 05:53:50PM -0400, James Cloos wrote:
> >>>>> "RHJ" == Robin H Johnson <robbat2@gentoo.org> writes:
> 
> RHJ> Some more issues for you:
> RHJ> 1. Increases the size of the Manifest by a minimum of 710 bytes _per_
> RHJ>    file. (4 bytes for 'GPG ', 700-900 for the hash, 1 for the field space, 5-12 bytes for the
> RHJ>    trailer).
> RHJ> 1.1. 55907 Manifest2 entries need this signing, so that's a ~38MiB
> RHJ>      increase in the tree size.
> RHJ> 2. Impossible to validate without Portage itself, or at least another
> RHJ>    tool to convert the signature back into a form readable by GnuPG.
> 
> >From the standpoint of someone using Gentoo to Get Work Done:
> RMD160 and SHA1 just waste space.  SHA2 is sufficient non-encrypted
> hashing.
Please read the tree-signing GLEPs. This is already coming up, the GLEPs
included the migration window for it.

> Put distfile sigs in $DISTDIR or $FILESDIR.  They are just too large
> for a line-per-entry file.
$DISTDIR is an interesting idea. $FILESDIR costs too many inodes.

> Include the signing keyid in the filename to support both allowing
> multiple devs to sign a file and an easy indication of who signed it.
You can extract keyid from any signature trivially.

> Have portage note in the ebuild log what was signed, by what key, and
> whether the sigs were true.
zmedico: can we include this in the repoman commit sig?

> Make failing on a bad sig optional (per overlay?) and make sure that
> even when portage /is/ configured to fail on a bad sig that it only
> fails that one package and anything in the current set which depends
> on that version of the failed package.  Don't stop everything just
> because /one/ package has a problem.
This is already controllable.

> And think about a way to sign Changelog entries.
We wanted commit-signing with the git migration...

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 330 bytes --]

Re: [gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[Zac Medico]

On 10/05/2010 05:26 PM, Robin H. Johnson wrote:
> On Tue, Oct 05, 2010 at 05:53:50PM -0400, James Cloos wrote:
>> Have portage note in the ebuild log what was signed, by what key, and
>> whether the sigs were true.
> zmedico: can we include this in the repoman commit sig?

Sure. Currently, repoman only signs the Manifest files in the ebuild
directories. For single package commits, that's just one file. However,
it's possible to do category-level or even full-repo level commits,
though they're relatively rare. For these cases we'd be listing all the
Manifest files that changed.

Are we counting the files listed in the signed Manifest as signed too?
In that case, you want it to list any files that changed during that
commit? Forgive me if this is a stupid question, because I haven't been
following the whole discussion.
-- 
Thanks,
Zac

Re: [gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1172 bytes --]

On Tue, Oct 05, 2010 at 05:49:31PM -0700, Zac Medico wrote:
> On 10/05/2010 05:26 PM, Robin H. Johnson wrote:
> > On Tue, Oct 05, 2010 at 05:53:50PM -0400, James Cloos wrote:
> >> Have portage note in the ebuild log what was signed, by what key, and
> >> whether the sigs were true.
> > zmedico: can we include this in the repoman commit sig?
> 
> Sure. Currently, repoman only signs the Manifest files in the ebuild
> directories. For single package commits, that's just one file. However,
> it's possible to do category-level or even full-repo level commits,
> though they're relatively rare. For these cases we'd be listing all the
> Manifest files that changed.
Sorry, I should have clarified. I was in favour of including the signing
key in the repoman commit message. The list of changed files is an
intrinsic property of the commit, so we don't need to duplicate it in
the commit message.

'(Signed Manifest commit)' - alter that to include the signing key env var.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 330 bytes --]

Re: [gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[Zac Medico]

On 10/06/2010 12:47 PM, Robin H. Johnson wrote:
> '(Signed Manifest commit)' - alter that to include the signing key env var.

Ok, it's in git now:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=c7d24916a47f08755932fdad1344f08808ad8022

-- 
Thanks,
Zac

Re: [gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 517 bytes --]

On Wed, Oct 06, 2010 at 01:31:21PM -0700, Zac Medico wrote:
> On 10/06/2010 12:47 PM, Robin H. Johnson wrote:
> > '(Signed Manifest commit)' - alter that to include the signing key env var.
> Ok, it's in git now:
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=c7d24916a47f08755932fdad1344f08808ad8022
Thx.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 330 bytes --]

Re: [gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[James Cloos]

>>>>> "RHJ" == Robin H Johnson <robbat2@gentoo.org> writes:

>> Include the signing keyid in the filename to support both allowing
>> multiple devs to sign a file and an easy indication of who signed it.

RHJ> You can extract keyid from any signature trivially.

But if it is not in the filename you cannot have multiple sig files.

>> Don't stop everything just because /one/ package has a problem.

RHJ> This is already controllable.

If you mean --keep-going, that may work sometimes, but never did when I
really needed it.

>> And think about a way to sign Changelog entries.

RHJ> We wanted commit-signing with the git migration...

Good choice.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [gentoo-dev] [enhancement proposal] Per-file Manifest GPG signatures

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1026 bytes --]

On Thu, Oct 07, 2010 at 10:17:01AM -0400, James Cloos wrote:
> >>>>> "RHJ" == Robin H Johnson <robbat2@gentoo.org> writes:
> >> Include the signing keyid in the filename to support both allowing
> >> multiple devs to sign a file and an easy indication of who signed it.
> RHJ> You can extract keyid from any signature trivially.
> But if it is not in the filename you cannot have multiple sig files.
This does still bloat the inode count. The variant was to have multiple
signed blocks inside the Manifest file.

> >> Don't stop everything just because /one/ package has a problem.
> RHJ> This is already controllable.
> If you mean --keep-going, that may work sometimes, but never did when I
> really needed it.
"FEATURES=-severe" iirc, but I do agree that more control over signature
validation in FEATURES would be beneficial.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 330 bytes --]