[gentoo-dev] .gitignore

[gentoo-dev] .gitignore

[Justin (jlec)]

[-- Attachment #1: Type: text/plain, Size: 136 bytes --]

Hi,

how do we maintain this file?

I like to propose to add the md5-cache into it. Which other files are of interest?

Justin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-dev] .gitignore

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

On 10 Aug 2015 08:28, Justin (jlec) wrote:
> how do we maintain this file?

like any other file.  git add && git commit.

> I like to propose to add the md5-cache into it. Which other files are of interest?

/distfiles/
/local/
/packages/

/metadata/md5-cache/
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] .gitignore

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 512 bytes --]

Dnia 2015-08-10, o godz. 02:42:21
Mike Frysinger <vapier@gentoo.org> napisał(a):

> On 10 Aug 2015 08:28, Justin (jlec) wrote:
> > how do we maintain this file?
> 
> like any other file.  git add && git commit.
> 
> > I like to propose to add the md5-cache into it. Which other files are of interest?
> 
> /distfiles/
> /local/
> /packages/

Those directories should not be ignored. Those should not exist for
a long time.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] .gitignore

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 591 bytes --]

On 10 Aug 2015 09:17, Michał Górny wrote:
> Dnia 2015-08-10, o godz. 02:42:21 Mike Frysinger napisał(a):
> > On 10 Aug 2015 08:28, Justin (jlec) wrote:
> > > I like to propose to add the md5-cache into it. Which other files are of interest?
> > 
> > /distfiles/
> > /local/
> > /packages/
> 
> Those directories should not be ignored. Those should not exist for
> a long time.

there's no reason people can't use these on their own system.  there's no
reason they should be added to the git repo which means, if a user opted
to utilize them, they should be ignored.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] .gitignore

[Justin (jlec)]

[-- Attachment #1: Type: text/plain, Size: 375 bytes --]

On 10/08/15 08:42, Mike Frysinger wrote:
> On 10 Aug 2015 08:28, Justin (jlec) wrote:
>> how do we maintain this file?
> 
> like any other file.  git add && git commit.
> 

I rather meant, if this file should only be modified after a discussion in a bug
or on a ml. Or if only QA is modifying this file. Or if anybody can play around
as she/he likes.

Justin



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 951 bytes --]

Re: [gentoo-dev] .gitignore

[Anthony G. Basile]

On 8/10/15 3:35 AM, Justin (jlec) wrote:
> On 10/08/15 08:42, Mike Frysinger wrote:
>> On 10 Aug 2015 08:28, Justin (jlec) wrote:
>>> how do we maintain this file?
>> like any other file.  git add && git commit.
>>
> I rather meant, if this file should only be modified after a discussion in a bug
> or on a ml. Or if only QA is modifying this file. Or if anybody can play around
> as she/he likes.
>
> Justin
>
>
That's how I interpreted your original question.  I think we should 
discuss modifying .gitignore on this list, along with any other far 
reaching changes which individual committers can make to git. (Can't 
think of another example right now, but there probably is.)

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] .gitignore

[Mike Gilbert]

On Mon, Aug 10, 2015 at 2:42 AM, Mike Frysinger <vapier@gentoo.org> wrote:
> On 10 Aug 2015 08:28, Justin (jlec) wrote:
>> how do we maintain this file?
>
> like any other file.  git add && git commit.
>
>> I like to propose to add the md5-cache into it. Which other files are of interest?
>
> /distfiles/
> /local/
> /packages/
>
> /metadata/md5-cache/
> -mike

Expanding on this: the rsync master creates the following
files/directories under metatdata. On my own system, I like to symlink
them to locations outside my repo so that related portage features
continue to work.

I would like to have these added in .gitignore.

metadata/dtd/ # used by something?
metadata/glsa/ # used by the GLSA utilities?
matadata/herds.xml # used by equery from gentoolkit
metadata/news/ # used by eselect news

Re: [gentoo-dev] .gitignore

[Rich Freeman]

On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org> wrote:
>
> Expanding on this: the rsync master creates the following
> files/directories under metatdata. On my own system, I like to symlink
> them to locations outside my repo so that related portage features
> continue to work.
>
> I would like to have these added in .gitignore.
>
> metadata/dtd/ # used by something?
> metadata/glsa/ # used by the GLSA utilities?
> matadata/herds.xml # used by equery from gentoolkit
> metadata/news/ # used by eselect news
>

As a side note, it probably wouldn't hurt to set up a guide for
running git on /usr/portage, including setting up these symlinks,
running egencache after emerge --sync, etc.  I imagine that this is a
configuration that many developers will tend to use, and with the
advent of git we may see more users who tend to contribute doing the
same.

-- 
Rich

Re: [gentoo-dev] .gitignore

[Daniel Campbell (zlg)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 08/10/2015 08:09 AM, Rich Freeman wrote:
> On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org>
> wrote:
>> 
>> Expanding on this: the rsync master creates the following 
>> files/directories under metatdata. On my own system, I like to
>> symlink them to locations outside my repo so that related portage
>> features continue to work.
>> 
>> I would like to have these added in .gitignore.
>> 
>> metadata/dtd/ # used by something? metadata/glsa/ # used by the
>> GLSA utilities? matadata/herds.xml # used by equery from
>> gentoolkit metadata/news/ # used by eselect news
>> 
> 
> As a side note, it probably wouldn't hurt to set up a guide for 
> running git on /usr/portage, including setting up these symlinks, 
> running egencache after emerge --sync, etc.  I imagine that this is
> a configuration that many developers will tend to use, and with
> the advent of git we may see more users who tend to contribute
> doing the same.
> 
++

- -- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJVyO2pAAoJEAEkDpRQOeFwmMwQAMBNw6VA3UEINvi6RgyW6NJJ
a+6QYB2RigTxOJP3nlnTXywahme1Mxtdp8X/rcebHZ0LUi5XSup+n6mGljcUHsx+
Gl09zShXoHxRdDX2JFsptt6YSVr7gXMgT83iFUDGRAKVPFIvJ4Q0jFzWjj0MD0KW
4363Oz2+5/Wdt85vKcLuT8QLG+9niEaxHab2VgauHieFYPALdTdhaEX0DTxTf/6s
M7oz8IN8p+iKbXbks0q0ZhsbJSIcOKLxE6IQ1alftFFeMkbP5wyxH5QLOFKlk6eG
oojNVXlxwvcz+nVnnYVPLhGDpPjdOndjkJ0P+uqsuLA/QIK7rwKy7VGDaFNX9zUD
2fxUNqKXstQXUBkvrzXDNDBpqWuh9v27bbS9Dx+5iJMVIUNm1YegM+JyIor9bBVT
UKeixNwYanYtWNJDGmFluc3vnIwuDqgMXC9dYrDdk3a1wPXj2l/kUmljl1wr5Ora
9BCaVSLAENg+VaNhQ8IkY5LcUnyLw/O3T4SLydqAJwc0u9+uwrcmSndwPAmRnp7U
eg/AccY1/PXARTK+u56yVzmApP6GHUz+tFIQ3frobYO3YOhT0xZojP8ecuMAkeiw
C4KRtVDRgNXmu/5FPvIMzhNNJPB0U1WFS7ZphOtGG2adRkXl4kGzmVi9QEFGGL3+
sDIclOHeOXp1LGP17Ek4
=jJqm
-----END PGP SIGNATURE-----

Re: [gentoo-dev] .gitignore

[Mike Gilbert]

On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org> wrote:
> On Mon, Aug 10, 2015 at 2:42 AM, Mike Frysinger <vapier@gentoo.org> wrote:
>> On 10 Aug 2015 08:28, Justin (jlec) wrote:
>>> how do we maintain this file?
>>
>> like any other file.  git add && git commit.
>>
>>> I like to propose to add the md5-cache into it. Which other files are of interest?
>>
>> /distfiles/
>> /local/
>> /packages/
>>
>> /metadata/md5-cache/
>> -mike
>
> Expanding on this: the rsync master creates the following
> files/directories under metatdata. On my own system, I like to symlink
> them to locations outside my repo so that related portage features
> continue to work.
>
> I would like to have these added in .gitignore.
>
> metadata/dtd/ # used by something?
> metadata/glsa/ # used by the GLSA utilities?
> matadata/herds.xml # used by equery from gentoolkit
> metadata/news/ # used by eselect news

Heh, Robin has already taken care of these. I did not notice the
.gitignore file in the metadata subdirectory.

rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[hasufell]

On 08/10/2015 05:09 PM, Rich Freeman wrote:
> On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org> wrote:
>>
>> Expanding on this: the rsync master creates the following
>> files/directories under metatdata. On my own system, I like to symlink
>> them to locations outside my repo so that related portage features
>> continue to work.
>>
>> I would like to have these added in .gitignore.
>>
>> metadata/dtd/ # used by something?
>> metadata/glsa/ # used by the GLSA utilities?
>> matadata/herds.xml # used by equery from gentoolkit
>> metadata/news/ # used by eselect news
>>
> 
> As a side note, it probably wouldn't hurt to set up a guide for
> running git on /usr/portage, including setting up these symlinks,
> running egencache after emerge --sync, etc.  I imagine that this is a
> configuration that many developers will tend to use, and with the
> advent of git we may see more users who tend to contribute doing the
> same.
> 

In fact, this should be the recommended way of running gentoo for
everyone. Our rsync methods are still inherently insecure (unless I
missed something), because:
1. machine key
2. profiles, eclasses and so on are not covered with a
signature/Manifest anyway

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1535 bytes --]

On Mon, 10 Aug 2015 22:13:23 +0200 hasufell wrote:
> On 08/10/2015 05:09 PM, Rich Freeman wrote:
> > On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org> wrote:
> >>
> >> Expanding on this: the rsync master creates the following
> >> files/directories under metatdata. On my own system, I like to symlink
> >> them to locations outside my repo so that related portage features
> >> continue to work.
> >>
> >> I would like to have these added in .gitignore.
> >>
> >> metadata/dtd/ # used by something?
> >> metadata/glsa/ # used by the GLSA utilities?
> >> matadata/herds.xml # used by equery from gentoolkit
> >> metadata/news/ # used by eselect news
> >>
> > 
> > As a side note, it probably wouldn't hurt to set up a guide for
> > running git on /usr/portage, including setting up these symlinks,
> > running egencache after emerge --sync, etc.  I imagine that this is a
> > configuration that many developers will tend to use, and with the
> > advent of git we may see more users who tend to contribute doing the
> > same.
> > 
> 
> In fact, this should be the recommended way of running gentoo for
> everyone. Our rsync methods are still inherently insecure (unless I
> missed something), because:
> 1. machine key
> 2. profiles, eclasses and so on are not covered with a
> signature/Manifest anyway
 
Not unless metadata cache will be synced too from a trusted source.
It takes too much time to generate, especially on non-brand-new
hardware.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1865 bytes --]

On Mon, 10 Aug 2015 23:47:21 +0300 Andrew Savchenko wrote:
> On Mon, 10 Aug 2015 22:13:23 +0200 hasufell wrote:
> > On 08/10/2015 05:09 PM, Rich Freeman wrote:
> > > On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org> wrote:
> > >>
> > >> Expanding on this: the rsync master creates the following
> > >> files/directories under metatdata. On my own system, I like to symlink
> > >> them to locations outside my repo so that related portage features
> > >> continue to work.
> > >>
> > >> I would like to have these added in .gitignore.
> > >>
> > >> metadata/dtd/ # used by something?
> > >> metadata/glsa/ # used by the GLSA utilities?
> > >> matadata/herds.xml # used by equery from gentoolkit
> > >> metadata/news/ # used by eselect news
> > >>
> > > 
> > > As a side note, it probably wouldn't hurt to set up a guide for
> > > running git on /usr/portage, including setting up these symlinks,
> > > running egencache after emerge --sync, etc.  I imagine that this is a
> > > configuration that many developers will tend to use, and with the
> > > advent of git we may see more users who tend to contribute doing the
> > > same.
> > > 
> > 
> > In fact, this should be the recommended way of running gentoo for
> > everyone. Our rsync methods are still inherently insecure (unless I
> > missed something), because:
> > 1. machine key
> > 2. profiles, eclasses and so on are not covered with a
> > signature/Manifest anyway
>  
> Not unless metadata cache will be synced too from a trusted source.
> It takes too much time to generate, especially on non-brand-new
> hardware.

Another issue: we will have to setup git mirrors as well (probably
reusing hosts providing rsync mirrors). I really doubt current
infrastructure will survive if all users will sync from its git.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[hasufell]

On 08/10/2015 10:47 PM, Andrew Savchenko wrote:
> On Mon, 10 Aug 2015 22:13:23 +0200 hasufell wrote:
>> On 08/10/2015 05:09 PM, Rich Freeman wrote:
>>> On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org> wrote:
>>>>
>>>> Expanding on this: the rsync master creates the following
>>>> files/directories under metatdata. On my own system, I like to symlink
>>>> them to locations outside my repo so that related portage features
>>>> continue to work.
>>>>
>>>> I would like to have these added in .gitignore.
>>>>
>>>> metadata/dtd/ # used by something?
>>>> metadata/glsa/ # used by the GLSA utilities?
>>>> matadata/herds.xml # used by equery from gentoolkit
>>>> metadata/news/ # used by eselect news
>>>>
>>>
>>> As a side note, it probably wouldn't hurt to set up a guide for
>>> running git on /usr/portage, including setting up these symlinks,
>>> running egencache after emerge --sync, etc.  I imagine that this is a
>>> configuration that many developers will tend to use, and with the
>>> advent of git we may see more users who tend to contribute doing the
>>> same.
>>>
>>
>> In fact, this should be the recommended way of running gentoo for
>> everyone. Our rsync methods are still inherently insecure (unless I
>> missed something), because:
>> 1. machine key
>> 2. profiles, eclasses and so on are not covered with a
>> signature/Manifest anyway
>  
> Not unless metadata cache will be synced too from a trusted source.
> It takes too much time to generate, especially on non-brand-new
> hardware.
> 

I was wondering if that could be automated in a separate branch (only
needs to update in 24h intervals).

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Aaron W. Swenson]

[-- Attachment #1: Type: text/plain, Size: 315 bytes --]

On 2015-08-10 23:49, Andrew Savchenko wrote:
> Another issue: we will have to setup git mirrors as well (probably
> reusing hosts providing rsync mirrors). I really doubt current
> infrastructure will survive if all users will sync from its git.

Users can fetch/pull from Github.

https://github.com/gentoo/gentoo

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 345 bytes --]

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1845 bytes --]

Dnia 2015-08-10, o godz. 23:47:21
Andrew Savchenko <bircoph@gentoo.org> napisał(a):

> On Mon, 10 Aug 2015 22:13:23 +0200 hasufell wrote:
> > On 08/10/2015 05:09 PM, Rich Freeman wrote:
> > > On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org> wrote:
> > >>
> > >> Expanding on this: the rsync master creates the following
> > >> files/directories under metatdata. On my own system, I like to symlink
> > >> them to locations outside my repo so that related portage features
> > >> continue to work.
> > >>
> > >> I would like to have these added in .gitignore.
> > >>
> > >> metadata/dtd/ # used by something?
> > >> metadata/glsa/ # used by the GLSA utilities?
> > >> matadata/herds.xml # used by equery from gentoolkit
> > >> metadata/news/ # used by eselect news
> > >>
> > > 
> > > As a side note, it probably wouldn't hurt to set up a guide for
> > > running git on /usr/portage, including setting up these symlinks,
> > > running egencache after emerge --sync, etc.  I imagine that this is a
> > > configuration that many developers will tend to use, and with the
> > > advent of git we may see more users who tend to contribute doing the
> > > same.
> > > 
> > 
> > In fact, this should be the recommended way of running gentoo for
> > everyone. Our rsync methods are still inherently insecure (unless I
> > missed something), because:
> > 1. machine key
> > 2. profiles, eclasses and so on are not covered with a
> > signature/Manifest anyway
>  
> Not unless metadata cache will be synced too from a trusted source.
> It takes too much time to generate, especially on non-brand-new
> hardware.

Err, it takes around 2 minutes to generate full cache with pkgcore on
some old Xeon. Updates are much faster.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2330 bytes --]

Dnia 2015-08-10, o godz. 23:49:55
Andrew Savchenko <bircoph@gentoo.org> napisał(a):

> On Mon, 10 Aug 2015 23:47:21 +0300 Andrew Savchenko wrote:
> > On Mon, 10 Aug 2015 22:13:23 +0200 hasufell wrote:
> > > On 08/10/2015 05:09 PM, Rich Freeman wrote:
> > > > On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org> wrote:
> > > >>
> > > >> Expanding on this: the rsync master creates the following
> > > >> files/directories under metatdata. On my own system, I like to symlink
> > > >> them to locations outside my repo so that related portage features
> > > >> continue to work.
> > > >>
> > > >> I would like to have these added in .gitignore.
> > > >>
> > > >> metadata/dtd/ # used by something?
> > > >> metadata/glsa/ # used by the GLSA utilities?
> > > >> matadata/herds.xml # used by equery from gentoolkit
> > > >> metadata/news/ # used by eselect news
> > > >>
> > > > 
> > > > As a side note, it probably wouldn't hurt to set up a guide for
> > > > running git on /usr/portage, including setting up these symlinks,
> > > > running egencache after emerge --sync, etc.  I imagine that this is a
> > > > configuration that many developers will tend to use, and with the
> > > > advent of git we may see more users who tend to contribute doing the
> > > > same.
> > > > 
> > > 
> > > In fact, this should be the recommended way of running gentoo for
> > > everyone. Our rsync methods are still inherently insecure (unless I
> > > missed something), because:
> > > 1. machine key
> > > 2. profiles, eclasses and so on are not covered with a
> > > signature/Manifest anyway
> >  
> > Not unless metadata cache will be synced too from a trusted source.
> > It takes too much time to generate, especially on non-brand-new
> > hardware.
> 
> Another issue: we will have to setup git mirrors as well (probably
> reusing hosts providing rsync mirrors). I really doubt current
> infrastructure will survive if all users will sync from its git.

Do you mean git mirrors of the original repo, or md5-cache propagated
copy for users syncing? The latter is available for a few months now
[1,2].

[1]:https://github.com/gentoo-mirror/
[2]:https://wiki.gentoo.org/wiki/Project:Repository_mirror_and_CI

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

[gentoo-dev] Re: rsync mirror security

[Matthias Maier]

[-- Attachment #1: Type: text/plain, Size: 259 bytes --]

> Users can fetch/pull from Github.

We could also provide automatic signed tags every 30min/1h/2h/whatever
(signed with a suitable infrastructure key). With that, the integrity of
a tagged git checkout can be easily verified on client side.

Best,
Matthias


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 820 bytes --]

Re: [gentoo-dev] Re: rsync mirror security

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]

On 10 Aug 2015 16:05, Matthias Maier wrote:
> > Users can fetch/pull from Github.
> 
> We could also provide automatic signed tags every 30min/1h/2h/whatever
> (signed with a suitable infrastructure key). With that, the integrity of
> a tagged git checkout can be easily verified on client side.

it would have to re-use the same tag name every time otherwise we end up with 
17.5k/8.7k/4.3k/whatever new tags per year ... a really bad idea

depending on how fast the process is, it could just be part of the receive hook
on the server that does the checking now.  that way the tag is always up to date
with every push a developer makes.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Re: rsync mirror security

[Kent Fredric]

On 11 August 2015 at 09:05, Matthias Maier <tamiko@gentoo.org> wrote:
> We could also provide automatic signed tags every 30min/1h/2h/whatever
> (signed with a suitable infrastructure key). With that, the integrity of
> a tagged git checkout can be easily verified on client side.


I'm distinctly under the impression that a signed tag doesn't really
give you anything a signed commit wouldn't.

That is, I was under the impression signing a tag only signs the
references themselves, and then relies on SHA1 referential integrity
beyond that.


Hence, a signed tag basically is a statement proving X author
authorized Y-SHA1, and then it subsequently implies that X author
authorized whatever Y-SHA1 refers to.

So adding additional tags *just* for the purpose of having a periodic
signature would give no benefit over the "all tags are signed, all
commits are signed" mechanism for git users, and the signed tag could
_not_ be verified against an RSYNC clone.

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] Re: rsync mirror security

[Kent Fredric]

On 11 August 2015 at 15:06, Mike Frysinger <vapier@gentoo.org> wrote:
> it would have to re-use the same tag name every time otherwise we end up with
> 17.5k/8.7k/4.3k/whatever new tags per year ... a really bad idea


I was very much under the impression git is not designed with repeated
tag replication in consideration.

The git tag documentation very much implies that any tag having its
reference changed will result in effort being required of everyone who
wishes to consume that tag. ( It literally brands  the act of
re-tagging things to be "insane" )

Tags are very much intended to be immutable references to commits.

If you need mutable references to commits, isn't that what branches are for?



-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] Re: rsync mirror security

[Matthias Maier]

> That is, I was under the impression signing a tag only signs the
> references themselves, and then relies on SHA1 referential integrity
> beyond that.

No, a signed tag verifies that the whole integrirty of the entire
repository, whereas a signed commit only authenticates the differences
introduced by a single commit.

As long as there are no conflicts, a signed commit can be rebased
freely (especially also on top of malicious commits...).

Best,
Matthias

Re: [gentoo-dev] Re: rsync mirror security

[Matthias Maier]

[-- Attachment #1: Type: text/plain, Size: 269 bytes --]


> it would have to re-use the same tag name every time otherwise we end up with 
> 17.5k/8.7k/4.3k/whatever new tags per year ... a really bad idea

Or we supply a signature of the sha1-sum of the tag in question by some
external procedure...

Best,
Matthias

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 820 bytes --]

Re: [gentoo-dev] Re: rsync mirror security

[Kent Fredric]

On 11 August 2015 at 15:44, Matthias Maier <tamiko@gentoo.org> wrote:
>
> No, a signed tag verifies that the whole integrirty of the entire
> repository, whereas a signed commit only authenticates the differences
> introduced by a single commit.


git tag -s test

cat ./.git/refs/tags/test
456d216e3d1894d62429daf0ec482c3afb087dbe

git cat-file tag 456d216e3d1894d62429daf0ec482c3afb087dbe
object 9ca77ee7f72902e4e89456ff560a670465969603
type commit
tag test
tagger Kent Fredric <kentfredric@gmail.com> 1439264837 +1200

A test tag
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAABCAAGBQJVyXBKAAoJEOhUMksTZqgg2/kP/iCXS12W57RB2wPQHgebgSpK
86zXXvXC5rqndTmGwOmYA9FcO/n2u+SMwk0ZGol9LWuvkKaW/6Wi/vzvG24lggWy
GxKRQTNHPXVHxwPQZOhj6fwS9EkC3rCSMWv82qLrbXvBqsH/dLXymq2nl+YDEGi1
lLkDWkX7EYWA6sgdnDhNzjPaHVC9P5qP1JDZOrKY0Qzm9JBDMl0xO9/faITrBMDi
BmVVHNELKQ9uN8BYxmQfHqUFKO2SWXFbqJftQ6LqpXmFHWDpasmY3gTMczPpQ47I
le+LPo0tT3Yk0fhBc8uk0/69kaHMa5hMmBPHuHh5ANWLPpxSyiDzCqqS9i8wPB+M
MONhAoVyLYaFUf62fBxa6kxKDdQuC5JRYjeiFs60k1uG/QI4OhjoIbbaaxJxQ0sy
45iZ3PBlVxbgxkpPRJtftr9PJBMDabekZbI5F6jX7S+x6G40O4ss1W1QnXsdFvqd
vJvVdIdnrGqu/6JXZpz2J65N3HfMqfj9PHNDJaxM6da6+z6HQ3JwvNSVum8dAaJn
jKoisQ7bEuXl2WOj5SCfAqjtOUp2pbYJCCb5QVHWuHCk53cvcY6FmGQPEzj42uVQ
bKSYGaJ3637t+NPysinifQv1HTfViP7lh/O3znsGj7qcm6DXGnHvkp84LFch6yiY
/oFbaDvWZ8zKyMSAJ9Ou
=Ieic
-----END PGP SIGNATURE-----



git cat-file tag 456d216e3d1894d62429daf0ec482c3afb087dbe > /tmp/sigfile
cp /tmp/sigfile /tmp/sigfile.asc

*edits both so sigfile has content, and asc file has signature*


gpg --verify /tmp/sigfile.asc
gpg: enabled debug flags: memstat
gpg: assuming signed data in '/tmp/sigfile'
gpg: Signature made Tue Aug 11 15:47:22 2015 NZST
gpg:                using RSA key E854324B1366A820
gpg: Good signature from "Kent Fredric (GMail)
<kentfredric@gmail.com>" [unknown]
gpg:                 aka "Kent Fredric (CPAN Author)
<kentnl@cpan.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 3D96 B36C 8FEA AC54 F5A3  DAE7 E854 324B 1366 A820
gpg: keydb: kid_not_found_table: total: 1
gpg: random usage: poolsize=600 mixed=0 polls=0/0 added=0/0
              outmix=0 getlvl1=0/0 getlvl2=0/0
gpg: secmem usage: 0/65536 bytes in 0 blocks


^^ - so its clear the signature is only on the tag data itself.

And what does the tag refer to?

object 9ca77ee7f72902e4e89456ff560a670465969603

What is that?


git cat-file -t 9ca77ee7f72902e4e89456ff560a670465969603
commit


So how is GPG verifying "The whole repository" ?

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] Re: rsync mirror security

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1190 bytes --]

On 11 Aug 2015 15:23, Kent Fredric wrote:
> On 11 August 2015 at 15:06, Mike Frysinger wrote:
> > it would have to re-use the same tag name every time otherwise we end up with
> > 17.5k/8.7k/4.3k/whatever new tags per year ... a really bad idea
> 
> I was very much under the impression git is not designed with repeated
> tag replication in consideration.

git has no problem fetching rewritten tags.  internally, it doesn't care
either -- a tag is merely a reference to an object.

> The git tag documentation very much implies that any tag having its
> reference changed will result in effort being required of everyone who
> wishes to consume that tag. ( It literally brands  the act of
> re-tagging things to be "insane" )
> 
> Tags are very much intended to be immutable references to commits.

the git docs take the stance that publishing any mutable names is wrong.
same goes for rebasing and publishing rewritten history.  that's simply
the recommended practice.  it doesn't mean that the world blows up when
you do rewrite things.

> If you need mutable references to commits, isn't that what branches are for?

no, that's not what they're for.  
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Re: rsync mirror security

[Matthias Maier]

On Mon, Aug 10, 2015, at 22:56 CDT, Kent Fredric <kentfredric@gmail.com> wrote:

> So how is GPG verifying "The whole repository" ?

You can verify the state of the repository via
  $ git fsck

after that you can verify that the current HEAD is tagged with a valid
and singed tag with something like

  $ git tag -v `git describe HEAD`

This verifies the integrity of the whole history up to HEAD - at least
if you consider sha1 to be cryptographically

Best,
Matthias


PS.: I think I was mistaken with respect to individually signed
commits - the verification seems to be stricter than I thought.

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Alexander Berntsen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 10/08/15 22:59, Aaron W. Swenson wrote:
> Users can fetch/pull from Github.
Users should not have to interface with or rely on proprietary
software to use Gentoo.
- -- 
Alexander
bernalex@gentoo.org
https://secure.plaimi.net/~alexander
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJVybKCAAoJENQqWdRUGk8BIWcQANRXyH+ZRJImlbASQM1QSApS
hr7Y8xFj8Wm9Ym8gTJCCxyWW8frW7yCyAJByHF6znRCjxl3qRGfKGnMd/gfYR5l8
UmRHVKaqborjzkhd6+W0EtjvxWJ6bwI2FGG4p+xCk7uVAN4V2leu2gxSeUTfWO18
DfJd92y7LM5xrzsmE5KUyB1uaHzdfxbZpR1tHix/jzK6Wpp90/lJVVL3AwnRz6Cu
T9TmTXkVnWck7DIHU8OKluxgdKrEbkKRkB5wzgESAHd75T9HvECwBpFcAXkdr4X2
vnYx4+ZXvydgu8WAFimAefbpbQVaCZd8AQ9XFOIo3tiy8di5ALlSKNXG6cdVlsue
GaU+qNkJavzOI+QtydJGogY7hhT3rAP1BBeeZxXdTjBO9qQUB3LCcK6IsXtbtQw/
+cbmPJPm4r1wRJD1x7NxU/1LWO3JNTcM1YxvskdKKiDYAnBadMEcjPHWt7vUA8QR
GWze8YvcAr6VDjPqd366kM/fvmY4Y/2TNl3wWCB18Nagoq6dx2ba0//4pMUpfOdS
/1WDMoJRQ/M4cfShgZvNnvCM7qNcJXzLc68fD1xzzvWEyhkOj/vydeztzNykLzop
BUgl08J0jgokbVAwxzlnOubtK2F7vSLu6hFBliVQINoNqbPpTjxE3NIJvPfFkeUL
qhbaV+CoPE7MNeH4a0lt
=asRz
-----END PGP SIGNATURE-----

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 522 bytes --]

Dnia 2015-08-11, o godz. 10:29:55
Alexander Berntsen <bernalex@gentoo.org> napisał(a):

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> On 10/08/15 22:59, Aaron W. Swenson wrote:
> > Users can fetch/pull from Github.
> Users should not have to interface with or rely on proprietary
> software to use Gentoo.

Then please provide them with true open-source infrastructure. And also
remember to block all the mirrors by default.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Kent Fredric]

On 11 August 2015 at 20:38, Michał Górny <mgorny@gentoo.org> wrote:
> Dnia 2015-08-11, o godz. 10:29:55
> Alexander Berntsen <bernalex@gentoo.org> napisał(a):
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA512
>>
>> On 10/08/15 22:59, Aaron W. Swenson wrote:
>> > Users can fetch/pull from Github.
>> Users should not have to interface with or rely on proprietary
>> software to use Gentoo.
>
> Then please provide them with true open-source infrastructure. And also
> remember to block all the mirrors by default.


Its fine to say "can" in the context of "They may if they want to, but
they are not forced to".

Having a quality infrastructure should happen in parallel to github mirrors.

Uses may use the proprietary one or the opensource one.

As long as nothing *demands* they use the proprietary instead of the
opensource one, and there is a working path that is usable and
covenient to avoid the proprietary ( which there is in this case ),
then there's no real foul.

The only downside is realistically, if all users cloned from gentoo
infra using git, then we would drown.

"Ban mirrors" wouldn't fix this problem, "Ban github" wouldn't fix this problem.

So you basically *must* implement a reasonable infrastructure, and
they can use github instead if it is more convenient for them.

To an extent this does imply we're relying that *some* users will use
github/other to decrease our server load.

Meh.

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: [gentoo-dev] Re: rsync mirror security

[Rich Freeman]

On Mon, Aug 10, 2015 at 11:44 PM, Matthias Maier <tamiko@gentoo.org> wrote:
>> That is, I was under the impression signing a tag only signs the
>> references themselves, and then relies on SHA1 referential integrity
>> beyond that.
>
> No, a signed tag verifies that the whole integrirty of the entire
> repository, whereas a signed commit only authenticates the differences
> introduced by a single commit.
>
> As long as there are no conflicts, a signed commit can be rebased
> freely (especially also on top of malicious commits...).
>

A signed commit and a signed tag are basically equivalent as far as
authentication of the contents of the tree go.  All a tag does is
reference a commit by sha1, and a commit references the top level
directory of the tree by sha1 in the state it was in when it was
created.

Sure, you can rebase a commit, but that doesn't actually change a
commit.  It creates one or more new commits in the place of a bunch of
existing ones with new sha1s, and points the current head at the last
one.  If the old commits are no longer referenced by any other heads
they will get garbage collected.  If you point two heads at the same
commit and do a rebase the history as seen by the other head won't
change at all.

Since a tag is just a label it is actually EASIER to tamper with than
a commit.  You can't change a commit without changing its hash.  tags
are referenced by name, not by hash, which is basically the whole
point, so you CAN change the content of a tag and have it keep the
same name.  Of course, if you try to push/pull that new tag git is
going to complain about the inconsistency, just as it does if you try
to do a non-fast-forward push and so on.

I don't think that having a bazillion tags or rewriting them
constantly adds any security to the tree.  What might add security for
end-users is if git automatically checked the push signatures, which
are the signatures that ensure that branches aren't tampered with
(which is what rebasing you bring up actually does).

-- 
Rich

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Rich Freeman]

On Tue, Aug 11, 2015 at 5:07 AM, Kent Fredric <kentfredric@gmail.com> wrote:
>
> Having a quality infrastructure should happen in parallel to github mirrors.
>
> Uses may use the proprietary one or the opensource one.
>

While I generally tend to agree with you, if we're just talking about
mirroring is this a real problem?

Right now Gentoo has a large number of rsync/http mirrors.  As far as
any of us are concerned, they're just an DNS address that speaks
rsync/http.  None of us have any idea what OS or software they're
running.  If one of our mirrors is IIS running on Windows 7, that is
pretty transparent to the end user.  They're just mirrors.

That is basically all github is in this case.  A commit shows up in
the gentoo infra repository, and some process somewhere pushes it to
the github repository.  If we were to set up an independent network of
git mirrors, they'd probably work the same way.  (Git should actually
be pretty easy to mirror.)  To an end user all they see is a DNS name
that talks whatever protocol git uses.  Short of an on-site inspection
you'd never be able to prove that it is actually FOSS.

Apologies if I sounds like an MS "open standards, not open source"
shill - but to some extent when you're talking about networked
services they work out to be the same thing.  I think it is far more
important to keep the infrastructure that creates the tree pure-FOSS
(and documented/published so that anybody who wants to could basically
"roll their own Gentoo").  If we use a more commercial service to just
help scale it up like a CDN or something like github, that isn't
really as essential to the essence of Gentoo.  I do think that people
who complain about depending on a github-based workflow have a
legitimate concern, but that isn't what we're talking about here.

In any case, nobody is getting rid of the rsync mirrors anytime soon,
so we don't have to be in any rush to figure this out.  Consider this
thinking out loud if you will...

-- 
Rich

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 460 bytes --]

On Tue, 11 Aug 2015 10:29:55 +0200
Alexander Berntsen <bernalex@gentoo.org> wrote:
> On 10/08/15 22:59, Aaron W. Swenson wrote:
> > Users can fetch/pull from Github.
> Users should not have to interface with or rely on proprietary
> software to use Gentoo.

Like the stuff running on the big expensive routers that make the
internets work? Can I have my tree delivered by pigeon, since I suspect
the post office runs Windows?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Rich Freeman]

On Tue, Aug 11, 2015 at 8:26 AM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Tue, 11 Aug 2015 10:29:55 +0200
> Alexander Berntsen <bernalex@gentoo.org> wrote:
>> On 10/08/15 22:59, Aaron W. Swenson wrote:
>> > Users can fetch/pull from Github.
>> Users should not have to interface with or rely on proprietary
>> software to use Gentoo.
>
> Like the stuff running on the big expensive routers that make the
> internets work? Can I have my tree delivered by pigeon, since I suspect
> the post office runs Windows?

Only if that pigeon has had its personal genome sequenced.

BTW, did I mention that once we get all the dev gpg keys sorted out
we're going to need a cheek swab from everybody?

-- 
Rich

Re: [gentoo-dev] Re: rsync mirror security

[Matthias Maier]

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]


> constantly adds any security to the tree.  What might add security for
> end-users is if git automatically checked the push signatures, which
> are the signatures that ensure that branches aren't tampered with
> (which is what rebasing you bring up actually does).

It is news to me that a signature from a push is also transported to a
subsequent pull request for a client, do you have some external
references for this procedure?

Regardless of the technical implementation, the fact still remains that
with the current git repositories (gentoo and the one populated with
metadata from gentoo-mirror) we might have another way of providing
a signed and tamper-proof [1] ebuild tree (apart from our daily, signed
snapshots).

Best,
Matthias

[1] At least as long our git infrastructure is not compromised...

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 820 bytes --]

Re: [gentoo-dev] Re: rsync mirror security

[Rich Freeman]

On Tue, Aug 11, 2015 at 10:53 AM, Matthias Maier <tamiko@gentoo.org> wrote:
>
>> constantly adds any security to the tree.  What might add security for
>> end-users is if git automatically checked the push signatures, which
>> are the signatures that ensure that branches aren't tampered with
>> (which is what rebasing you bring up actually does).
>
> It is news to me that a signature from a push is also transported to a
> subsequent pull request for a client, do you have some external
> references for this procedure?
>

They're stored in the tree under the ref refs/push-certs.  I have no
idea how to go about verifying them - they're pretty new so there
aren't a lot of docs.  I had no idea they were even there until Robin
answered a similar question I asked him.

git ls-remote for those curious about what other refs are lying around.

-- 
Rich

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2069 bytes --]

Dnia 2015-08-10, o godz. 22:51:59
hasufell <hasufell@gentoo.org> napisał(a):

> On 08/10/2015 10:47 PM, Andrew Savchenko wrote:
> > On Mon, 10 Aug 2015 22:13:23 +0200 hasufell wrote:
> >> On 08/10/2015 05:09 PM, Rich Freeman wrote:
> >>> On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org> wrote:
> >>>>
> >>>> Expanding on this: the rsync master creates the following
> >>>> files/directories under metatdata. On my own system, I like to symlink
> >>>> them to locations outside my repo so that related portage features
> >>>> continue to work.
> >>>>
> >>>> I would like to have these added in .gitignore.
> >>>>
> >>>> metadata/dtd/ # used by something?
> >>>> metadata/glsa/ # used by the GLSA utilities?
> >>>> matadata/herds.xml # used by equery from gentoolkit
> >>>> metadata/news/ # used by eselect news
> >>>>
> >>>
> >>> As a side note, it probably wouldn't hurt to set up a guide for
> >>> running git on /usr/portage, including setting up these symlinks,
> >>> running egencache after emerge --sync, etc.  I imagine that this is a
> >>> configuration that many developers will tend to use, and with the
> >>> advent of git we may see more users who tend to contribute doing the
> >>> same.
> >>>
> >>
> >> In fact, this should be the recommended way of running gentoo for
> >> everyone. Our rsync methods are still inherently insecure (unless I
> >> missed something), because:
> >> 1. machine key
> >> 2. profiles, eclasses and so on are not covered with a
> >> signature/Manifest anyway
> >  
> > Not unless metadata cache will be synced too from a trusted source.
> > It takes too much time to generate, especially on non-brand-new
> > hardware.
> > 
> 
> I was wondering if that could be automated in a separate branch (only
> needs to update in 24h intervals).

Please don't cruft the repo with huge metadata. And I have
metadata-applied mirrors for all repositories at [1].

[1]:https://github.com/gentoo-mirror/

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[hasufell]

On 08/11/2015 06:42 PM, Michał Górny wrote:
> Dnia 2015-08-10, o godz. 22:51:59
> hasufell <hasufell@gentoo.org> napisał(a):
> 
>>
>> I was wondering if that could be automated in a separate branch (only
>> needs to update in 24h intervals).
> 
> Please don't cruft the repo with huge metadata. And I have
> metadata-applied mirrors for all repositories at [1].
> 
> [1]:https://github.com/gentoo-mirror/
> 

The problem with those mirrors is... the history is gone and the
signatures as well. So people would have to clone the metadata-cache
from that mirror, put it into the real mirror clone and then probably
still update it via egencache, because they are not perfectly in sync.

Re: rsync mirror security (WAS: Re: [gentoo-dev] .gitignore)

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

Dnia 2015-08-11, o godz. 18:45:40
hasufell <hasufell@gentoo.org> napisał(a):

> On 08/11/2015 06:42 PM, Michał Górny wrote:
> > Dnia 2015-08-10, o godz. 22:51:59
> > hasufell <hasufell@gentoo.org> napisał(a):
> > 
> >>
> >> I was wondering if that could be automated in a separate branch (only
> >> needs to update in 24h intervals).
> > 
> > Please don't cruft the repo with huge metadata. And I have
> > metadata-applied mirrors for all repositories at [1].
> > 
> > [1]:https://github.com/gentoo-mirror/
> > 
> 
> The problem with those mirrors is... the history is gone and the
> signatures as well. So people would have to clone the metadata-cache
> from that mirror, put it into the real mirror clone and then probably
> still update it via egencache, because they are not perfectly in sync.

I know. I'm planning to improve it when I have some time. So far this
was easier because not all repos are git, and I'm not really into
trying hard to convert other VCS-es.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] .gitignore

[hasufell]

On 08/10/2015 05:09 PM, Rich Freeman wrote:
> On Mon, Aug 10, 2015 at 11:04 AM, Mike Gilbert <floppym@gentoo.org> wrote:
>>
>> Expanding on this: the rsync master creates the following
>> files/directories under metatdata. On my own system, I like to symlink
>> them to locations outside my repo so that related portage features
>> continue to work.
>>
>> I would like to have these added in .gitignore.
>>
>> metadata/dtd/ # used by something?
>> metadata/glsa/ # used by the GLSA utilities?
>> matadata/herds.xml # used by equery from gentoolkit
>> metadata/news/ # used by eselect news
>>
> 
> As a side note, it probably wouldn't hurt to set up a guide for
> running git on /usr/portage, including setting up these symlinks,
> running egencache after emerge --sync, etc.  I imagine that this is a
> configuration that many developers will tend to use, and with the
> advent of git we may see more users who tend to contribute doing the
> same.
> 

https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Using_the_gentoo_git_checkout_as_your_local_tree