[gentoo-dev] Cleaning up integration of external repos into ::gentoo

[gentoo-dev] Cleaning up integration of external repos into ::gentoo

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1371 bytes --]

Hi,

Now that we have the official git repository, I've switched user-facing
git mirrors from rsync->git to the real git. As a result, users are now
complaining that some random Gentoo metadata has disappeared.

As you may have noticed already, I am *really* unhappy about adding
special conditionals with hardcoded URLs (and ugly mixing stuff) for
Gentoo. So if you really don't want to put the stuff into the repo,
please at least:

1. Move DTDs to a git repository (seriously, I don't want to have to
touch CVS).

2. Strip the year from gentoo-news [1]. I think we have all
the agreement to proceed there. I will prepare a GLEP update today,
and if given OK, I can update the repo, and the scripts.

3. Provide a machine-readable description where those files/directories
come from. Not sure how that should look but how about something along
the lines of repos.conf:

  metadata/external-repos.conf::

    [gentoo-news]
    location = metadata/news
    sync-type = git
    sync-uri = https://anongit.gentoo.org/proj/gentoo-news.git

    [herds]
    location = metadata/herds.xml
    # or do we want to use git here?
    sync-type = http
    sync-uri = https://api.gentoo.org/packages/herds.xml

Comments?

[1]:https://bugs.gentoo.org/show_bug.cgi?id=523828

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] Cleaning up integration of external repos into ::gentoo

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1549 bytes --]

On Thu, 13 Aug 2015 09:12:30 +0200 Michał Górny wrote:
> Hi,
> 
> Now that we have the official git repository, I've switched user-facing
> git mirrors from rsync->git to the real git. As a result, users are now
> complaining that some random Gentoo metadata has disappeared.
> 
> As you may have noticed already, I am *really* unhappy about adding
> special conditionals with hardcoded URLs (and ugly mixing stuff) for
> Gentoo. So if you really don't want to put the stuff into the repo,
> please at least:

IMO the best way will be to put mandatory stuff in the
repo/gentoo.git.
 
> 1. Move DTDs to a git repository (seriously, I don't want to have to
> touch CVS).
> 
> 2. Strip the year from gentoo-news [1]. I think we have all
> the agreement to proceed there. I will prepare a GLEP update today,
> and if given OK, I can update the repo, and the scripts.
> 
> 3. Provide a machine-readable description where those files/directories
> come from. Not sure how that should look but how about something along
> the lines of repos.conf:
> 
>   metadata/external-repos.conf::
> 
>     [gentoo-news]
>     location = metadata/news
>     sync-type = git
>     sync-uri = https://anongit.gentoo.org/proj/gentoo-news.git
> 
>     [herds]
>     location = metadata/herds.xml
>     # or do we want to use git here?
>     sync-type = http
>     sync-uri = https://api.gentoo.org/packages/herds.xml
> 
> Comments?
> 
> [1]:https://bugs.gentoo.org/show_bug.cgi?id=523828
> 


Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] Cleaning up integration of external repos into ::gentoo

[Rich Freeman]

On Thu, Aug 13, 2015 at 5:26 AM, Andrew Savchenko <bircoph@gentoo.org> wrote:
> On Thu, 13 Aug 2015 09:12:30 +0200 Michał Górny wrote:
>> Hi,
>>
>> Now that we have the official git repository, I've switched user-facing
>> git mirrors from rsync->git to the real git. As a result, users are now
>> complaining that some random Gentoo metadata has disappeared.
>>
>> As you may have noticed already, I am *really* unhappy about adding
>> special conditionals with hardcoded URLs (and ugly mixing stuff) for
>> Gentoo. So if you really don't want to put the stuff into the repo,
>> please at least:
>
> IMO the best way will be to put mandatory stuff in the
> repo/gentoo.git.

Tend to agree.  The main reason not to would be if they were
maintained by different groups and we wanted to control access.  If
"news" were about PR that might be a real issue, but the GLEP news is
more about package notifications, and the security team probably also
needs tree access as well, so at least at present I'm not sure I see
the need to separate them.  GLSA state is also pretty closely
associated with tree state so it really seems like it should be in the
tree.

If we do go the route of a machine-readable external-repos.conf or
something like that then perhaps it would make sense to have the
client just sync this directly, rather than gluing it all together
before sending it to them.  That would also simplify things.

-- 
Rich

Re: [gentoo-dev] Cleaning up integration of external repos into ::gentoo

[Taahir Ahmed]

[-- Attachment #1: Type: text/plain, Size: 1321 bytes --]

On Thu, Aug 13, 2015 at 5:26 AM, Andrew Savchenko <bircoph@gentoo.org> wrote:
> IMO the best way will be to put mandatory stuff in the
> repo/gentoo.git.

On Thursday 13 August 2015 07:02:59 Rich Freeman wrote:
> Tend to agree.  The main reason not to would be if they were
> maintained by different groups and we wanted to control access.  If
> "news" were about PR that might be a real issue, but the GLEP news is
> more about package notifications, and the security team probably also
> needs tree access as well, so at least at present I'm not sure I see
> the need to separate them.  GLSA state is also pretty closely
> associated with tree state so it really seems like it should be in the
> tree.

Is there some consensus on integrating this stuff into the main
repository?

I've recently converted my cave setup to directly sync from the git
mirror.  Hasufell wrote an excellent example configuration for cave [1],
but 90% of the configuration changes are hooks for gluing clones of dtd,
glsa, herds, and news into the checked-out copy of repos/gentoo.git.

Things would be a lot simpler if these were just in repos/gentoo.git,
especially since they are tightly coupled to the state of the tree.

[1] https://github.com/hasufell/paludis-gentoo-git-config/tree/master/etc/paludis/hooks/sync_all_post

Taahir Ahmed

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]