* [gentoo-dev] RFC: Add new remote-id types in metadata.dtd @ 2012-04-19 15:31 Corentin Chary 2012-04-19 16:54 ` Michał Górny ` (5 more replies) 0 siblings, 6 replies; 27+ messages in thread From: Corentin Chary @ 2012-04-19 15:31 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 992 bytes --] Add rubygems, github, gitorious, pecl, pear, bitbucket. All of them are handled by my remoteids.py script. ref: https://bugs.gentoo.org/show_bug.cgi?id=406287 ref: https://github.com/iksaif/portage-janitor/blob/master/remoteids.py --- a/metadata/dtd/metadata.dtd 2010-03-02 18:52:11.000000000 +0100 +++ b/metadata/dtd/metadata.dtd 2012-04-19 14:22:14.077954310 +0200 @@ -61,7 +61,7 @@ <!ELEMENT bugs-to (#PCDATA)> <!-- specify a type of package identification tracker --> <!ELEMENT remote-id (#PCDATA)> - <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) #REQUIRED> + <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gitorious|pecl|pear|bitbucket) #REQUIRED> <!-- category/package information for cross-linking in descriptions and useflag descriptions --> -- Corentin Chary http://xf.iksaif.net/ [-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-19 15:31 [gentoo-dev] RFC: Add new remote-id types in metadata.dtd Corentin Chary @ 2012-04-19 16:54 ` Michał Górny 2012-04-19 19:32 ` Corentin Chary 2012-04-19 18:08 ` "Paweł Hajdan, Jr." ` (4 subsequent siblings) 5 siblings, 1 reply; 27+ messages in thread From: Michał Górny @ 2012-04-19 16:54 UTC (permalink / raw To: gentoo-dev; +Cc: corentin.chary [-- Attachment #1: Type: text/plain, Size: 495 bytes --] On Thu, 19 Apr 2012 17:31:11 +0200 Corentin Chary <corentin.chary@gmail.com> wrote: > - <!ATTLIST remote-id type > (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) > #REQUIRED> > + <!ATTLIST remote-id type > (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gitorious|pecl|pear|bitbucket) > #REQUIRED> Wouldn't it be better if we kept them sorted? -- Best regards, Michał Górny [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 316 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-19 16:54 ` Michał Górny @ 2012-04-19 19:32 ` Corentin Chary 2012-04-21 9:32 ` Michał Górny 0 siblings, 1 reply; 27+ messages in thread From: Corentin Chary @ 2012-04-19 19:32 UTC (permalink / raw To: Michał Górny; +Cc: gentoo-dev On Thu, Apr 19, 2012 at 6:54 PM, Michał Górny <mgorny@gentoo.org> wrote: > On Thu, 19 Apr 2012 17:31:11 +0200 > Corentin Chary <corentin.chary@gmail.com> wrote: > >> - <!ATTLIST remote-id type >> (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) >> #REQUIRED> >> + <!ATTLIST remote-id type >> (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gitorious|pecl|pear|bitbucket) >> #REQUIRED> > > Wouldn't it be better if we kept them sorted? There were not, so I just appended them, but I can provide another patch sorting them too if preferred. -- Corentin Chary http://xf.iksaif.net ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-19 19:32 ` Corentin Chary @ 2012-04-21 9:32 ` Michał Górny 2012-04-21 11:38 ` Corentin Chary 0 siblings, 1 reply; 27+ messages in thread From: Michał Górny @ 2012-04-21 9:32 UTC (permalink / raw To: gentoo-dev; +Cc: corentin.chary [-- Attachment #1: Type: text/plain, Size: 945 bytes --] On Thu, 19 Apr 2012 21:32:49 +0200 Corentin Chary <corentin.chary@gmail.com> wrote: > On Thu, Apr 19, 2012 at 6:54 PM, Michał Górny <mgorny@gentoo.org> > wrote: > > On Thu, 19 Apr 2012 17:31:11 +0200 > > Corentin Chary <corentin.chary@gmail.com> wrote: > > > >> - <!ATTLIST remote-id type > >> (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) > >> #REQUIRED> > >> + <!ATTLIST remote-id type > >> (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gitorious|pecl|pear|bitbucket) > >> #REQUIRED> > > > > Wouldn't it be better if we kept them sorted? > > There were not, so I just appended them, but I can provide another > patch sorting them too if preferred. I'll appreciate if you could do that, along with that cpan-module addition, and if noone objects, I'll just commit that. -- Best regards, Michał Górny [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 316 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-21 9:32 ` Michał Górny @ 2012-04-21 11:38 ` Corentin Chary 0 siblings, 0 replies; 27+ messages in thread From: Corentin Chary @ 2012-04-21 11:38 UTC (permalink / raw To: Michał Górny; +Cc: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1843 bytes --] On Sat, Apr 21, 2012 at 11:32:26AM +0200, Michał Górny wrote: > On Thu, 19 Apr 2012 21:32:49 +0200 > Corentin Chary <corentin.chary@gmail.com> wrote: > > > On Thu, Apr 19, 2012 at 6:54 PM, Michał Górny <mgorny@gentoo.org> > > wrote: > > > On Thu, 19 Apr 2012 17:31:11 +0200 > > > Corentin Chary <corentin.chary@gmail.com> wrote: > > > > > >> - <!ATTLIST remote-id type > > >> (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) > > >> #REQUIRED> > > >> + <!ATTLIST remote-id type > > >> (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gitorious|pecl|pear|bitbucket) > > >> #REQUIRED> > > > > > > Wouldn't it be better if we kept them sorted? > > > > There were not, so I just appended them, but I can provide another > > patch sorting them too if preferred. > > I'll appreciate if you could do that, along with that cpan-module > addition, and if noone objects, I'll just commit that. > > -- > Best regards, > Michał Górny --- a/metadata/dtd/metadata.dtd 2010-03-02 18:52:11.000000000 +0100 +++ b/metadata/dtd/metadata.dtd 2012-04-21 13:09:52.557223402 +0200 @@ -61,7 +61,7 @@ <!ELEMENT bugs-to (#PCDATA)> <!-- specify a type of package identification tracker --> <!ELEMENT remote-id (#PCDATA)> - <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) #REQUIRED> + <!ATTLIST remote-id type (bitbucket|cpan|cpan-module|cran|ctan|freshmeat|github|gitorious|google-code|launchpad|pear|pecl|pypi|rubyforge|rubygems|sourceforge|sourceforge-jp|vim) #REQUIRED> <!-- category/package information for cross-linking in descriptions and useflag descriptions --> -- Corentin Chary http://xf.iksaif.net/ [-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-19 15:31 [gentoo-dev] RFC: Add new remote-id types in metadata.dtd Corentin Chary 2012-04-19 16:54 ` Michał Górny @ 2012-04-19 18:08 ` "Paweł Hajdan, Jr." 2012-04-19 18:57 ` Michał Górny 2012-04-19 20:00 ` Robin H. Johnson ` (3 subsequent siblings) 5 siblings, 1 reply; 27+ messages in thread From: "Paweł Hajdan, Jr." @ 2012-04-19 18:08 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 334 bytes --] On 4/19/12 5:31 PM, Corentin Chary wrote: > Add rubygems, github, gitorious, pecl, pear, bitbucket. > All of them are handled by my remoteids.py script. Just making sure: do github, gitorious and bitbucket provide file hosting? I know they host repos, but for most ebuilds where remote-id would be useful, we need tarballs. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 203 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-19 18:08 ` "Paweł Hajdan, Jr." @ 2012-04-19 18:57 ` Michał Górny 0 siblings, 0 replies; 27+ messages in thread From: Michał Górny @ 2012-04-19 18:57 UTC (permalink / raw To: gentoo-dev; +Cc: phajdan.jr [-- Attachment #1: Type: text/plain, Size: 494 bytes --] On Thu, 19 Apr 2012 20:08:09 +0200 ""Paweł Hajdan, Jr."" <phajdan.jr@gentoo.org> wrote: > On 4/19/12 5:31 PM, Corentin Chary wrote: > > Add rubygems, github, gitorious, pecl, pear, bitbucket. > > All of them are handled by my remoteids.py script. > > Just making sure: do github, gitorious and bitbucket provide file > hosting? I know they host repos, but for most ebuilds where remote-id > would be useful, we need tarballs. Yes, they do. -- Best regards, Michał Górny [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 316 bytes --] ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-19 15:31 [gentoo-dev] RFC: Add new remote-id types in metadata.dtd Corentin Chary 2012-04-19 16:54 ` Michał Górny 2012-04-19 18:08 ` "Paweł Hajdan, Jr." @ 2012-04-19 20:00 ` Robin H. Johnson 2012-04-20 7:37 ` Kent Fredric ` (2 subsequent siblings) 5 siblings, 0 replies; 27+ messages in thread From: Robin H. Johnson @ 2012-04-19 20:00 UTC (permalink / raw To: gentoo-dev On Thu, Apr 19, 2012 at 05:31:11PM +0200, Corentin Chary wrote: > Add rubygems, github, gitorious, pecl, pear, bitbucket. > All of them are handled by my remoteids.py script. Can we get Launchpad added as well? -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robbat2@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-19 15:31 [gentoo-dev] RFC: Add new remote-id types in metadata.dtd Corentin Chary ` (2 preceding siblings ...) 2012-04-19 20:00 ` Robin H. Johnson @ 2012-04-20 7:37 ` Kent Fredric 2012-04-20 7:46 ` Corentin Chary 2012-05-22 14:16 ` [gentoo-dev] " Michael Weber 2012-06-02 17:30 ` Michael Weber 5 siblings, 1 reply; 27+ messages in thread From: Kent Fredric @ 2012-04-20 7:37 UTC (permalink / raw To: gentoo-dev On 20 April 2012 03:31, Corentin Chary <corentin.chary@gmail.com> wrote: > Add rubygems, github, gitorious, pecl, pear, bitbucket. > All of them are handled by my remoteids.py script. > > ref: https://bugs.gentoo.org/show_bug.cgi?id=406287 > ref: https://github.com/iksaif/portage-janitor/blob/master/remoteids.py > > --- a/metadata/dtd/metadata.dtd 2010-03-02 18:52:11.000000000 +0100 > +++ b/metadata/dtd/metadata.dtd 2012-04-19 14:22:14.077954310 +0200 > @@ -61,7 +61,7 @@ > <!ELEMENT bugs-to (#PCDATA)> > <!-- specify a type of package identification tracker --> > <!ELEMENT remote-id (#PCDATA)> > - <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) #REQUIRED> > + <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gitorious|pecl|pear|bitbucket) #REQUIRED> > > <!-- category/package information for cross-linking in descriptions > and useflag descriptions --> > > -- > Corentin Chary > http://xf.iksaif.net/ I suggested last week on #gentoo-perl that it might be nice to have 'cpan' and 'cpan-module' ( or something like that ) to disambiguate 2 queryable terms. ( where 'cpan' => 'the package name on cpan' ) For some purposes, its most convenient to use the distribution name, and for other purposes, (ie: cpan clients) its more convenient to use a Module name, and its not easy to translate between the two, as Module names sometimes switch between packages they're shipped in. For instance, a while ago, the BioPerl module was shipped in a distribution 'bioperl' , which has only recently been changed to BioPerl http://api.metacpan.org/release/_search?q=distribution:bioperl&fields=archive,author,date,download_url http://api.metacpan.org/release/_search?q=distribution:BioPerl&fields=archive,author,date,download_url vs http://api.metacpan.org/module/_search?q=module.name:Bio\:\:Perl&fields=distribution,author,release -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-20 7:37 ` Kent Fredric @ 2012-04-20 7:46 ` Corentin Chary 2012-04-20 8:26 ` Kent Fredric 0 siblings, 1 reply; 27+ messages in thread From: Corentin Chary @ 2012-04-20 7:46 UTC (permalink / raw To: gentoo-dev On Fri, Apr 20, 2012 at 9:37 AM, Kent Fredric <kentfredric@gmail.com> wrote: > On 20 April 2012 03:31, Corentin Chary <corentin.chary@gmail.com> wrote: >> Add rubygems, github, gitorious, pecl, pear, bitbucket. >> All of them are handled by my remoteids.py script. >> >> ref: https://bugs.gentoo.org/show_bug.cgi?id=406287 >> ref: https://github.com/iksaif/portage-janitor/blob/master/remoteids.py >> >> --- a/metadata/dtd/metadata.dtd 2010-03-02 18:52:11.000000000 +0100 >> +++ b/metadata/dtd/metadata.dtd 2012-04-19 14:22:14.077954310 +0200 >> @@ -61,7 +61,7 @@ >> <!ELEMENT bugs-to (#PCDATA)> >> <!-- specify a type of package identification tracker --> >> <!ELEMENT remote-id (#PCDATA)> >> - <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) #REQUIRED> >> + <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gitorious|pecl|pear|bitbucket) #REQUIRED> >> >> <!-- category/package information for cross-linking in descriptions >> and useflag descriptions --> >> >> -- >> Corentin Chary >> http://xf.iksaif.net/ > > > I suggested last week on #gentoo-perl that it might be nice to have > 'cpan' and 'cpan-module' ( or something like that ) to disambiguate 2 > queryable terms. ( where 'cpan' => 'the package name on cpan' ) > > For some purposes, its most convenient to use the distribution name, > and for other purposes, (ie: cpan clients) its more convenient to use > a Module name, and its not easy to translate between the two, as > Module names sometimes switch between packages they're shipped in. > > For instance, a while ago, the BioPerl module was shipped in a > distribution 'bioperl' , which has only recently been changed to > BioPerl > > > http://api.metacpan.org/release/_search?q=distribution:bioperl&fields=archive,author,date,download_url > > http://api.metacpan.org/release/_search?q=distribution:BioPerl&fields=archive,author,date,download_url > > vs > > > http://api.metacpan.org/module/_search?q=module.name:Bio\:\:Perl&fields=distribution,author,release Looks sane since the goal of remote-id is being able to identify the package upstream. Do you think you could patch remotesid.py to generate tags for cpan / cpan-modules ? Or at least give me a pseudo-algo that does the trick. Thanks :) -- Corentin Chary http://xf.iksaif.net ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-20 7:46 ` Corentin Chary @ 2012-04-20 8:26 ` Kent Fredric 2012-04-20 11:21 ` Corentin Chary 0 siblings, 1 reply; 27+ messages in thread From: Kent Fredric @ 2012-04-20 8:26 UTC (permalink / raw To: gentoo-dev; +Cc: perl On 20 April 2012 19:46, Corentin Chary <corentin.chary@gmail.com> wrote: > On Fri, Apr 20, 2012 at 9:37 AM, Kent Fredric <kentfredric@gmail.com> wrote: >> On 20 April 2012 03:31, Corentin Chary <corentin.chary@gmail.com> wrote: >>> Add rubygems, github, gitorious, pecl, pear, bitbucket. >>> All of them are handled by my remoteids.py script. >>> >>> ref: https://bugs.gentoo.org/show_bug.cgi?id=406287 >>> ref: https://github.com/iksaif/portage-janitor/blob/master/remoteids.py >>> >>> --- a/metadata/dtd/metadata.dtd 2010-03-02 18:52:11.000000000 +0100 >>> +++ b/metadata/dtd/metadata.dtd 2012-04-19 14:22:14.077954310 +0200 >>> @@ -61,7 +61,7 @@ >>> <!ELEMENT bugs-to (#PCDATA)> >>> <!-- specify a type of package identification tracker --> >>> <!ELEMENT remote-id (#PCDATA)> >>> - <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) #REQUIRED> >>> + <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gitorious|pecl|pear|bitbucket) #REQUIRED> >>> >>> <!-- category/package information for cross-linking in descriptions >>> and useflag descriptions --> >>> >>> -- >>> Corentin Chary >>> http://xf.iksaif.net/ >> >> >> I suggested last week on #gentoo-perl that it might be nice to have >> 'cpan' and 'cpan-module' ( or something like that ) to disambiguate 2 >> queryable terms. ( where 'cpan' => 'the package name on cpan' ) >> >> For some purposes, its most convenient to use the distribution name, >> and for other purposes, (ie: cpan clients) its more convenient to use >> a Module name, and its not easy to translate between the two, as >> Module names sometimes switch between packages they're shipped in. >> >> For instance, a while ago, the BioPerl module was shipped in a >> distribution 'bioperl' , which has only recently been changed to >> BioPerl >> >> >> http://api.metacpan.org/release/_search?q=distribution:bioperl&fields=archive,author,date,download_url >> >> http://api.metacpan.org/release/_search?q=distribution:BioPerl&fields=archive,author,date,download_url >> >> vs >> >> >> http://api.metacpan.org/module/_search?q=module.name:Bio\:\:Perl&fields=distribution,author,release > > Looks sane since the goal of remote-id is being able to identify the > package upstream. > Do you think you could patch remotesid.py to generate tags for cpan / > cpan-modules ? Or at least give me a pseudo-algo that does the trick. > Thanks :) > > -- > Corentin Chary > http://xf.iksaif.net > That is sadly not straight forward. Extracting the package name can be straight forward if you have the URL, because the package name is literally the same as the archive name in SRC_URI , sans version information. However, if you look at many perl ebuilds, you'll notice many lack this field and we've got other things in place, so the current parsing technique you use to detect uses of SRC_URI wont work there ( I could be wrong, I don't fully grok your python code ) And more-over, determining the value of 'cpan-module' may be impossible without access to the tar.gz itself, or querying the MetaCPAN API. Usually, upstream are sensible and have package names which closely correspond with the module names, ie: "Dist::Zilla" is shipped in 'Dist-Zilla-$VERSION.tar.gz', but there are many packages which dont do this, such as this notable example: https://metacpan.org/release/Scalar-List-Utils , which has no modules corresponding to the package name, and no way to divine the/a 'main' module from the package itself. ( and this is exacerbated by packages changing names, or package joins ( 2 packages becoming 1 via releasing modules together ), and package splits ( 1 package rips into 2 sets of modules ). Essentially, using a cpan-module as an identifier is somewhat "forwards only" , and even then, what it will resolve to is governed by time. This is fine for CPAN clients, which do the resolution hot, using the whole of CPAN as their data, if a user asks for "Foo::Bar", their cpan client will ask a cpan server ( or regularly (hourly) updated list ) as to what package that module can be found in ( and this only returns the most recent package, so name changes and so-forth are invisible to the user ). And being helpful to CPAN clients is one of the reasons we want this value as a specifiable option in the first place. For us, its easier to track the package name, and then when that has to change we can manually resolve the issue -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-20 8:26 ` Kent Fredric @ 2012-04-20 11:21 ` Corentin Chary 2012-04-20 12:39 ` Kent Fredric 0 siblings, 1 reply; 27+ messages in thread From: Corentin Chary @ 2012-04-20 11:21 UTC (permalink / raw To: gentoo-dev; +Cc: perl On Fri, Apr 20, 2012 at 10:26 AM, Kent Fredric <kentfredric@gmail.com> wrote: > On 20 April 2012 19:46, Corentin Chary <corentin.chary@gmail.com> wrote: >> On Fri, Apr 20, 2012 at 9:37 AM, Kent Fredric <kentfredric@gmail.com> wrote: >>> On 20 April 2012 03:31, Corentin Chary <corentin.chary@gmail.com> wrote: >>>> Add rubygems, github, gitorious, pecl, pear, bitbucket. >>>> All of them are handled by my remoteids.py script. >>>> >>>> ref: https://bugs.gentoo.org/show_bug.cgi?id=406287 >>>> ref: https://github.com/iksaif/portage-janitor/blob/master/remoteids.py >>>> >>>> --- a/metadata/dtd/metadata.dtd 2010-03-02 18:52:11.000000000 +0100 >>>> +++ b/metadata/dtd/metadata.dtd 2012-04-19 14:22:14.077954310 +0200 >>>> @@ -61,7 +61,7 @@ >>>> <!ELEMENT bugs-to (#PCDATA)> >>>> <!-- specify a type of package identification tracker --> >>>> <!ELEMENT remote-id (#PCDATA)> >>>> - <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) #REQUIRED> >>>> + <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gitorious|pecl|pear|bitbucket) #REQUIRED> >>>> >>>> <!-- category/package information for cross-linking in descriptions >>>> and useflag descriptions --> >>>> >>>> -- >>>> Corentin Chary >>>> http://xf.iksaif.net/ >>> >>> >>> I suggested last week on #gentoo-perl that it might be nice to have >>> 'cpan' and 'cpan-module' ( or something like that ) to disambiguate 2 >>> queryable terms. ( where 'cpan' => 'the package name on cpan' ) >>> >>> For some purposes, its most convenient to use the distribution name, >>> and for other purposes, (ie: cpan clients) its more convenient to use >>> a Module name, and its not easy to translate between the two, as >>> Module names sometimes switch between packages they're shipped in. >>> >>> For instance, a while ago, the BioPerl module was shipped in a >>> distribution 'bioperl' , which has only recently been changed to >>> BioPerl >>> >>> >>> http://api.metacpan.org/release/_search?q=distribution:bioperl&fields=archive,author,date,download_url >>> >>> http://api.metacpan.org/release/_search?q=distribution:BioPerl&fields=archive,author,date,download_url >>> >>> vs >>> >>> >>> http://api.metacpan.org/module/_search?q=module.name:Bio\:\:Perl&fields=distribution,author,release >> >> Looks sane since the goal of remote-id is being able to identify the >> package upstream. >> Do you think you could patch remotesid.py to generate tags for cpan / >> cpan-modules ? Or at least give me a pseudo-algo that does the trick. >> Thanks :) >> >> -- >> Corentin Chary >> http://xf.iksaif.net >> > > > That is sadly not straight forward. Extracting the package name can > be straight forward if you have the URL, because the package name is > literally the same as the archive name in SRC_URI , sans version > information. > > However, if you look at many perl ebuilds, you'll notice many lack > this field and we've got other things in place, so the current parsing > technique you use to detect uses of SRC_URI wont work there ( I could > be wrong, I don't fully grok your python code ) Currently it uses SRC_URI and HOMEPAGE, but honestly it wouldn't be hard to use any other environment variable and to do some checks on a webservice. Anyway for tricky cases it can still be done by hand. > And more-over, determining the value of 'cpan-module' may be > impossible without access to the tar.gz itself, or querying the > MetaCPAN API. > > Usually, upstream are sensible and have package names which closely > correspond with the module names, ie: "Dist::Zilla" is shipped in > 'Dist-Zilla-$VERSION.tar.gz', but there are many packages which dont > do this, such as this notable example: > https://metacpan.org/release/Scalar-List-Utils , which has no modules > corresponding to the package name, and no way to divine the/a 'main' > module from the package itself. ( and this is exacerbated by packages > changing names, or package joins ( 2 packages becoming 1 via releasing > modules together ), and package splits ( 1 package rips into 2 sets > of modules ). > > Essentially, using a cpan-module as an identifier is somewhat > "forwards only" , and even then, what it will resolve to is governed > by time. > > This is fine for CPAN clients, which do the resolution hot, using the > whole of CPAN as their data, if a user asks for "Foo::Bar", their cpan > client will ask a cpan server ( or regularly (hourly) updated list ) > as to what package that module can be found in ( and this only returns > the most recent package, so name changes and so-forth are invisible to > the user ). > > And being helpful to CPAN clients is one of the reasons we want this > value as a specifiable option in the first place. For us, its easier > to track the package name, and then when that has to change we can > manually resolve the issue > > -- > Kent > > perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, > 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" > > http://kent-fredric.fox.geek.nz > -- Corentin Chary http://xf.iksaif.net ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-20 11:21 ` Corentin Chary @ 2012-04-20 12:39 ` Kent Fredric 2012-04-20 13:34 ` Corentin Chary 0 siblings, 1 reply; 27+ messages in thread From: Kent Fredric @ 2012-04-20 12:39 UTC (permalink / raw To: gentoo-dev On 20 April 2012 23:21, Corentin Chary <corentin.chary@gmail.com> wrote: > > Currently it uses SRC_URI and HOMEPAGE, but honestly it wouldn't be > hard to use any other environment variable and to do some checks on a > webservice. > Anyway for tricky cases it can still be done by hand. > > -- > Corentin Chary > http://xf.iksaif.net > If you really want to support Perl Modules, ( which theres not much need for at present, looks like the team have gone through already for the most part and added remote-id's where possible already ), anything that inherits 'perl-module.eclass' has a bit of magic, in that neither SRC_URI or HOMEPAGE is required in the ebuild, and it just gets the package name from what gentoo is using. We've tried to be as close to upstream as possible for the ease of maintenance. However, there are still exception cases, for instance, BioPerl has to define 'MY_PN' to tell the perl-module eclass to use a different token ( and when this is present, it should be sufficient to say that that should be the remote-id instead of the package name: see dev-perl/Moose # an example with neither src_uri or homepage see sci-biology/bioperl # an example where the package name has been forced overridden as its changed upstream But resolving module names is much trickier, its easy-ish to map a module name to a package using the service, but doing it the other way round is not so straight forward, as one package can have many modules, and its common in perl to state dependencies in terms of the module to require, not the package its in, but there's also often a defacto "main module". But I'm myself still working out how to best do that with the service , so auto-populating a "cpan-module" identifier can be left to later, its just something I considered "useful" to have metadata wise because that value is more useful to users. -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-20 12:39 ` Kent Fredric @ 2012-04-20 13:34 ` Corentin Chary 2012-04-20 19:35 ` Kent Fredric 0 siblings, 1 reply; 27+ messages in thread From: Corentin Chary @ 2012-04-20 13:34 UTC (permalink / raw To: gentoo-dev On Sat, Apr 21, 2012 at 12:39:26AM +1200, Kent Fredric wrote: > If you really want to support Perl Modules, ( which theres not much > need for at present, looks like the team have gone through already for > the most part and added remote-id's where possible already ), anything > that inherits 'perl-module.eclass' has a bit of magic, in that neither > SRC_URI or HOMEPAGE is required in the ebuild, and it just gets the > package name from what gentoo is using. We've tried to be as close to > upstream as possible for the ease of maintenance. Actually the eclass seems to fill SRC_URI and HOMEPAGE, so when you source the ebuild they are available. > However, there are still exception cases, for instance, BioPerl has to > define 'MY_PN' to tell the perl-module eclass to use a different token > ( and when this is present, it should be sufficient to say that that > should be the remote-id instead of the package name: > see dev-perl/Moose # an example with neither src_uri or homepage > see sci-biology/bioperl # an example where the package name has > been forced overridden as its changed upstream > > But resolving module names is much trickier, its easy-ish to map a > module name to a package using the service, but doing it the other way > round is not so straight forward, as one package can have many > modules, and its common in perl to state dependencies in terms of the > module to require, not the package its in, but there's also often a > defacto "main module". > > But I'm myself still working out how to best do that with the service > , so auto-populating a "cpan-module" identifier can be left to later, > its just something I considered "useful" to have metadata wise because > that value is more useful to users. Yeah, not very important, but seems to work with this patch: https://github.com/iksaif/portage-janitor/commit/972aff94744741e34e99f917337430d245883c48 Example: $ python remoteids.py --diff WWW-Bugzilla Moose bioperl --- a/dev-perl/WWW-Bugzilla/metadata.xml +++ b/dev-perl/WWW-Bugzilla/metadata.xml @@ -3,6 +3,7 @@ <pkgmetadata> <herd>perl</herd> <upstream> + <remote-id type="cpan-module">WWW::Bugzilla</remote-id> <remote-id type="cpan">WWW-Bugzilla</remote-id> </upstream> </pkgmetadata> --- a/dev-perl/Moose/metadata.xml +++ b/dev-perl/Moose/metadata.xml @@ -3,6 +3,7 @@ <pkgmetadata> <herd>perl</herd> <upstream> + <remote-id type="cpan-module">Moose</remote-id> <remote-id type="cpan">Moose</remote-id> </upstream> </pkgmetadata> --- a/sci-biology/bioperl/metadata.xml +++ b/sci-biology/bioperl/metadata.xml @@ -8,6 +8,7 @@ <flag name="db">Install sci-biology/bioperl-run</flag> </use> <upstream> + <remote-id type="cpan-module">BioPerl</remote-id> <remote-id type="cpan">BioPerl</remote-id> </upstream> </pkgmetadata> -- Corentin Chary http://xf.iksaif.net/ ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-20 13:34 ` Corentin Chary @ 2012-04-20 19:35 ` Kent Fredric 2012-04-20 20:33 ` Corentin Chary 0 siblings, 1 reply; 27+ messages in thread From: Kent Fredric @ 2012-04-20 19:35 UTC (permalink / raw To: gentoo-dev On 21 April 2012 01:34, Corentin Chary <corentin.chary@gmail.com> wrote: > Yeah, not very important, but seems to work with this patch: > https://github.com/iksaif/portage-janitor/commit/972aff94744741e34e99f917337430d245883c48 > > Example: > $ python remoteids.py --diff WWW-Bugzilla Moose bioperl > --- a/dev-perl/WWW-Bugzilla/metadata.xml > +++ b/dev-perl/WWW-Bugzilla/metadata.xml > @@ -3,6 +3,7 @@ > <pkgmetadata> ... > -- > Corentin Chary > http://xf.iksaif.net/ > What does it to for say, Scalar-List-Utils ? or libwww-perl ? -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-20 19:35 ` Kent Fredric @ 2012-04-20 20:33 ` Corentin Chary 2012-04-21 3:33 ` Kent Fredric 0 siblings, 1 reply; 27+ messages in thread From: Corentin Chary @ 2012-04-20 20:33 UTC (permalink / raw To: gentoo-dev On Fri, Apr 20, 2012 at 9:35 PM, Kent Fredric <kentfredric@gmail.com> wrote: > On 21 April 2012 01:34, Corentin Chary <corentin.chary@gmail.com> wrote: >> Yeah, not very important, but seems to work with this patch: >> https://github.com/iksaif/portage-janitor/commit/972aff94744741e34e99f917337430d245883c48 >> >> Example: >> $ python remoteids.py --diff WWW-Bugzilla Moose bioperl >> --- a/dev-perl/WWW-Bugzilla/metadata.xml >> +++ b/dev-perl/WWW-Bugzilla/metadata.xml >> @@ -3,6 +3,7 @@ >> <pkgmetadata> > ... >> -- >> Corentin Chary >> http://xf.iksaif.net/ >> > > What does it to for say, Scalar-List-Utils ? or libwww-perl ? Nothing, it's one of the cases you'll have to handle by-hand. Another option is to parse the HTML from http://search.cpan.org/dist/Scalar-List-Utils/ but I'm not a fan of HTML-parsing. -- Corentin Chary http://xf.iksaif.net ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-20 20:33 ` Corentin Chary @ 2012-04-21 3:33 ` Kent Fredric 2012-04-21 9:18 ` Corentin Chary 0 siblings, 1 reply; 27+ messages in thread From: Kent Fredric @ 2012-04-21 3:33 UTC (permalink / raw To: gentoo-dev On 21 April 2012 08:33, Corentin Chary <corentin.chary@gmail.com> wrote: > On Fri, Apr 20, 2012 at 9:35 PM, Kent Fredric <kentfredric@gmail.com> wrote: >> On 21 April 2012 01:34, Corentin Chary <corentin.chary@gmail.com> wrote: >>> Yeah, not very important, but seems to work with this patch: >>> https://github.com/iksaif/portage-janitor/commit/972aff94744741e34e99f917337430d245883c48 >>> >>> Example: >>> $ python remoteids.py --diff WWW-Bugzilla Moose bioperl >>> --- a/dev-perl/WWW-Bugzilla/metadata.xml >>> +++ b/dev-perl/WWW-Bugzilla/metadata.xml >>> @@ -3,6 +3,7 @@ >>> <pkgmetadata> >> ... >>> -- >>> Corentin Chary >>> http://xf.iksaif.net/ >>> >> >> What does it to for say, Scalar-List-Utils ? or libwww-perl ? > > Nothing, it's one of the cases you'll have to handle by-hand. > Another option is to parse the HTML from > http://search.cpan.org/dist/Scalar-List-Utils/ but I'm not a fan of > HTML-parsing. > > -- > Corentin Chary > http://xf.iksaif.net > http://api.metacpan.org/release/Scalar-List-Utils Will be far better an option than parsing HTML =) And even better: curl -XPOST 'http://api.metacpan.org/module/_search' -d ' { "fields": [ "module.name", "release" ], "query": { "constant_score": { "filter" : { "and" : [ { "term": { "distribution":"Scalar-List-Utils" } }, { "term": { "status":"latest"} }, { "term": { "mime":"text/x-script.perl-module"}}, { "term": { "indexed":"true"}}, { "term": { "module.authorized":"true"}} ] } } } } ' -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-21 3:33 ` Kent Fredric @ 2012-04-21 9:18 ` Corentin Chary 2012-05-12 19:43 ` [gentoo-dev] " Torsten Veller 0 siblings, 1 reply; 27+ messages in thread From: Corentin Chary @ 2012-04-21 9:18 UTC (permalink / raw To: gentoo-dev On Sat, Apr 21, 2012 at 03:33:18PM +1200, Kent Fredric wrote: > On 21 April 2012 08:33, Corentin Chary <corentin.chary@gmail.com> wrote: > > On Fri, Apr 20, 2012 at 9:35 PM, Kent Fredric <kentfredric@gmail.com> wrote: > >> On 21 April 2012 01:34, Corentin Chary <corentin.chary@gmail.com> wrote: > >>> Yeah, not very important, but seems to work with this patch: > >>> https://github.com/iksaif/portage-janitor/commit/972aff94744741e34e99f917337430d245883c48 > > http://api.metacpan.org/release/Scalar-List-Utils > > Will be far better an option than parsing HTML =) > > And even better: > > curl -XPOST 'http://api.metacpan.org/module/_search' -d ' > { > "fields": [ > "module.name", > "release" > ], > "query": { > "constant_score": { > "filter" : { > "and" : [ > { "term": { "distribution":"Scalar-List-Utils" } }, > { "term": { "status":"latest"} }, > { "term": { "mime":"text/x-script.perl-module"}}, > { "term": { "indexed":"true"}}, > { "term": { "module.authorized":"true"}} > ] > } > } > } > } > ' > It is ! $ ./remoteids.py --diff WWW-Bugzilla Moose bioperl Scalar-List-Utils libwww-perl --- a/dev-perl/WWW-Bugzilla/metadata.xml +++ b/dev-perl/WWW-Bugzilla/metadata.xml @@ -3,6 +3,8 @@ <pkgmetadata> <herd>perl</herd> <upstream> + <remote-id type="cpan-module">WWW::Bugzilla::Search</remote-id> + <remote-id type="cpan-module">WWW::Bugzilla</remote-id> <remote-id type="cpan">WWW-Bugzilla</remote-id> </upstream> </pkgmetadata> --- a/dev-perl/Moose/metadata.xml +++ b/dev-perl/Moose/metadata.xml @@ -3,6 +3,17 @@ <pkgmetadata> <herd>perl</herd> <upstream> + <remote-id type="cpan-module">Class::MOP</remote-id> + <remote-id type="cpan-module">Moose::Meta::Attribute::Native::Trait::Counter</remote-id> + <remote-id type="cpan-module">Moose::Meta::Attribute::Native::Trait::String</remote-id> + <remote-id type="cpan-module">Moose::Meta::Method::Delegation</remote-id> + <remote-id type="cpan-module">Moose::Meta::TypeConstraint::Role</remote-id> + <remote-id type="cpan-module">Moose::Meta::Attribute::Custom::Moose</remote-id> + <remote-id type="cpan-module">Moose::Meta::Attribute</remote-id> + <remote-id type="cpan-module">Moose::Meta::Method</remote-id> + <remote-id type="cpan-module">Moose::Error::Croak</remote-id> + <remote-id type="cpan-module">Moose::Util::MetaRole</remote-id> + <remote-id type="cpan-module">Moose::Role</remote-id> <remote-id type="cpan">Moose</remote-id> </upstream> </pkgmetadata> --- a/perl-core/Scalar-List-Utils/metadata.xml +++ b/perl-core/Scalar-List-Utils/metadata.xml @@ -3,6 +3,9 @@ <pkgmetadata> <herd>perl</herd> <upstream> + <remote-id type="cpan-module">List::Util</remote-id> + <remote-id type="cpan-module">Scalar::Util</remote-id> + <remote-id type="cpan-module">List::Util::XS</remote-id> <remote-id type="cpan">Scalar-List-Utils</remote-id> </upstream> </pkgmetadata> --- a/dev-perl/libwww-perl/metadata.xml +++ b/dev-perl/libwww-perl/metadata.xml @@ -3,6 +3,17 @@ <pkgmetadata> <herd>perl</herd> <upstream> + <remote-id type="cpan-module">LWP::Protocol::data</remote-id> + <remote-id type="cpan-module">LWP::Protocol::gopher</remote-id> + <remote-id type="cpan-module">LWP::Debug</remote-id> + <remote-id type="cpan-module">LWP::UserAgent</remote-id> + <remote-id type="cpan-module">LWP::Authen::Digest</remote-id> + <remote-id type="cpan-module">LWP::Protocol::file</remote-id> + <remote-id type="cpan-module">LWP::Protocol::loopback</remote-id> + <remote-id type="cpan-module">LWP::Protocol::MyFTP</remote-id> + <remote-id type="cpan-module">LWP::Protocol::ftp</remote-id> + <remote-id type="cpan-module">LWP::Protocol::cpan</remote-id> + <remote-id type="cpan-module">LWP::MemberMixin</remote-id> <remote-id type="cpan">libwww-perl</remote-id> </upstream> </pkgmetadata> -- Corentin Chary http://xf.iksaif.net/ ^ permalink raw reply [flat|nested] 27+ messages in thread
* [gentoo-dev] Re: RFC: Add new remote-id types in metadata.dtd 2012-04-21 9:18 ` Corentin Chary @ 2012-05-12 19:43 ` Torsten Veller 2012-05-17 0:02 ` Kent Fredric 2012-05-17 0:05 ` [gentoo-dev] Re: RFC: Add new remote-id types in metadata.dtd Kent Fredric 0 siblings, 2 replies; 27+ messages in thread From: Torsten Veller @ 2012-05-12 19:43 UTC (permalink / raw To: gentoo-dev * Corentin Chary <corentin.chary@gmail.com>: > On Sat, Apr 21, 2012 at 03:33:18PM +1200, Kent Fredric wrote: > > { "term": { "status":"latest"} }, > > { "term": { "module.authorized":"true"}} What does this mean? - "latest"? this term looks like maintenance work. - what is "authorized"? It doesn't even list "Moose" for Moose? 02packages.details.txt.gz lists 72 package names for Moose-2.0602. > --- a/dev-perl/Moose/metadata.xml > +++ b/dev-perl/Moose/metadata.xml > @@ -3,6 +3,17 @@ > <pkgmetadata> > <herd>perl</herd> > <upstream> > + <remote-id type="cpan-module">Class::MOP</remote-id> > + <remote-id type="cpan-module">Moose::Meta::Attribute::Native::Trait::Counter</remote-id> > + <remote-id type="cpan-module">Moose::Meta::Attribute::Native::Trait::String</remote-id> > + <remote-id type="cpan-module">Moose::Meta::Method::Delegation</remote-id> > + <remote-id type="cpan-module">Moose::Meta::TypeConstraint::Role</remote-id> > + <remote-id type="cpan-module">Moose::Meta::Attribute::Custom::Moose</remote-id> > + <remote-id type="cpan-module">Moose::Meta::Attribute</remote-id> > + <remote-id type="cpan-module">Moose::Meta::Method</remote-id> > + <remote-id type="cpan-module">Moose::Error::Croak</remote-id> > + <remote-id type="cpan-module">Moose::Util::MetaRole</remote-id> > + <remote-id type="cpan-module">Moose::Role</remote-id> > <remote-id type="cpan">Moose</remote-id> > </upstream> > </pkgmetadata> ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] Re: RFC: Add new remote-id types in metadata.dtd 2012-05-12 19:43 ` [gentoo-dev] " Torsten Veller @ 2012-05-17 0:02 ` Kent Fredric 2012-05-18 6:46 ` Corentin Chary 2012-05-17 0:05 ` [gentoo-dev] Re: RFC: Add new remote-id types in metadata.dtd Kent Fredric 1 sibling, 1 reply; 27+ messages in thread From: Kent Fredric @ 2012-05-17 0:02 UTC (permalink / raw To: gentoo-dev On 13 May 2012 07:43, Torsten Veller <tove@gentoo.org> wrote: > * Corentin Chary <corentin.chary@gmail.com>: >> On Sat, Apr 21, 2012 at 03:33:18PM +1200, Kent Fredric wrote: >> > { "term": { "status":"latest"} }, >> > { "term": { "module.authorized":"true"}} > > What does this mean? > - "latest"? this term looks like maintenance work. > - what is "authorized"? "latest" means that it will fetch metadata for whatever is deemed the most recent non-dev release, which is really the only sane option to go for if you want a list of modules that currently pertain to the distribution. You could request *all* releases and then find a union of elements ... but that would be both erroneous and very time consuming. > It doesn't even list "Moose" for Moose? Its probably falling outside the initial 10 results, I forgot it did that, > 02packages.details.txt.gz lists 72 package names for Moose-2.0602. > Need to bolt on a { "size": 100 } to the query to expand how may results it will return. curl -XPOST 'http://api.metacpan.org/module/_search' -d ' { "fields": [ "module.name", "release" ], "query": { "constant_score": { "filter" : { "and" : [ { "term": { "distribution":"Moose" } }, { "term": { "status":"latest"} }, { "term": { "mime":"text/x-script.perl-module"}}, { "term": { "indexed":"true"}}, { "term": { "module.authorized":"true"}} ] } } }, "size": 100 } ^ that | grep module.name | wc -l # 83 -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] Re: RFC: Add new remote-id types in metadata.dtd 2012-05-17 0:02 ` Kent Fredric @ 2012-05-18 6:46 ` Corentin Chary 2012-06-02 17:11 ` [gentoo-dev] remote-id cpan-module Torsten Veller 0 siblings, 1 reply; 27+ messages in thread From: Corentin Chary @ 2012-05-18 6:46 UTC (permalink / raw To: gentoo-dev On Thu, May 17, 2012 at 2:02 AM, Kent Fredric <kentfredric@gmail.com> wrote: > On 13 May 2012 07:43, Torsten Veller <tove@gentoo.org> wrote: >> * Corentin Chary <corentin.chary@gmail.com>: >>> On Sat, Apr 21, 2012 at 03:33:18PM +1200, Kent Fredric wrote: >>> > { "term": { "status":"latest"} }, >>> > { "term": { "module.authorized":"true"}} >> >> What does this mean? >> - "latest"? this term looks like maintenance work. >> - what is "authorized"? > > "latest" means that it will fetch metadata for whatever is deemed the > most recent non-dev release, which is really the only sane option to > go for if you want a list of modules that currently pertain to the > distribution. You could request *all* releases and then find a union > of elements ... but that would be both erroneous and very time > consuming. > >> It doesn't even list "Moose" for Moose? > > Its probably falling outside the initial 10 results, I forgot it did that, > >> 02packages.details.txt.gz lists 72 package names for Moose-2.0602. >> > > Need to bolt on a { "size": 100 } to the query to expand how may > results it will return. Updated remotesid.py to use that, correctly add Moose in the diff now ! > curl -XPOST 'http://api.metacpan.org/module/_search' -d ' > { > "fields": [ > "module.name", > "release" > ], > "query": { > "constant_score": { > "filter" : { > "and" : [ > { "term": { "distribution":"Moose" } }, > { "term": { "status":"latest"} }, > { "term": { > "mime":"text/x-script.perl-module"}}, > { "term": { "indexed":"true"}}, > { "term": { "module.authorized":"true"}} > ] > > } > } > }, > "size": 100 > } > > ^ that | grep module.name | wc -l # 83 > > -- > Kent > > perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, > 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" > > http://kent-fredric.fox.geek.nz > ^ permalink raw reply [flat|nested] 27+ messages in thread
* [gentoo-dev] remote-id cpan-module 2012-05-18 6:46 ` Corentin Chary @ 2012-06-02 17:11 ` Torsten Veller 2012-06-13 12:07 ` Corentin Chary 0 siblings, 1 reply; 27+ messages in thread From: Torsten Veller @ 2012-06-02 17:11 UTC (permalink / raw To: gentoo-dev * Corentin Chary <iksaif@gentoo.org>: > On Thu, May 17, 2012 at 2:02 AM, Kent Fredric <kentfredric@gmail.com> wrote: > > On 13 May 2012 07:43, Torsten Veller <tove@gentoo.org> wrote: > >> It doesn't even list "Moose" for Moose? > > > > Its probably falling outside the initial 10 results, I forgot it did that, > > > >> 02packages.details.txt.gz lists 72 package names for Moose-2.0602. > >> > > > > Need to bolt on a { "size": 100 } to the query to expand how may > > results it will return. > > Updated remotesid.py to use that, correctly add Moose in the diff now ! metadata.dtd was updated per bug #406287 and it contains the cpan-module remote-id. The current patch for dev-perl/* is roughly 800k big: http://dev.gentoo.org/~tove/files/devperlremoteids.patch I am going to update the files in the next days. Now it would be a good time to voice your concerns. -- Thanks Torsten ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] remote-id cpan-module 2012-06-02 17:11 ` [gentoo-dev] remote-id cpan-module Torsten Veller @ 2012-06-13 12:07 ` Corentin Chary 0 siblings, 0 replies; 27+ messages in thread From: Corentin Chary @ 2012-06-13 12:07 UTC (permalink / raw To: gentoo-dev On Sat, Jun 2, 2012 at 7:11 PM, Torsten Veller <tove@gentoo.org> wrote: > * Corentin Chary <iksaif@gentoo.org>: >> On Thu, May 17, 2012 at 2:02 AM, Kent Fredric <kentfredric@gmail.com> wrote: >> > On 13 May 2012 07:43, Torsten Veller <tove@gentoo.org> wrote: >> >> It doesn't even list "Moose" for Moose? >> > >> > Its probably falling outside the initial 10 results, I forgot it did that, >> > >> >> 02packages.details.txt.gz lists 72 package names for Moose-2.0602. >> >> >> > >> > Need to bolt on a { "size": 100 } to the query to expand how may >> > results it will return. >> >> Updated remotesid.py to use that, correctly add Moose in the diff now ! > > metadata.dtd was updated per bug #406287 and it contains the cpan-module > remote-id. > > The current patch for dev-perl/* is roughly 800k big: > http://dev.gentoo.org/~tove/files/devperlremoteids.patch > > I am going to update the files in the next days. > Now it would be a good time to voice your concerns. > > -- > Thanks > Torsten > I pushed some fixes in remoteids.py, it now have the ability to remove dead remote-id. You should regenerate your patch, or use this one: http://xf.iksaif.net/dev/gentoo/remoteids/dev-perl.diff Note that I just made a small change that will make the remote-id sorted (which is great for cpan-module), if you want it, just regenerate a patch with: eix --only-names -C dev-perl | ./remoteids.py --diff --check-all Thanks, -- Corentin Chary ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] Re: RFC: Add new remote-id types in metadata.dtd 2012-05-12 19:43 ` [gentoo-dev] " Torsten Veller 2012-05-17 0:02 ` Kent Fredric @ 2012-05-17 0:05 ` Kent Fredric 1 sibling, 0 replies; 27+ messages in thread From: Kent Fredric @ 2012-05-17 0:05 UTC (permalink / raw To: gentoo-dev On 13 May 2012 07:43, Torsten Veller <tove@gentoo.org> wrote: > * Corentin Chary <corentin.chary@gmail.com>: >> On Sat, Apr 21, 2012 at 03:33:18PM +1200, Kent Fredric wrote: >> > { "term": { "status":"latest"} }, >> > { "term": { "module.authorized":"true"}} > > What does this mean? > - "latest"? this term looks like maintenance work. > - what is "authorized"? Oh, and authorized is just "Has this individual release been released by a person who has been granted permission to release that module", as per the PAUSE authentication system. ( To compare with indexed, which can be turned on/off by the releaser with the 'no index' flags , to get the authorization bit the module owner has to grant your PAUSE name rights ) -- Kent perl -e "print substr( \"edrgmaM SPA NOcomil.ic\\@tfrken\", \$_ * 3, 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );" http://kent-fredric.fox.geek.nz ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-19 15:31 [gentoo-dev] RFC: Add new remote-id types in metadata.dtd Corentin Chary ` (3 preceding siblings ...) 2012-04-20 7:37 ` Kent Fredric @ 2012-05-22 14:16 ` Michael Weber 2012-06-02 17:30 ` Michael Weber 5 siblings, 0 replies; 27+ messages in thread From: Michael Weber @ 2012-05-22 14:16 UTC (permalink / raw To: gentoo-dev -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Can we bump our gentoo-x86/skel.metadata.xml and app-vim/gentoo-syntax:/usr/share/vim/vimfiles/plugin/newmetadata.vim files to display some upstream+remote-id lines to make these tags more prominent? <upstream> <maintainer status="active"> <email></email> <name></name> </maintainer> <changelog></changelog> <doc lang="en"></doc> <bugs-to></bugs-to> <remote-id type=""></remote-id> </upstream> On 04/19/2012 05:31 PM, Corentin Chary wrote: > Add rubygems, github, gitorious, pecl, pear, bitbucket. All of them > are handled by my remoteids.py script. > > ref: https://bugs.gentoo.org/show_bug.cgi?id=406287 ref: > https://github.com/iksaif/portage-janitor/blob/master/remoteids.py > > --- a/metadata/dtd/metadata.dtd 2010-03-02 18:52:11.000000000 > +0100 +++ b/metadata/dtd/metadata.dtd 2012-04-19 14:22:14.077954310 > +0200 @@ -61,7 +61,7 @@ <!ELEMENT bugs-to (#PCDATA)> <!-- specify a > type of package identification tracker --> <!ELEMENT remote-id > (#PCDATA)> - <!ATTLIST remote-id type > (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) > #REQUIRED> + <!ATTLIST remote-id type > (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gitorious|pecl|pear|bitbucket) > #REQUIRED> > > <!-- category/package information for cross-linking in > descriptions and useflag descriptions --> > - -- - -- Gentoo Dev http://xmw.de/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk+7n6IACgkQknrdDGLu8JB0bwEAlh9AilEx700DVEwQYD2KJxtc nLrzXyCrZZLZLE6cpX8BAJIj05i+LN8ZLJOH3bAtcSp41YG4EarviiKTdEpy2Yon =CQje -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-04-19 15:31 [gentoo-dev] RFC: Add new remote-id types in metadata.dtd Corentin Chary ` (4 preceding siblings ...) 2012-05-22 14:16 ` [gentoo-dev] " Michael Weber @ 2012-06-02 17:30 ` Michael Weber 2012-06-04 8:14 ` Corentin Chary 5 siblings, 1 reply; 27+ messages in thread From: Michael Weber @ 2012-06-02 17:30 UTC (permalink / raw To: gentoo-dev -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Is there any way to verify the <remote-id> data? What programs/scripts use these fields btw? I could imagine a test like (i.e. <remote-id type="sourceforge">$a</remote-id> ) does http://www.fs.net/$a exist. Michael - -- Gentoo Dev http://xmw.de/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk/KTbwACgkQknrdDGLu8JC47wD/WhUl5SqDnd9SZM7xyYWE3NG6 GN/lt6/0J+7W070szVYA/00BA1r0jBA3i93FpvobgvFiKlZSTLPaN5sJCJrrqULR =Uufg -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [gentoo-dev] RFC: Add new remote-id types in metadata.dtd 2012-06-02 17:30 ` Michael Weber @ 2012-06-04 8:14 ` Corentin Chary 0 siblings, 0 replies; 27+ messages in thread From: Corentin Chary @ 2012-06-04 8:14 UTC (permalink / raw To: gentoo-dev On Sat, Jun 2, 2012 at 7:30 PM, Michael Weber <xmw@gentoo.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Is there any way to verify the <remote-id> data? > > What programs/scripts use these fields btw? euscan will soon, and I guess p.g.o will too. > I could imagine a test like (i.e. <remote-id > type="sourceforge">$a</remote-id> ) > does http://www.fs.net/$a exist. > > Michael That could probably be done in remoteids.py ^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2012-06-13 12:11 UTC | newest] Thread overview: 27+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-04-19 15:31 [gentoo-dev] RFC: Add new remote-id types in metadata.dtd Corentin Chary 2012-04-19 16:54 ` Michał Górny 2012-04-19 19:32 ` Corentin Chary 2012-04-21 9:32 ` Michał Górny 2012-04-21 11:38 ` Corentin Chary 2012-04-19 18:08 ` "Paweł Hajdan, Jr." 2012-04-19 18:57 ` Michał Górny 2012-04-19 20:00 ` Robin H. Johnson 2012-04-20 7:37 ` Kent Fredric 2012-04-20 7:46 ` Corentin Chary 2012-04-20 8:26 ` Kent Fredric 2012-04-20 11:21 ` Corentin Chary 2012-04-20 12:39 ` Kent Fredric 2012-04-20 13:34 ` Corentin Chary 2012-04-20 19:35 ` Kent Fredric 2012-04-20 20:33 ` Corentin Chary 2012-04-21 3:33 ` Kent Fredric 2012-04-21 9:18 ` Corentin Chary 2012-05-12 19:43 ` [gentoo-dev] " Torsten Veller 2012-05-17 0:02 ` Kent Fredric 2012-05-18 6:46 ` Corentin Chary 2012-06-02 17:11 ` [gentoo-dev] remote-id cpan-module Torsten Veller 2012-06-13 12:07 ` Corentin Chary 2012-05-17 0:05 ` [gentoo-dev] Re: RFC: Add new remote-id types in metadata.dtd Kent Fredric 2012-05-22 14:16 ` [gentoo-dev] " Michael Weber 2012-06-02 17:30 ` Michael Weber 2012-06-04 8:14 ` Corentin Chary
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox