[gentoo-soc] Multiple Repository Support in Portage - Week 3

[gentoo-soc] Multiple Repository Support in Portage - Week 3

[Otávio Pontes]

Hi,
Last week i was working with adding to portage support for emerging
packages from one specific repository.
Atom class was modified to support a new field, with repository name.
Now before package category it is possible to add a repository name
followed by '::'.
So emerge gentoo::app-portage/eix is the eix from repository gentoo.
Atom class is already working, but emerge is still ignoring this
repository name.
It was also necessary to change other places in portage because slot
separator is ':' and portage was getting confused when atom has '::'.
This week work will be to use this atom information in portdbapi and
in the dependency resolver to use ebuilds from the specified
repository.

Otávio

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Auke Booij]

Paludis actually puts the repository name after the package, for
example app-portage/eix::gentoo. Is there any specific reason you
decided to do it your way? I don't know how pkgcore does this, btw,
just interested to hear your reasoning.

tulcod.

2010/6/15 Otávio Pontes <otaviobp@gmail.com>:
> Hi,
> Last week i was working with adding to portage support for emerging
> packages from one specific repository.
> Atom class was modified to support a new field, with repository name.
> Now before package category it is possible to add a repository name
> followed by '::'.
> So emerge gentoo::app-portage/eix is the eix from repository gentoo.
> Atom class is already working, but emerge is still ignoring this
> repository name.
> It was also necessary to change other places in portage because slot
> separator is ':' and portage was getting confused when atom has '::'.
> This week work will be to use this atom information in portdbapi and
> in the dependency resolver to use ebuilds from the specified
> repository.
>
> Otávio
>
>

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]

On Tue, 2010-06-15 at 19:42 +0200, Auke Booij wrote:
> Paludis actually puts the repository name after the package, for
> example app-portage/eix::gentoo. Is there any specific reason you
> decided to do it your way? I don't know how pkgcore does this, btw,
> just interested to hear your reasoning.
> 
> tulcod.


Yes pkgcore is the same repo syntax as paludis.  So why mess with a
standard the other 2 package mangers have had for a few years already?
It'll be confusing as hell for users trying to figure out and keep
straight between PM's and forum postings saying one way, then another.
Also for someone like me coding a gui frontend it means having 2
different code blocks to do the same thing, one for each syntax style :(

> 
> 2010/6/15 Otávio Pontes <otaviobp@gmail.com>:
> > Hi,
> > Last week i was working with adding to portage support for emerging
> > packages from one specific repository.
> > Atom class was modified to support a new field, with repository name.
> > Now before package category it is possible to add a repository name
> > followed by '::'.
> > So emerge gentoo::app-portage/eix is the eix from repository gentoo.
> > Atom class is already working, but emerge is still ignoring this
> > repository name.
> > It was also necessary to change other places in portage because slot
> > separator is ':' and portage was getting confused when atom has '::'.
> > This week work will be to use this atom information in portdbapi and
> > in the dependency resolver to use ebuilds from the specified
> > repository.
> >
> > Otávio
> >
> >
> 


-- 
Brian Dolbec <brian.dolbec@gmail.com>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Otávio Pontes]

On Tue, Jun 15, 2010 at 19:11, Brian Dolbec <brian.dolbec@gmail.com> wrote:
> On Tue, 2010-06-15 at 19:42 +0200, Auke Booij wrote:
>> Paludis actually puts the repository name after the package, for
>> example app-portage/eix::gentoo. Is there any specific reason you
>> decided to do it your way? I don't know how pkgcore does this, btw,
>> just interested to hear your reasoning.

There isn't a specific reason i did this way. Actually, i have read
the Paludis docs and copied the '::' from they. But i misunderstood.
I will change it to fit the paludis and pkgcore way.
Thanks for the hint.

Otávio Pontes

>>
>> tulcod.
>
>
> Yes pkgcore is the same repo syntax as paludis.  So why mess with a
> standard the other 2 package mangers have had for a few years already?
> It'll be confusing as hell for users trying to figure out and keep
> straight between PM's and forum postings saying one way, then another.
> Also for someone like me coding a gui frontend it means having 2
> different code blocks to do the same thing, one for each syntax style :(
>
>>
>> 2010/6/15 Otávio Pontes <otaviobp@gmail.com>:
>> > Hi,
>> > Last week i was working with adding to portage support for emerging
>> > packages from one specific repository.
>> > Atom class was modified to support a new field, with repository name.
>> > Now before package category it is possible to add a repository name
>> > followed by '::'.
>> > So emerge gentoo::app-portage/eix is the eix from repository gentoo.
>> > Atom class is already working, but emerge is still ignoring this
>> > repository name.
>> > It was also necessary to change other places in portage because slot
>> > separator is ':' and portage was getting confused when atom has '::'.
>> > This week work will be to use this atom information in portdbapi and
>> > in the dependency resolver to use ebuilds from the specified
>> > repository.
>> >
>> > Otávio
>> >
>> >
>>
>
>
> --
> Brian Dolbec <brian.dolbec@gmail.com>
>

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Otávio Pontes]

I couldn't find in paludis documentation (nor in pkgcore) where should
go repository name when using useflags to emerge a package. In current
Portage you can use the Atom category/package-version:slot[useflags].
Should i place repository name after or before useflags? How is done
in paludis pkgcore? category/package-version:slot[useflags]::repo_name
or category/package-version:slot::repo_name[useflags]?

Thanks,
Otávio Pontes

2010/6/15 Otávio Pontes <otaviobp@gmail.com>:
> On Tue, Jun 15, 2010 at 19:11, Brian Dolbec <brian.dolbec@gmail.com> wrote:
>> On Tue, 2010-06-15 at 19:42 +0200, Auke Booij wrote:
>>> Paludis actually puts the repository name after the package, for
>>> example app-portage/eix::gentoo. Is there any specific reason you
>>> decided to do it your way? I don't know how pkgcore does this, btw,
>>> just interested to hear your reasoning.
>
> There isn't a specific reason i did this way. Actually, i have read
> the Paludis docs and copied the '::' from they. But i misunderstood.
> I will change it to fit the paludis and pkgcore way.
> Thanks for the hint.
>
> Otávio Pontes
>
>>>
>>> tulcod.
>>
>>
>> Yes pkgcore is the same repo syntax as paludis.  So why mess with a
>> standard the other 2 package mangers have had for a few years already?
>> It'll be confusing as hell for users trying to figure out and keep
>> straight between PM's and forum postings saying one way, then another.
>> Also for someone like me coding a gui frontend it means having 2
>> different code blocks to do the same thing, one for each syntax style :(
>>
>>>
>>> 2010/6/15 Otávio Pontes <otaviobp@gmail.com>:
>>> > Hi,
>>> > Last week i was working with adding to portage support for emerging
>>> > packages from one specific repository.
>>> > Atom class was modified to support a new field, with repository name.
>>> > Now before package category it is possible to add a repository name
>>> > followed by '::'.
>>> > So emerge gentoo::app-portage/eix is the eix from repository gentoo.
>>> > Atom class is already working, but emerge is still ignoring this
>>> > repository name.
>>> > It was also necessary to change other places in portage because slot
>>> > separator is ':' and portage was getting confused when atom has '::'.
>>> > This week work will be to use this atom information in portdbapi and
>>> > in the dependency resolver to use ebuilds from the specified
>>> > repository.
>>> >
>>> > Otávio
>>> >
>>> >
>>>
>>
>>
>> --
>> Brian Dolbec <brian.dolbec@gmail.com>
>>
>

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Auke Booij]

2010/6/16 Otávio Pontes <otaviobp@gmail.com>:
> I couldn't find in paludis documentation (nor in pkgcore) where should
> go repository name when using useflags to emerge a package.

I don't think it's considered "right" to depend on a /repository/.
Instead, you depend on packages, versions, slots and useflags, but if
you really want to be able to do this I guess you should speak with
the PM devs.

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Nirbheek Chauhan]

On Wed, Jun 16, 2010 at 9:17 PM, Auke Booij <auke@tulcod.com> wrote:
> 2010/6/16 Otávio Pontes <otaviobp@gmail.com>:
>> I couldn't find in paludis documentation (nor in pkgcore) where should
>> go repository name when using useflags to emerge a package.
>
> I don't think it's considered "right" to depend on a /repository/.
> Instead, you depend on packages, versions, slots and useflags, but if
> you really want to be able to do this I guess you should speak with
> the PM devs.
>

I agree with this. I believe that the primary usage was for
command-line specification, and perhaps per-repo mask-reverting in
package.mask, etc. Definitely not for usage in the gentoo-x86 tree,
and *very strongly* discouraged for overlays.


-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Zac Medico]

On 06/16/2010 07:36 AM, Otávio Pontes wrote:
> I couldn't find in paludis documentation (nor in pkgcore) where should
> go repository name when using useflags to emerge a package. In current
> Portage you can use the Atom category/package-version:slot[useflags].
> Should i place repository name after or before useflags? How is done
> in paludis pkgcore? category/package-version:slot[useflags]::repo_name
> or category/package-version:slot::repo_name[useflags]?

I'd say that it should be furthest to the right, as in
category/package-version:slot[useflags]::repo_name. You can add
support to portage.dep.Atom if you want. We'll just have to ensure
that the package dependency parsing code will reject such atoms as
invalid.
-- 
Thanks,
Zac

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

On Wed, Jun 16, 2010 at 05:47:42PM +0200, Auke Booij wrote:
> 2010/6/16 Otávio Pontes <otaviobp@gmail.com>:
> > I couldn't find in paludis documentation (nor in pkgcore) where should
> > go repository name when using useflags to emerge a package.
> 
> I don't think it's considered "right" to depend on a /repository/.
> Instead, you depend on packages, versions, slots and useflags, but if
> you really want to be able to do this I guess you should speak with
> the PM devs.

It's actually alright- both from the commandline, and ebuild's 
specifying explicit repository deps.  Ebuild's rarely should be 
doing this in my opinion, but there are instants where it's 
necessary in my experience.  Basically, leave the support there, if 
people really care they get to add some marker to the repository 
blocking usage of repo deps.

As for syntax, it's cat/pkg:slotting-if-any:repo[flag-bits]

So for specifying a slot and a repo, and disabling build, 
sys-apps/portage:0:gentoo[-build]

If you don't care about the slot, you leave it empty- 
sys-apps/portage::gentoo[-build]

CC'ing ciaran either way- in testing paludis 0.46, 
sys-apps/portage:0:gentoo isn't flying... (the use ordering however 
does).

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 2458 bytes --]

On Sat, Jun 19, 2010 at 12:06:15PM +0100, Ciaran McCreesh wrote:
> I'm not subscribed to the SOC list, so if this doesn't make it through,
> please repost...
> 
> On Fri, 18 Jun 2010 20:48:54 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> > On Wed, Jun 16, 2010 at 05:47:42PM +0200, Auke Booij wrote:
> > > 2010/6/16 Otávio Pontes <otaviobp@gmail.com>:
> > > > I couldn't find in paludis documentation (nor in pkgcore) where
> > > > should go repository name when using useflags to emerge a package.
> 
> http://paludis.pioto.org/configuration/specs.html is what Paludis
> supports for user specs.
> 
> > As for syntax, it's cat/pkg:slotting-if-any:repo[flag-bits]
> > 
> > So for specifying a slot and a repo, and disabling build, 
> > sys-apps/portage:0:gentoo[-build]
> > 
> > If you don't care about the slot, you leave it empty- 
> > sys-apps/portage::gentoo[-build]
> > 
> > CC'ing ciaran either way- in testing paludis 0.46, 
> > sys-apps/portage:0:gentoo isn't flying... (the use ordering however 
> > does).
> 
> ::gentoo not :gentoo (and only in contexts where it's a 'user' dep
> spec, not one covered by an EAPI).


Crapola.  Didn't notice that paludis requires doubling up there- 
specifically, for pkgcore

sys-apps/portage
sys-apps/portage:0
sys-apps/portage:0:gentoo
sys-apps/portage::gentoo
sys-apps/portage:0:gentoo[-build]
sys-apps/portage::gentoo[-build]

From pkgcore's view, the 1st ':' is required- if there is no slot dep, 
leave it empty.  The second demarks the start of a repository dep.

For paludis, it's based on consecutive ':' ; so literally the above is
sys-apps/portage
sys-apps/portage:0
sys-apps/portage:0::gentoo
sys-apps/portage::gentoo
sys-apps/portage:0::gentoo[-build]
sys-apps/portage::gentoo[-build]


People's feedback is kind of needed on that one due to the diff. in 
syntax...


> Also note that that specifically only covers packages that are *in*
> the ::gentoo repository. The package you have installed *from* the
> gentoo repository is ::gentoo->/ or ::gentoo->installed .

'->' doesn't work very well though; specifically, the '-' is allowed 
as part of a repository name.  Backtracking parsers can handle that 
fine, it's just nonoptimal to have one of the splitter char's be part 
of the allowed character set for a name.

@, or some other char that is explicitly disallow seems a better 
choice.

Comments?

thanks-
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1882 bytes --]

On Sat, 19 Jun 2010 13:18:21 -0700
Brian Harring <ferringb@gmail.com> wrote:
> From pkgcore's view, the 1st ':' is required- if there is no slot
> dep, leave it empty.  The second demarks the start of a repository
> dep.

I prefer the "always ::" form, simply because it's easier to go "if
repo, then add ::repo" than it is to go "if repo, then add :repo, but
also stick another : in there if we've not done a slot". What a repo
dep looks like shouldn't change based upon whether there's also a slot
dep there.

> > Also note that that specifically only covers packages that are *in*
> > the ::gentoo repository. The package you have installed *from* the
> > gentoo repository is ::gentoo->/ or ::gentoo->installed .
> 
> '->' doesn't work very well though; specifically, the '-' is allowed 
> as part of a repository name.  Backtracking parsers can handle that 
> fine, it's just nonoptimal to have one of the splitter char's be part 
> of the allowed character set for a name.

Eh, I'd hardly take "your parser must support a single character
lookahead" to be a defining argument, especially when you consider the
mess that's needed to split pkg-ver up.

> @, or some other char that is explicitly disallow seems a better 
> choice.

The rationale behind -> is that it reads clearly. ::from->to reads like
what it does. Experience has shown that users do need and make use of
both forms (although unfortunately they're less inclined to appreciate
the difference between ::installed and ::/ ), so it's best to pick
something they can eventually understand.

On a semi-related note: real world experience has also shown that every
time we've thought about using ::repo deps of any kind in a repository
rather than a user context, it's been the wrong thing to do, and the
right thing to do is almost always [feature(-)].

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1850 bytes --]

On Sat, Jun 19, 2010 at 10:37:30PM +0100, Ciaran McCreesh wrote:
> On Sat, 19 Jun 2010 13:18:21 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> > From pkgcore's view, the 1st ':' is required- if there is no slot
> > dep, leave it empty.  The second demarks the start of a repository
> > dep.
> 
> I prefer the "always ::" form, simply because it's easier to go "if
> repo, then add ::repo" than it is to go "if repo, then add :repo, but
> also stick another : in there if we've not done a slot". What a repo
> dep looks like shouldn't change based upon whether there's also a slot
> dep there.

Arguable, but I'll convert pkgcore to the paludis syntax.  Next 
release will have it.


> > @, or some other char that is explicitly disallow seems a better 
> > choice.
> 
> The rationale behind -> is that it reads clearly. ::from->to reads like
> what it does. Experience has shown that users do need and make use of
> both forms (although unfortunately they're less inclined to appreciate
> the difference between ::installed and ::/ ), so it's best to pick
> something they can eventually understand.

@ carries a similar meaning however; give me 
pkg::src_repo@target_repo .  Reads a bit better imo (I want pkg from 
src_repo that is in target_repo), and also isn't composed of valid 
repository name chars.

Other's opinions on that one would be useful.


> On a semi-related note: real world experience has also shown that every
> time we've thought about using ::repo deps of any kind in a repository
> rather than a user context, it's been the wrong thing to do, and the
> right thing to do is almost always [feature(-)].

Block it at the repo policy level, rather than at the ebuild level- 
you get the same end result, but leave open a door for people who 
fall outside your usecases.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-soc] Multiple Repository Support in Portage - Week 3

[Otávio Pontes]

On Sat, Jun 19, 2010 at 17:18, Brian Harring <ferringb@gmail.com> wrote:
> On Sat, Jun 19, 2010 at 12:06:15PM +0100, Ciaran McCreesh wrote:
>> I'm not subscribed to the SOC list, so if this doesn't make it through,
>> please repost...
>>
>> On Fri, 18 Jun 2010 20:48:54 -0700
>> Brian Harring <ferringb@gmail.com> wrote:
>> > On Wed, Jun 16, 2010 at 05:47:42PM +0200, Auke Booij wrote:
>> > > 2010/6/16 Otávio Pontes <otaviobp@gmail.com>:
>> > > > I couldn't find in paludis documentation (nor in pkgcore) where
>> > > > should go repository name when using useflags to emerge a package.
>>
>> http://paludis.pioto.org/configuration/specs.html is what Paludis
>> supports for user specs.
>>
>> > As for syntax, it's cat/pkg:slotting-if-any:repo[flag-bits]
>> >
>> > So for specifying a slot and a repo, and disabling build,
>> > sys-apps/portage:0:gentoo[-build]
>> >
>> > If you don't care about the slot, you leave it empty-
>> > sys-apps/portage::gentoo[-build]
>> >
>> > CC'ing ciaran either way- in testing paludis 0.46,
>> > sys-apps/portage:0:gentoo isn't flying... (the use ordering however
>> > does).
>>
>> ::gentoo not :gentoo (and only in contexts where it's a 'user' dep
>> spec, not one covered by an EAPI).
>
>
> Crapola.  Didn't notice that paludis requires doubling up there-
> specifically, for pkgcore
>
> sys-apps/portage
> sys-apps/portage:0
> sys-apps/portage:0:gentoo
> sys-apps/portage::gentoo
> sys-apps/portage:0:gentoo[-build]
> sys-apps/portage::gentoo[-build]
>
> From pkgcore's view, the 1st ':' is required- if there is no slot dep,
> leave it empty.  The second demarks the start of a repository dep.
>
> For paludis, it's based on consecutive ':' ; so literally the above is
> sys-apps/portage
> sys-apps/portage:0
> sys-apps/portage:0::gentoo
> sys-apps/portage::gentoo
> sys-apps/portage:0::gentoo[-build]
> sys-apps/portage::gentoo[-build]

I am doing exactly this way now. With '::', after slot and before useflags.

>
>
> People's feedback is kind of needed on that one due to the diff. in
> syntax...
>
>
>> Also note that that specifically only covers packages that are *in*
>> the ::gentoo repository. The package you have installed *from* the
>> gentoo repository is ::gentoo->/ or ::gentoo->installed .
>
> '->' doesn't work very well though; specifically, the '-' is allowed
> as part of a repository name.  Backtracking parsers can handle that
> fine, it's just nonoptimal to have one of the splitter char's be part
> of the allowed character set for a name.
>
> @, or some other char that is explicitly disallow seems a better
> choice.
>
> Comments?
>
> thanks-
> ~harring
>