[gentoo-portage-dev] few more portage-ng "feature proposals"

[gentoo-portage-dev] few more portage-ng "feature proposals"

[George Shapovalov]

Hi gang.

Here go a few more ideas I decided to throw in. 
I must mention that these are mostly reiterations of some points that were 
raised at different occasions in previous (and often old; yea, I really am 
doing too much of "hey this was discussed like a year ago" stuff lately :)) 
discussions of portage functionality/enhancements. Thus effectively this is 
going to be more of an overview than a new proposal. I did not seem to see 
them mentioned yet, but in case some of them are dups I apologise.
So, here goes:

1. Multilevel categories. For example:
dev-*  =>  dev/langs/{procedural/,functional/...}
                      lib/...
                      util/...
Support for arbitrary "deepness" is preferable.
The tree is getting bigger and bigger and this feature should keep it 
browsable for some time to come. Addition of similar category browser to 
http://packages.gentoo.org/ would be nice too.
(I couldn't get through to it for some reason now to check if its not there. 
So if it is I apologise :)).

2. Support for duplicate package names under different categories.
There was a (mildly) heated discussion some time ago on that one with may be 
only silghtly more people on the pro side. Having dealt with my portion of 
dups and having been in situation when already included package with the same 
name complicates the (naming) issue I arrived to conclusion that it is 
inevitable. The reality is that we already have couple of such dups and we 
have to deal with this anyway (btw, even with present portage things seem to 
work even if in a semisupported fashion).

These two "features" are connected, so I'll put here some common technical 
elaboration. The debate on "dups" one mostly concentrated around flat vs 
categorised namespace or, on the user level, search vs browse paradigm. 
Following the "Gentoo is about choice" line of thought I see only one 
possible decision here, - to provide both features :). The solution may be 
simple - treat the "category path" as a part of package name. For the search 
purposes make the search routines a bit more intelligent, making it to accept 
request for searches in "basename" only or any other part of the 
categorization.

What should happen when the dups are requested or encountered?
If during search - report all matches (just as it does it now anyway),
if during action request (e.g. emerge or unmerge), stop and ask for 
clarification.

One other argument for abandoning categorization was that it is hard to select 
"one true" category for a package in many cases as it really might belong to 
multiple. This immediately suggests the following:

3.  Add search keywords (or some other tags) to packages.
I believe this was considered for the extended metadata format in the future 
and even mentioned not so long ago (although I don't remember where and when 
and I don't see relevant GLEP, should I file one?). I am bringing this up 
largerly to complete "the picture" started in pp 1 and 2.

Taken together these three "enhancements" should make it possible to maintain 
both search and browsing ability for portage database even as it continues to 
grow.


Now last but not least for this email

4. Partial portage checkout.
This also was already touted. One of the variants was even recently proposed 
right here, in this list. I am mentioning this mostly to reiterate and remind 
other possible approaches to this.

4.1 "Thematic" checkouts.
Goes along the (proposed) work on "partitioning" the tree for particular 
tasks. The "corporate" theme was mentioned not so long ago (something along 
these lines had a "server" tag in discussions happened about 6-12 month ago I 
believe). As long as there is work on supporting a thematic tree "partition" 
and as long as it is in a reasonablt consistent state it should be possible 
to provide such thematic checkout functionality

4.2 "Stability" checkouts.
Similar partitioning, but based aroung the presumed stability of packages. 
This may follow the present arch/~arch divide or more levels as/if we 
introduce them. Apparently this should be inclusive, i.e. poeple who want 
"testing" stuff are goind\g to get "stable" as well.

These two seem like a lot of work, but in reality major part of this work is 
done in package maintainership and other regular activity (that would 
otherwise happen anyway). portage-ng would basically just need to provide 
some support functionality for that.

On the contrary the next one (the one that was proposed not so long ago) seems 
to require not as trivial extension to portage, as it would need some 
nontrivial infrastructure support.

4.3 "On demand" checkouts.
I cannot find it right now, but I remember this was proposed on this list just 
a bit over a week ago. Or was it on -dev? Anyway, the idea is to checkout 
some "base" collection by default and then do checkouts of more stuff as user 
requestes (may be indirectly as dependencies).

Oh, and we might consider getting all three (or more if they are requested) of 
them, they don't seem to be mutually exclusive :).

George


--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] few more portage-ng "feature proposals"

[Pieter Van den Abeele]

On 17 Dec 2003, at 09:52, George Shapovalov wrote:

> 1. Multilevel categories. For example:
> dev-*  =>  dev/langs/{procedural/,functional/...}
>                       lib/...
>                       util/...
> Support for arbitrary "deepness" is preferable.
> The tree is getting bigger and bigger and this feature should keep it
> browsable for some time to come. Addition of similar category browser 
> to
> http://packages.gentoo.org/ would be nice too.
> (I couldn't get through to it for some reason now to check if its not 
> there.
> So if it is I apologise :)).

I'd support this by considering system components to have unique names 
and consider categories to be syntactic sugar, however if names are not 
unique :

> 2. Support for duplicate package names under different categories.
> There was a (mildly) heated discussion some time ago on that one with 
> may be
> only silghtly more people on the pro side. Having dealt with my 
> portion of
> dups and having been in situation when already included package with 
> the same
> name complicates the (naming) issue I arrived to conclusion that it is
> inevitable. The reality is that we already have couple of such dups 
> and we
> have to deal with this anyway (btw, even with present portage things 
> seem to
> work even if in a semisupported fashion).
>
> These two "features" are connected, so I'll put here some common 
> technical
> elaboration. The debate on "dups" one mostly concentrated around flat 
> vs
> categorised namespace or, on the user level, search vs browse paradigm.
> Following the "Gentoo is about choice" line of thought I see only one
> possible decision here, - to provide both features :). The solution 
> may be
> simple - treat the "category path" as a part of package name.

when names aren't unique, create a unique name simply by taking into 
account category. (category no longer is syntactic sugar, but gets a 
semantic.

> 3.  Add search keywords (or some other tags) to packages.
> I believe this was considered for the extended metadata format in the 
> future
> and even mentioned not so long ago (although I don't remember where 
> and when
> and I don't see relevant GLEP, should I file one?). I am bringing this 
> up
> largerly to complete "the picture" started in pp 1 and 2.
>
> Taken together these three "enhancements" should make it possible to 
> maintain
> both search and browsing ability for portage database even as it 
> continues to
> grow.

that would be trivial to implement, but it does have a (small) impact 
on finding packages for which a given keyword matches. But there are 
many ways to overcome this :-)

> Now last but not least for this email
>
> 4. Partial portage checkout.
> This also was already touted. One of the variants was even recently 
> proposed
> right here, in this list. I am mentioning this mostly to reiterate and 
> remind
> other possible approaches to this.

Storing the 'knowledge base' in a database (such as Mysql) or in a 
ports dir which allows for partial checkout should all be options 
available to the user, or at least implementable.

> 4.1 "Thematic" checkouts.
> Goes along the (proposed) work on "partitioning" the tree for 
> particular
> tasks. The "corporate" theme was mentioned not so long ago (something 
> along
> these lines had a "server" tag in discussions happened about 6-12 
> month ago I
> believe). As long as there is work on supporting a thematic tree 
> "partition"
> and as long as it is in a reasonablt consistent state it should be 
> possible
> to provide such thematic checkout functionality

see previous answer. What exactly do you mean by 'checkout'? I'd 
interpret checkout as: updating (through rsync or whatever) only the 
relevant (using thema) part of the tree.
This might be difficult to implement server wise. I see no problems 
client wise.

> 4.2 "Stability" checkouts.
> Similar partitioning, but based aroung the presumed stability of 
> packages.
> This may follow the present arch/~arch divide or more levels as/if we
> introduce them. Apparently this should be inclusive, i.e. poeple who 
> want
> "testing" stuff are goind\g to get "stable" as well.

A keyword can be seen as a tag,

Do you mean only for instance: updating only the stable packages and/or 
tagged 'foo'? This might become a problem server-side, but 
theoretically it should be possible.

> These two seem like a lot of work, but in reality major part of this 
> work is
> done in package maintainership and other regular activity (that would
> otherwise happen anyway). portage-ng would basically just need to 
> provide
> some support functionality for that.
>
> On the contrary the next one (the one that was proposed not so long 
> ago) seems
> to require not as trivial extension to portage, as it would need some
> nontrivial infrastructure support.
>
> 4.3 "On demand" checkouts.
> I cannot find it right now, but I remember this was proposed on this 
> list just
> a bit over a week ago. Or was it on -dev? Anyway, the idea is to 
> checkout
> some "base" collection by default and then do checkouts of more stuff 
> as user
> requestes (may be indirectly as dependencies).

That would also be possible, the problem is server side. No problem at 
all client side.

> Oh, and we might consider getting all three (or more if they are 
> requested) of
> them, they don't seem to be mutually exclusive :).

sure. I only see problems server wise, if I remember it correctly the 
current mirrors have much bandwidth to spare but less storage and 
computing power.

Pieter

> George
>
>
> --
> gentoo-portage-dev@gentoo.org mailing list


--
gentoo-portage-dev@gentoo.org mailing list