[gentoo-dev] RFC: split up media-sound/ category

[gentoo-dev] RFC: split up media-sound/ category

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 531 bytes --]

Hello,

As we discussed for a while, the media-sound/ category has grown very
large and it may be a good idea to split it.

Right now, it contains audio players, editing software, converters,
sound systems and a lot of other utilities related to sound. Splitting
that up would make looking up software easier for users (e.g. if I want
to take a look at what audio players we have, I don't need to see all
other programs).

What do you think? What new category/-ies do you suggest?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Jesús J. Guerrero Botella]

2011/6/21 Michał Górny <mgorny@gentoo.org>:
> Hello,
>
> As we discussed for a while, the media-sound/ category has grown very
> large and it may be a good idea to split it.
>
> Right now, it contains audio players, editing software, converters,
> sound systems and a lot of other utilities related to sound. Splitting
> that up would make looking up software easier for users (e.g. if I want
> to take a look at what audio players we have, I don't need to see all
> other programs).
>
> What do you think? What new category/-ies do you suggest?

I'd been always against the current classification. It's a bit
confusing to me (it's just my -very subjective- view, of course).

I'd gladly remove the "media-" thing as a whole, and do something like:

gfx-viewers
gfx-editors
gfx-plugins
gfx-libs
sound-players (and maybe sound-radio)
sound-editors
sound-plugins
sound-libs
video-players (and, maybe, only maybe video-tv)
video-editors
video-plugins
video-libs

Then move fonts elsewhere or just leave them as they are.

This is just a big picture though. I don't know if the degree of
granularity that you get with (i.e.) sdl can be achieved with other
libs, so some libs might fit into all of gfx-libs, sound-libs and
video-libs.

Just my random thought, as a mere long-time gentoo user.

-- 
Jesús Guerrero Botella

Re: [gentoo-dev] RFC: split up media-sound/ category

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 224 bytes --]

On Tue, 21 Jun 2011 16:40:59 +0200
Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com> wrote:

> Then move fonts elsewhere or just leave them as they are.

x11-fonts? :P

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Steve Dibb]

On 06/21/2011 08:40 AM, Jesús J. Guerrero Botella wrote:
> 2011/6/21 Michał Górny<mgorny@gentoo.org>:
>> Hello,
>>
>> As we discussed for a while, the media-sound/ category has grown very
>> large and it may be a good idea to split it.
>>
>> Right now, it contains audio players, editing software, converters,
>> sound systems and a lot of other utilities related to sound. Splitting
>> that up would make looking up software easier for users (e.g. if I want
>> to take a look at what audio players we have, I don't need to see all
>> other programs).
>>
>> What do you think? What new category/-ies do you suggest?
>
> I'd been always against the current classification. It's a bit
> confusing to me (it's just my -very subjective- view, of course).
>
> I'd gladly remove the "media-" thing as a whole, and do something like:
>
> gfx-viewers
> gfx-editors
> gfx-plugins
> gfx-libs
> sound-players (and maybe sound-radio)
> sound-editors
> sound-plugins
> sound-libs
> video-players (and, maybe, only maybe video-tv)
> video-editors
> video-plugins
> video-libs

I think 'audio' works better than 'sound' for category prefix of 
audio/sound related apps.

Also, leave video-* in media-video/ and media-libs/  They can fit fine 
in there.  Plus video-players are multimedia players (they don't play 
back just video, for instance).

Steve

Re: [gentoo-dev] RFC: split up media-sound/ category

[Alexis Ballier]

On Tue, 21 Jun 2011 17:21:44 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> On Tue, 21 Jun 2011 16:40:59 +0200
> Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com> wrote:
> 
> > Then move fonts elsewhere or just leave them as they are.
> 
> x11-fonts? :P
> 

Why? Fonts may not be related to X or any graphical interface;
they're used to display subtitles in media players or render documents
with TeX & cie.

A.

[gentoo-dev] Re: RFC: split up media-sound/ category

[Duncan]

Steve Dibb posted on Tue, 21 Jun 2011 09:55:08 -0600 as excerpted:

> On 06/21/2011 08:40 AM, Jesús J. Guerrero Botella wrote:
>> 2011/6/21 Michał Górny<mgorny@gentoo.org>:

>>> media-sound/ category has grown very large and [could be split].
>>> [I]t contains audio players, editing software, converters,
>>> sound systems and a lot of other utilities

>>> What new category/-ies do you suggest?

>> I'd gladly remove the "media-" thing as a whole, and do something like:
>>
>> gfx-viewers gfx-editors gfx-plugins gfx-libs sound-players sound-radio?
>> sound-editors sound-plugins sound-libs video-players video-tv?
>> video-editors video-plugins video-libs
> 
> I think 'audio' works better than 'sound' for category prefix of
> audio/sound related apps.

++ to audio-xxxx

IMO gfx-xxxx looks like some texting/personal shortcut,
not appropriately professional for a (meta-)distro like gentoo.

graphics-xxxx would be quite reasonable, however.
 
> Also, leave video-* in media-video/ and media-libs/  They can fit fine
> in there.  Plus video-players are multimedia players (they don't play
> back just video, for instance).

Right.  mplayer, xine, etc, work just fine as for instance mp3 players.  
So media-xxxx is fine for them.

Plus that means we don't have to either leave media-fonts orphaned or 
find something else appropriate for it.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 969 bytes --]

On Tue, 21 Jun 2011 20:29:23 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

> >> gfx-viewers gfx-editors gfx-plugins gfx-libs sound-players
> >> sound-radio? sound-editors sound-plugins sound-libs video-players
> >> video-tv? video-editors video-plugins video-libs
> > 
> > I think 'audio' works better than 'sound' for category prefix of
> > audio/sound related apps.
> 
> ++ to audio-xxxx
> 
> IMO gfx-xxxx looks like some texting/personal shortcut,
> not appropriately professional for a (meta-)distro like gentoo.

Like in 'media-gfx'? :P

> > Also, leave video-* in media-video/ and media-libs/  They can fit
> > fine in there.  Plus video-players are multimedia players (they
> > don't play back just video, for instance).
> 
> Right.  mplayer, xine, etc, work just fine as for instance mp3
> players. So media-xxxx is fine for them.

But probably we'll move it into media-players and so on then.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-dev] Re: RFC: split up media-sound/ category

[Duncan]

Michał Górny posted on Tue, 21 Jun 2011 22:37:46 +0200 as excerpted:

>> IMO gfx-xxxx looks like some texting/personal shortcut,
>> not appropriately professional for a (meta-)distro like gentoo.
> 
> Like in 'media-gfx'? :P

The initial gfx is rather worse, but yes.  Just as the devmanual states 
about the changelog, package categories should be "proper" English 
(words).  So *wtf* and *gfx* are equally inappropriate.  IMO, of course.

(It's worth noting that the above opinion doesn't extend to individual 
package names, however.  If the authors of a program are fine with 
calling it the gimp (the gfx tool), or clit (convert-lit, an ebook 
converter), or whatever, fine with me.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Jesús J. Guerrero Botella]

audio-* sound better.
graphics-* sounds better, if I suggested gfx it was just it's how we
have it now in media-gfx (I never liked that either).

I also think that media-* can stay. So, something like this would be
closer, I guess.

graphics-viewers
graphics-editors
graphics-plugins
graphics-libs
audio-players (and maybe sound-radio)
audio-editors
audio-plugins
audio-libs
media-players (and, maybe, only maybe video-tv)
media-editors
media-plugins
media-libs

As I said above, some libraries might not be easy to fit in a single
category, so it might be necessary to just keep all of them under
media-libs. I am not sure. What I'd really love would be to get a
mechanism into portage so that a package can be in many categories at
the same time because, well, is gwenview a viewer or an editor? Is
kaffeine a media-player or a tv receiver? Is ffmpeg a media-lib, a
sound-editor or a sound-lib? You get the idea.

Roughly, one way to achieve this could be by using symlinks along with
virtuals, but that's another entirely different thing.

-- 
Jesús Guerrero Botella

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Kent Fredric]

On 22 June 2011 08:37, Michał Górny <mgorny@gentoo.org> wrote:
>> Right.  mplayer, xine, etc, work just fine as for instance mp3
>> players. So media-xxxx is fine for them.
>
> But probably we'll move it into media-players and so on then.

mplayer and some other things most-often used for playback might be a
bit of a hard-to-categorise subject though, ie: mplayer ships with
mencoder, which is more editing spectrum. I'm lead to believe you can
do some degree of transcoding with VLC too, but I might have a memory
problem ( I recall reading something along those lines, but it may be
"future tense" )

incidentally, I'd be in favour of splitting mplayer and mencoder into
seperate dists if it were sanely plausible to do so.


-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

On Wed, 22 Jun 2011 21:18:26 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> incidentally, I'd be in favour of splitting mplayer and mencoder into
> seperate dists if it were sanely plausible to do so.

A same thing would be great, say, for ffmpeg. Right now we just have
USE=encode which enable all encoder dependencies based on other flags.
What if I wanted a minimal encoding support and maximal decoding?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Joshua Saddler]

[-- Attachment #1: Type: text/plain, Size: 1270 bytes --]

On Tue, 21 Jun 2011 16:24:07 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> Hello,
> 
> As we discussed for a while, the media-sound/ category has grown
> very large and it may be a good idea to split it.
> 
> Right now, it contains audio players, editing software, converters,
> sound systems and a lot of other utilities related to sound.
> Splitting that up would make looking up software easier for users
> (e.g. if I want to take a look at what audio players we have, I
> don't need to see all other programs).
> 
> What do you think? What new category/-ies do you suggest?
> 

Whatever happened to implementing tags for the Portage tree? The idea
behind tags was to avoid spamming users with more and more
directories, especially for apps that are hard to categorize. Which,
arguably, is most of them. We have tags for weblog content/topics;
tags for ebuilds are also a good idea.

Splitting up the media-* categories doesn't solve any problems for
the packages that do many different things, whether encoding,
playing, editing, wrapping, connecting, etc. Many media apps belong
in 3 or 4 or more categories. Tags are the right solution for these,
rather than being pigeonholed into just one category, which only
reflects one use.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Kent Fredric]

On 22 June 2011 21:38, Joshua Saddler <nightmorph@gentoo.org> wrote:
>
> Whatever happened to implementing tags for the Portage tree? The idea
> behind tags was to avoid spamming users with more and more
> directories, especially for apps that are hard to categorize. Which,

Agreed. From the perspective of what will need to happen with regard
to package moves, implementing this change is likely to be incredibly
disruptive to users, and possibly cause a few wtfs/cry fests. ( Worst
case scenario pessimism I admit ).

I'd love a tag solution, that'd be nice, is there a GLEP for it yet?
And if so, how long will it take to get this "tag" feature supported
by EAPI standards?

-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Tags (Was: RFC: split up media-sound/ category)

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 603 bytes --]

On Wed, 22 Jun 2011 21:55:18 +1200
Kent Fredric <kentfredric@gmail.com> wrote:
> I'd love a tag solution, that'd be nice, is there a GLEP for it yet?
> And if so, how long will it take to get this "tag" feature supported
> by EAPI standards?

The slow parts are coming up with a good design, getting the Council to
approve it, and getting Portage to implement it. The fast part is
getting the PMS bit done.

The problem with tags is that all we've heard so far is "we should have
tags!", with no description of what tags are, what they'll solve or how
they're used.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Zac Medico]

On 06/22/2011 02:38 AM, Joshua Saddler wrote:
> Whatever happened to implementing tags for the Portage tree? The idea
> behind tags was to avoid spamming users with more and more
> directories, especially for apps that are hard to categorize. Which,
> arguably, is most of them. We have tags for weblog content/topics;
> tags for ebuilds are also a good idea.

It could be implemented with a metadata.xml extension, or possibly an
new ebuild variable. A GLEP would be a good way to propose such an
extension.

> Splitting up the media-* categories doesn't solve any problems for
> the packages that do many different things, whether encoding,
> playing, editing, wrapping, connecting, etc. Many media apps belong
> in 3 or 4 or more categories. Tags are the right solution for these,
> rather than being pigeonholed into just one category, which only
> reflects one use.

I tend to agree that addition of more categories probably isn't very
useful. However, I wouldn't be opposed to people adding new categories
if they believe that it will be useful somehow.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFC: split up media-sound/ category

[Kent Fredric]

On 23 June 2011 09:46, Zac Medico <zmedico@gentoo.org> wrote:

> It could be implemented with a metadata.xml extension, or possibly an
> new ebuild variable. A GLEP would be a good way to propose such an
> extension.

I'd myself see metadata.xml or ebuild variables, on their own, less
than useful.

For me, the primary benefit of the category system is it makes it easy
to locate and install packages I wish to install, and hidden metadata
stored in the package itself would prove very unhelpful in this
regard.

In order for this metadata to be of any use to a user, it would need
to have some way to facilitate its use, whether it be a fake generated
directory of symlinks, or a dedicated program ( like debians aptitude
) for exploring this data in a tag-oriented way.

( And any solution which requires software to iterate and index the
entirety of portage to accumulate this tag data, user side, will be
considerably unfavourable in my eyes )

-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] RFC: split up media-sound/ category

[Zac Medico]

On 06/22/2011 04:58 PM, Kent Fredric wrote:
> On 23 June 2011 09:46, Zac Medico <zmedico@gentoo.org> wrote:
> 
>> It could be implemented with a metadata.xml extension, or possibly an
>> new ebuild variable. A GLEP would be a good way to propose such an
>> extension.
> 
> I'd myself see metadata.xml or ebuild variables, on their own, less
> than useful.
> 
> For me, the primary benefit of the category system is it makes it easy
> to locate and install packages I wish to install, and hidden metadata
> stored in the package itself would prove very unhelpful in this
> regard.

Of course, support would have to be integrated into search tools.

> In order for this metadata to be of any use to a user, it would need
> to have some way to facilitate its use, whether it be a fake generated
> directory of symlinks, or a dedicated program ( like debians aptitude
> ) for exploring this data in a tag-oriented way.
> 
> ( And any solution which requires software to iterate and index the
> entirety of portage to accumulate this tag data, user side, will be
> considerably unfavourable in my eyes )

For example, we could extend egencache to generate an index of tags,
similar to how it generates use.local.desc from all of the metadata.xml
files.
-- 
Thanks,
Zac

Re: [gentoo-dev] Tags (Was: RFC: split up media-sound/ category)

[Wyatt Epp]

On Wed, Jun 22, 2011 at 14:19, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Wed, 22 Jun 2011 21:55:18 +1200
> Kent Fredric <kentfredric@gmail.com> wrote:
>> I'd love a tag solution, that'd be nice, is there a GLEP for it yet?
>> And if so, how long will it take to get this "tag" feature supported
>> by EAPI standards?
>
> The slow parts are coming up with a good design, getting the Council to
> approve it, and getting Portage to implement it. The fast part is
> getting the PMS bit done.
>
> The problem with tags is that all we've heard so far is "we should have
> tags!", with no description of what tags are, what they'll solve or how
> they're used.
>
> --
> Ciaran McCreesh
>

Tags are basically keywords you can use to describe packages, allowing
you to easily search and explore your options based on what the
packages actually does (if we want to get technical, anything that
identifies a package is a sort of tag: name, version, license, set,
checksum, etc.).  It's just a vocabulary that eases the burden of
human lookup.  The categories we have now are essentially (pairs of)
tags tied to a treelike structure in an actual filesystem, and I'd
wager that's a decent place to start, too-- probably the most
prominent problem I can see with the current method comes from these
edge cases where one category is obviously not enough.  The obvious
solution is probably to just stick our semantic metadata into the
metadata.xml.  So for...say, media-video/kdenlive,
<cat>media-video</cat>[1] becomes more like this:

<cat>
<tag>media</tag>
<tag>video</tag>
<tag>kde</tag>
<tag>editors</tag>
</cat>

The canonical tag list needn't even expand beyond what we have already
(for the time being; attempting to keep your vocabulary entirely
static is a Bad Thing.  Humans are amazing at finding new things that
need tagging.  Getting ahead of myself, though).

In the practical sense, we can probably just whip out a quick script
and get 98% coverage; package maintainers should be encouraged to add
relevant tags to the packages under their care as needed.

--Wyatt, hoping this text is plain as it says it is.  Sorry if it's not.

[1] Let's just assume for the sake of argument that kdenlive actually
has a <cat> field in its metadata file.

[gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Duncan]

Wyatt Epp posted on Wed, 22 Jun 2011 20:57:47 -0400 as excerpted:

> On Wed, Jun 22, 2011 at 14:19, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
>> On Wed, 22 Jun 2011 21:55:18 +1200 Kent Fredric <kentfredric@gmail.com>
>> wrote:
>>> I'd love a tag solution, that'd be nice, is there a GLEP for it yet?

>> The slow parts are coming up with a good design, getting the Council to
>> approve it, and getting Portage to implement it. The fast part is
>> getting the PMS bit done.
>>
>> The problem with tags is that all we've heard so far is "we should have
>> tags!", with no description of what tags are, what they'll solve or how
>> they're used.

> Tags are basically keywords you can use to describe packages, allowing
> you to easily search and explore your options based on what the packages
> actually does (if we want to get technical, anything that identifies a
> package is a sort of tag: name, version, license, set, checksum, etc.).

Umm... I believe Ciaran meant "no description" in the practical PM 
implementation rules sense, not in the general definition sense, which I 
suppose most folks here understand by now.

Ciaran is, after all, a (arguably the) prime mover behind PMS, defining 
the standards to which Gentoo package managers must conform.  So his 
concern is very likely to be in that regard -- actually seeing a draft 
GLEP defining the technical rules in enough detail that the three PMs can 
implement it to the point that if they disagree on how a tag should be 
implemented and treated, it's clear which one's bugged and which ones 
following the standard, so the bugged one can be fixed.

Until that happens, or at least is actually in process, it's all talk.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Wyatt Epp]

On Wed, Jun 22, 2011 at 21:25, Duncan <1i5t5.duncan@cox.net> wrote:
> Umm... I believe Ciaran meant "no description" in the practical PM
> implementation rules sense, not in the general definition sense, which I
> suppose most folks here understand by now.
>
Most is not all. ;)  In general, I try not to assume everyone is on
the same page; one of the things academia got right.

> Until that happens, or at least is actually in process, it's all talk.
>
Shall we call it "in process" right now, then?  My impression was he
was calling for us to get down to brass tacks and hammer this out for
real.  Apologies in advance for the long post.

As far as what I've said already, a quick read of the PMS tells me
that "[metadata.xml's] exact format is strictly beyond the scope" of
it.  Would it be acceptable to add this to the ebuilds themselves?
Otherwise, at least the tags become mandatory and drag the xml into
this.  Given that encoding tags into directory paths is why we're
talking about this in the first place, that's out; the third obvious
solution is a separate file for each package, but....yeah, not where I
would personally go with it without thinking long and hard about the
other two first.

The directory paths themselves....well, one solution I noted from the
other thread was to populate tag directories with symlinks.  I've done
similar things, but always thought of it as a hack, so I'm reluctant
to advocate for building a deployable semantic system on top of it-- I
could potentially be convinced otherwise, though.  Given that tags and
categories have roughly the same purpose and end result, a flat ebuild
directory referenced only by its metadata should certainly be
possible.  If this is going to happen, and happen right, what this all
looks like in the filesystem is moot anyway.

I bring this up because there are several packages with the same name
and different qualification.  Obviously, they'll have different tags
because they're not the same thing, but neither should they share the
same directory.  So the simple solution is to just change the package
names so we avoid collision and preserve our flat ontology (I've
forgotten the objection to doing this; please forgive).  The next
simplest solution is to just name the directories as hashes in-tree
and cover it up with software magic (I'm pretty sure this ends up
pretty ugly, implementation-wise).

For the sake of migration, packages should probably have their current
category/directory added to the tags; deprecate and remove after
sufficient time (I think this is one of those two-year changes?).

Those are roughly my thoughts for the time being.  Let's do this thing!

Regards,
Wyatt

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1230 bytes --]

On Thu, 23 Jun 2011 01:25:57 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:
> Umm... I believe Ciaran meant "no description" in the practical PM 
> implementation rules sense, not in the general definition sense,
> which I suppose most folks here understand by now.

No, it's worse than that. Consider the following questions. Most people
will say that tags "obviously" have a particular answer for each
question, but they disagree on which answer it is...

First: how do tags relate to categories? Are they independent, a
refinement or a replacement?

Second: which of the following are tags suitable for? Searching.
Browsing just using 'ls' etc. Browsing using tools (e.g. a website or a
package manager command). Using as dependencies.

Third: are tags a property of ebuilds, of packages, of a repository, or
entirely external?

Fourth: do tags in any way identify a package?

Fifth: how do overlays fit in to all of this?

From the previous times we've had this discussion, I very much get the
impression that right now "we'll use tags!" is an utterly nebulous
answer to a problem that's about on par with "we'll replace all use of
oil by green technology by next week!".

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Jesús J. Guerrero Botella]

2011/6/23 Kent Fredric <kentfredric@gmail.com>:
> On 23 June 2011 09:46, Zac Medico <zmedico@gentoo.org> wrote:
>
> In order for this metadata to be of any use to a user, it would need
> to have some way to facilitate its use, whether it be a fake generated
> directory of symlinks, or a dedicated program ( like debians aptitude
> ) for exploring this data in a tag-oriented way.

The problem is that there's no official GUI for portage, and I don't
think we should have that either.  Symlinks are clean, and portage has
always been file-oriented so I see no problem with using them for
this. All we need is to deference the symlink to find the real name of
the package and put it in world instead of the symlinked name, so the
rest of packages won't even need to be retouched to fix the
dependencies. I don't really know if it's that simple as it sounds,
but it's an idea.

For the user, it will be a convenient way to look into media-tv, and
find there all the tv players like kaffeine and mplayer that s/he
would not have found otherwise. Even portage managers like portato
will list the packages following the directory structure, so I think
we should concentrate on that, rather than doing fancy things that
won't be useful for a thousand years. Tags might be elegant, but I
don't think they are practical for the average Gentoo user, which
probably is the kind of user that sets USE="-semantic-desktop" to
avoid using the whole kde tagging system. I also don't know if the
advantages of tagging are really worth all the pain to implement. And
after that, every Gentoo user will have to learn a new way to interact
with portage when it comes to searching the package s/he needs.

-- 
Jesús Guerrero Botella

[gentoo-dev] Re: RFC: split up media-sound/ category

[Duncan]

Jesús J. Guerrero Botella posted on Thu, 23 Jun 2011 08:15:44 +0200 as
excerpted:

> Symlinks are clean, and portage has always been file-oriented so I see
> no problem with using them for this.

It has been some years since I've seen the argument made, but if I'm not 
mistaken, at least back in 2004-ish when I first switched to Gentoo, the 
argument against in-tree symlinking (or multi-hard-linking, for that 
matter) of any kind (other than the obvious directory hard-linking) was 
that we wanted to keep the tree at least minimally deployable on non-Unix 
filesystems like fat/ntfs.  Note that while a user's profile uses a 
symlink, the symlink is on /etc (which is thus implied to be a Unix 
filesystem with symlinking capacities) pointing /into/ the tree, NOT 
actually PART OF the tree.

One scenario in which this might be a factor is that of someone doing 
their syncs and source downloads at work where they have lots of 
bandwidth available, then sneakernetting it home on a fat32 formatted 
thumbdrive.

Now it can be argued that the flexibility benefit of multi-category 
packages trumps that of being able to put the tree on fat or whatever, 
but there IS a definite loss of tree portability that's implied, and thus 
a tradeoff to be considered.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Jesús J. Guerrero Botella]

2011/6/23 Duncan <1i5t5.duncan@cox.net>:
> Jesús J. Guerrero Botella posted on Thu, 23 Jun 2011 08:15:44 +0200 as
> excerpted:
>
>> Symlinks are clean, and portage has always been file-oriented so I see
>> no problem with using them for this.
>
> It has been some years since I've seen the argument made, but if I'm not
> mistaken, at least back in 2004-ish when I first switched to Gentoo, the
> argument against in-tree symlinking (or multi-hard-linking, for that
> matter) of any kind (other than the obvious directory hard-linking) was
> that we wanted to keep the tree at least minimally deployable on non-Unix
> filesystems like fat/ntfs.  Note that while a user's profile uses a
> symlink, the symlink is on /etc (which is thus implied to be a Unix
> filesystem with symlinking capacities) pointing /into/ the tree, NOT
> actually PART OF the tree.
>
> One scenario in which this might be a factor is that of someone doing
> their syncs and source downloads at work where they have lots of
> bandwidth available, then sneakernetting it home on a fat32 formatted
> thumbdrive.
>
> Now it can be argued that the flexibility benefit of multi-category
> packages trumps that of being able to put the tree on fat or whatever,
> but there IS a definite loss of tree portability that's implied, and thus
> a tradeoff to be considered.

Yes, that's true. But it's also true that besides the symlinks, the
portage tree will be broken the same moment you put it into a fat
volume, because it will directly erase all the permissions and
ownership metadata. So, the thing is already broken, why should we
care if it breaks a bit more? Seriously, limiting ourselves because of
an fs that not even MS uses any longer is not a smart thing to do, in
my opinion.

-- 
Jesús Guerrero Botella

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 458 bytes --]

On Thu, 23 Jun 2011 11:08:35 +0200
Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com> wrote:
> But it's also true that besides the symlinks, the portage tree will
> be broken the same moment you put it into a fat volume, because it
> will directly erase all the permissions and ownership metadata.

Those don't matter to a package manager, beyond that they're readable
by every process. What does matter is mtimes.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Aaron W. Swenson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 06/22/2011 05:18 AM, Kent Fredric wrote:
> On 22 June 2011 08:37, Michał Górny <mgorny@gentoo.org> wrote:
>>> Right.  mplayer, xine, etc, work just fine as for instance mp3
>>> players. So media-xxxx is fine for them.
>>
>> But probably we'll move it into media-players and so on then.
> 
> mplayer and some other things most-often used for playback might be a
> bit of a hard-to-categorise subject though, ie: mplayer ships with
> mencoder, which is more editing spectrum. I'm lead to believe you can
> do some degree of transcoding with VLC too, but I might have a memory
> problem ( I recall reading something along those lines, but it may be
> "future tense" )
> 
> incidentally, I'd be in favour of splitting mplayer and mencoder into
> seperate dists if it were sanely plausible to do so.
> 
> 
I'd venture that most people grab mplayer for its playback capabilities
more often than its ability to modify media.

- - Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iF4EAREIAAYFAk4Dvq4ACgkQCOhwUhu5AEk3DQD7B9rllKzVCBEHmclLmvIfF5jN
09s4pF9iqc6PVriDyPUA/i/jHmmjHxHc6tBq59Ao8jNI4aU4tuoxFDhmatCOxJcp
=m8pk
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Wyatt Epp]

On Thu, Jun 23, 2011 at 02:14, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> First: how do tags relate to categories? Are they independent, a
> refinement or a replacement?
>
I would suggest they be a replacement because categories are just an
overly limited subset of a proper tagging scheme.

> Second: which of the following are tags suitable for? Searching.
> Browsing just using 'ls' etc. Browsing using tools (e.g. a website or a
> package manager command). Using as dependencies.
>
Yes, Maybe, Yes, Probably Not.

> Third: are tags a property of ebuilds, of packages, of a repository, or
> entirely external?
>
Likely packages.  Though I suppose the potential exists for a version
change to add or remove features and change tags; so, ebuilds?

> Fourth: do tags in any way identify a package?
>
Uniquely?  No.

Regards,
Wyatt

Re: [gentoo-dev] RFC: split up media-sound/ category

[Zac Medico]

On 06/22/2011 11:15 PM, Jesús J. Guerrero Botella wrote:
> Symlinks are clean, and portage has
> always been file-oriented so I see no problem with using them for
> this. All we need is to deference the symlink to find the real name of
> the package and put it in world instead of the symlinked name, so the
> rest of packages won't even need to be retouched to fix the
> dependencies. I don't really know if it's that simple as it sounds,
> but it's an idea.

For some reason, using symlinks to represent tags seems like an odd
approach to me. I think it would be much more sensible to put them in
metadata.xml or an ebuild variable. If for some reason you want symlinks
representing the tags (I don't know why you would), you can always use a
script to generate symlinks from metadata.xml files.
-- 
Thanks,
Zac

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Kent Fredric]

Another question I think that needs be asked, is "What does a tag look
like", or "What sort of values can a 'tag' have?".

People are probably assuming it will be some arbitrary name like
"video" or "sound" or soforth, but there is potential to use tags for
more than that.

Consider ideas such as tag-spacing.

Intent:  editing, viewing, processing, validating, etc
Material: video, images, sound, text, etc
Formats: mp3, jpeg, etc.

( ie: it might be useful to list all programs that are known to work
with a given format )

Should this tag namespace be part of the tag itself? Or something different?

/tags/intent/editing/*

or

/tags/intent:editing/*


-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 600 bytes --]

On Thu, 23 Jun 2011 20:07:50 -0400
Wyatt Epp <wyatt.epp@gmail.com> wrote:

> On Thu, Jun 23, 2011 at 02:14, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
> > First: how do tags relate to categories? Are they independent, a
> > refinement or a replacement?
> >
> I would suggest they be a replacement because categories are just an
> overly limited subset of a proper tagging scheme.

How about packages like app-doc/pms and media-sound/pms? Shall we
prepare to have PM saying 'specify one more tag to distinguish between
the packages'?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 674 bytes --]

On Thu, 23 Jun 2011 20:07:50 -0400
Wyatt Epp <wyatt.epp@gmail.com> wrote:
> On Thu, Jun 23, 2011 at 02:14, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
> > First: how do tags relate to categories? Are they independent, a
> > refinement or a replacement?
>
> I would suggest they be a replacement because categories are just an
> overly limited subset of a proper tagging scheme.

If you abolish categories in favour of tags, but tags don't uniquely
identify a package, how do you uniquely identify a package?

Remember that your solution has to work with overlays and with tags
being changed after a package is installed.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Zac Medico]

On 06/23/2011 05:07 PM, Wyatt Epp wrote:
> On Thu, Jun 23, 2011 at 02:14, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
>> First: how do tags relate to categories? Are they independent, a
>> refinement or a replacement?
>>
> I would suggest they be a replacement because categories are just an
> overly limited subset of a proper tagging scheme.

Since categories and tags can easily coexist, you might want to rethink
that.  It's relatively easy to implement a tagging mechanism, while
(unnecessarily) ripping out the existing category framework is a big
chore that may not have any practical value.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFC: split up media-sound/ category

[Jesús J. Guerrero Botella]

2011/6/24 Zac Medico <zmedico@gentoo.org>:
> On 06/22/2011 11:15 PM, Jesús J. Guerrero Botella wrote:
>> Symlinks are clean, and portage has
>> always been file-oriented so I see no problem with using them for
>> this. All we need is to deference the symlink to find the real name of
>> the package and put it in world instead of the symlinked name, so the
>> rest of packages won't even need to be retouched to fix the
>> dependencies. I don't really know if it's that simple as it sounds,
>> but it's an idea.
>
> For some reason, using symlinks to represent tags seems like an odd
> approach to me. I think it would be much more sensible to put them in
> metadata.xml or an ebuild variable. If for some reason you want symlinks
> representing the tags (I don't know why you would), you can always use a
> script to generate symlinks from metadata.xml files.

You might not like it, but Gentoo categories have always been
directories, not words into metadata.xml. Most portage tools rely on
that. Not a strong argument, I know that. But someone used this
argument when someone else wanted to put portage into a database
instead of an fs-based tree. That was long ago, admittedly, don't know
if that conversation ever came up again.

I've personally never bothered to learn how to use external tools
anyway, so I just navigate the tree using command line tools when I
need to know something about a given package. I am sure I am not alone
in that regard. I guess I could also "nano metadata.xml", ugh!

Some portage GUIs also use this categorization scheme, like portato or
porthole (not that they are important at all, but they illustrate the
trend).

Maybe it's just my mind model is archaic, but I can't really agree
with tagging for massive trees. I wouldn't drop all my 40 thousand
songs into a single folder and rely on tagging to keep them at hand.
Portage has way more files so I don't see how tagging would be better
for it than it would be for my music collection. I might be too much
influenced by *nix (and DOS) OSes at this stage to be able to see the
advantages of tagging (besides the decorative function), I admit.

-- 
Jesús Guerrero Botella

Re: [gentoo-dev] RFC: split up media-sound/ category

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 295 bytes --]

On Fri, 24 Jun 2011 09:52:03 +0200
Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com> wrote:
> You might not like it, but Gentoo categories have always been
> directories, not words into metadata.xml.

So tags are in some way related to categories then?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Jesús J. Guerrero Botella]

2011/6/24 Ciaran McCreesh <ciaran.mccreesh@googlemail.com>:
> On Fri, 24 Jun 2011 09:52:03 +0200
> Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com> wrote:
>> You might not like it, but Gentoo categories have always been
>> directories, not words into metadata.xml.
>
> So tags are in some way related to categories then?

For me, tags would be an abstract concept, related to

A) categories (as in the current dir-based category concept)
B) descriptions
C) maybe, user tastes, so they should be flexible, just like id3

I see them as an instrumental and optional thing, but I don't think
they should be the base of an fs-based system (because then the system
wouldn't be fs-based, that is).

-- 
Jesús Guerrero Botella

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Wyatt Epp]

On Fri, Jun 24, 2011 at 02:51, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> If you abolish categories in favour of tags, but tags don't uniquely
> identify a package, how do you uniquely identify a package?
>
> Remember that your solution has to work with overlays and with tags
> being changed after a package is installed.
>
I seem to have misunderstood the thrust of your question?  media-sound
is a category (tag); each package still has its name, URI, files
associated with it and their checksums.  A combination of a tag or two
and package name is going to be plenty for such a small data set
excepting some pretty absurd circumstance where four projects all
choose the same name and do the same thing.  Alternatively, we could
just make names unique in the first place and nip that problem in the
bud forever.  Either way, tags changing isn't especially different
from categories changing.

On Fri, Jun 24, 2011 at 03:18, Zac Medico <zmedico@gentoo.org> wrote:
> On 06/23/2011 05:07 PM, Wyatt Epp wrote:
> Since categories and tags can easily coexist, you might want to rethink
> that.  It's relatively easy to implement a tagging mechanism, while
> (unnecessarily) ripping out the existing category framework is a big
> chore that may not have any practical value.

I was more thinking that in the long term it's reduplication of effort
and annoying.  I probably should have worded it as "tags deprecate
categories".  You're right that practically speaking, once we figure
out where to put it, it's roughly within reach.

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Kent Fredric]

On 24 June 2011 20:14, Wyatt Epp <wyatt.epp@gmail.com> wrote:
> On Fri, Jun 24, 2011 at 02:51, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
>> If you abolish categories in favour of tags, but tags don't uniquely
>> identify a package, how do you uniquely identify a package?
>>
>> Remember that your solution has to work with overlays and with tags
>> being changed after a package is installed.
>>
> I seem to have misunderstood the thrust of your question?  media-sound
> is a category (tag); each package still has its name, URI, files
> associated with it and their checksums.  A combination of a tag or two
> and package name is going to be plenty for such a small data set
> excepting some pretty absurd circumstance where four projects all
> choose the same name and do the same thing.  Alternatively, we could
> just make names unique in the first place and nip that problem in the
> bud forever.  Either way, tags changing isn't especially different
> from categories changing.
>

Not really, after all, files will always have 1 canonical path, and
all the other paths will be some form of reference to that canonical
path ( ie: symlinks, a text-file with the canonical name, etc )

Tags would only be used for user access, not for internal dependency
cycles, and internals would ultimately use the canonical path for the
ebuild in sourcing the ebuild and generating the respective folder in
the VDB.

As it stands, when something changes category, many things must
happen. All the things that depend on it must change, the package must
be relocated in the VDB, and all the things that depend on it in the
VDB need to be patched to refer to the newly located category name.

With tags, I'd expect none of this neccessary, all that would change
is the metadata index that maps tags to package names.

> I was more thinking that in the long term it's reduplication of effort
> and annoying.  I probably should have worded it as "tags deprecate
> categories".  You're right that practically speaking, once we figure

I myself can't see tags immediately replacing categories, not without
needing to do something asinine like putting all the ebuilds in a
top-level directory with no parent categories, named by SHA1 Sum, as
the "Canonical pool"  of packages which all the tag "references" point
to.

Moving to that state of things seems like a /long/ way off, if ever.


-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Nathan Phillip Brink]

[-- Attachment #1: Type: text/plain, Size: 1514 bytes --]

On Wed, Jun 22, 2011 at 11:20:40PM -0400, Wyatt Epp wrote:
> I bring this up because there are several packages with the same name
> and different qualification.  Obviously, they'll have different tags
> because they're not the same thing, but neither should they share the
> same directory.  So the simple solution is to just change the package
> names so we avoid collision and preserve our flat ontology (I've
> forgotten the objection to doing this; please forgive).

I believe that the objection is that it is better to follow upstreams'
package names as directly as possible. This would look better and be
less confusing than having a package named git and git-core, like I've
seen elsewhere. Having categories would also prevent changing an
ebuild's name from upstream's name only for the sake of giving it a
unique name in Gentoo.

I think that in most cases, when package name collisions happen, the
colliding packages differ enough that they'd conceivably be in
different portage categories, letting them be uniquely identified in
Gentoo. If someone is planning on writing a new program, he likely
knows about already-existing alternatives to this package. The author
of a new sound editing suite would not name his suite `sox' because
the author cannot help but to know that media-sound/sox exists. But
someone writing some new sax thing might play off of `sax' and name it
`sox', though this is hypothetical ;-).

-- 
binki

Look out for missing or extraneous apostrophes!

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Tags (Was: RFC: split up media-sound/ category)

[Nathan Phillip Brink]

[-- Attachment #1: Type: text/plain, Size: 1645 bytes --]

On Wed, Jun 22, 2011 at 08:57:47PM -0400, Wyatt Epp wrote:
> Tags are basically keywords you can use to describe packages, allowing
> you to easily search and explore your options based on what the
> packages actually does (if we want to get technical, anything that
> identifies a package is a sort of tag: name, version, license, set,
> checksum, etc.). ??It's just a vocabulary that eases the burden of
> human lookup. ??The categories we have now are essentially (pairs of)
> tags tied to a treelike structure in an actual filesystem, and I'd
> wager that's a decent place to start, too-- probably the most
> prominent problem I can see with the current method comes from these
> edge cases where one category is obviously not enough. ??The obvious
> solution is probably to just stick our semantic metadata into the
> metadata.xml. ??So for...say, media-video/kdenlive,
> <cat>media-video</cat>[1] becomes more like this:
> 
> <cat>
> <tag>media</tag>
> <tag>video</tag>
> <tag>kde</tag>
> <tag>editors</tag>
> </cat>

I'm going to just interpret this as a suggestion for a modification to
metadata.xml ;-). Could this not just be:

  <tags>
    <tag>kde</tag>
    <tag>editors</tag>
  </tags>

Then in the category's metadata.xml, at media-video/metdata.xml, you
can fill in the rest:

  <tags>
    <tag>media</tag>
    <tag>video</tag>
  </tags>

It would be nice to take advantage of the existing categories in
Gentoo instead of having to duplicate all of this information over and
over -- if this is to be done with metadata.xml.

-- 
binki

Look out for missing or extraneous apostrophes!

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: Tags (Was: RFC: split up media-sound/ category)

[Kent Fredric]

On 25 June 2011 00:51, Nathan Phillip Brink <binki@gentoo.org> wrote:
> On Wed, Jun 22, 2011 at 11:20:40PM -0400, Wyatt Epp wrote:
>> I bring this up because there are several packages with the same name
>> and different qualification.  Obviously, they'll have different tags
>> because they're not the same thing, but neither should they share the
>> same directory.  So the simple solution is to just change the package
>> names so we avoid collision and preserve our flat ontology (I've
>> forgotten the objection to doing this; please forgive).
>
> I believe that the objection is that it is better to follow upstreams'
> package names as directly as possible. This would look better and be
> less confusing than having a package named git and git-core, like I've
> seen elsewhere. Having categories would also prevent changing an
> ebuild's name from upstream's name only for the sake of giving it a
> unique name in Gentoo.

You could possibly abuse the tag feature to solve this issue in
interesting ways.

Both could have:

<tag class="common-name">git</tag>

and thus people looking for 'git' would be more likely to find it.

You could also perhaps extend this idea to other forms of metadata
that users are likely to want to be able to search, perhaps:

<tag class="provides-binary">git</tag>

But that specific usage would probably be deemed slightly abusive. (
as for instance, you may wish to also list what the binary is usually
called, and what gentoo ships it as to prevent collisions )

>
> I think that in most cases, when package name collisions happen, the
> colliding packages differ enough that they'd conceivably be in
> different portage categories, letting them be uniquely identified in
> Gentoo. If someone is planning on writing a new program, he likely
> knows about already-existing alternatives to this package. The author
> of a new sound editing suite would not name his suite `sox' because
> the author cannot help but to know that media-sound/sox exists. But
> someone writing some new sax thing might play off of `sax' and name it
> `sox', though this is hypothetical ;-).


But with this, you could store one as media-sound/sox-translator and
the other as media-sound/sox-saxophone  or something equally unique
but arbitrary and tag them both with

<tag class="common-name">sox</tag>

Also, with this in mind, it may be better to have some types of tags
that are only aggregated in an index of sorts, and others which are
also perhaps made available as a tree of symlinks.

> --
> binki
>
> Look out for missing or extraneous apostrophes!
>



-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Tags (Was: RFC: split up media-sound/ category)

[Kent Fredric]

On 25 June 2011 00:57, Nathan Phillip Brink <binki@gentoo.org> wrote:
> On Wed, Jun 22, 2011 at 08:57:47PM -0400, Wyatt Epp wrote:

>> <cat>
>> <tag>media</tag>
>> <tag>video</tag>
>> <tag>kde</tag>
>> <tag>editors</tag>
>> </cat>
>

I'm strongly of the mind that by making the tag system arbitrarily
flat, you might be prematurely limiting yourself, as well as risking a
future where the "tag index" is a sea of meaningless words.

Tags in my mind, should be grouped by the sort of information they are
trying to convey, as opposed to being arbitrary and completely
un-grouped.

The present category system only has one namespace, which is more or
less "what-you-use-it-for", and if your tag system is likewise going
to take that vector as the only approach, you will ultimately end up
duplicating the category system, albeit without the present limitation
that means one package can only exist in one place.

This need not be the case, we can suggest alternative tag namespaces,
such as : The sorts of files it supports working with, the sorts of
things it can read, the sorts of things it can write.

At present, things that migrate one type of media to another, such as
pdf -> image , image -> pdf, image -> video , video -> images , etc
have to be forced to a sort of useless categorisation system.

However, if via tag data, we were able to annotate a) what can be
written and b) what can be read, this system could be leveraged to
epic proportions of win.

   tag-lookup --supporting $( file ./foo );
   ....
   Read/Write:
       foobarnator - Blah blah blah
   Read:
       foo-bar       - Blah blah
       foo-bjaz     - Blah blah blah
  Write:
       a2foo         - Blah Blah


   tag-lookup --verbose --supporting $( file ./foo );
   ....
   Read/Write:
       foobarnator - Blah blah blah
                                - reads x , y , z , foo
                                - writes a, b, c, foo
   Read:
       foo-bar       - Blah blah
                                - reads foo
                                - writes text
       foo-bjaz     - Blah blah blah
                                - reads foo, bar
                                - writes text, mp3
  Write:
       a2foo         - Blah Blah
                              -reads mp3, png, jpeg
                              -writes foo


As a side note, it may be beneficial to tag a package version
specifically for some of the above mentioned features. Especially if
you wish to support my "provides-binary" suggestion, because the
shipped binary may change from one version/slot to another.

I'm not sure if there's a way to provide data on a per-version level
yet in Metadata.xml, but I am assuming there's not as I don't see it
documented.

<pkgmetadata>
     <versionspecific>
         <slot>2</slot>
         <pkgmetadata>
                 ... normal stuff
         </pkgmetadata>
    </versionspecific>
     <versionspecific>
         <maxversion>1.0</maxversion>
         <maxversion>1.999</maxversion>
         <pkgmetadata>
                 ... normal stuff
         </pkgmetadata>
    </versionspecific>
</pkgmetadata>


Or something similar.



-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Tags (Was: RFC: split up media-sound/ category)

[Wyatt Epp]

On Sat, Jun 25, 2011 at 02:49, Kent Fredric <kentfredric@gmail.com> wrote:
> I'm strongly of the mind that by making the tag system arbitrarily
> flat, you might be prematurely limiting yourself, as well as risking a
> future where the "tag index" is a sea of meaningless words.
>
> Tags in my mind, should be grouped by the sort of information they are
> trying to convey, as opposed to being arbitrary and completely
> un-grouped.
>
> The present category system only has one namespace, which is more or
> less "what-you-use-it-for", and if your tag system is likewise going
> to take that vector as the only approach, you will ultimately end up
> duplicating the category system, albeit without the present limitation
> that means one package can only exist in one place.
>
> This need not be the case, we can suggest alternative tag namespaces,
> such as : The sorts of files it supports working with, the sorts of
> things it can read, the sorts of things it can write.
>
> At present, things that migrate one type of media to another, such as
> pdf -> image , image -> pdf, image -> video , video -> images , etc
> have to be forced to a sort of useless categorisation system.
>
> However, if via tag data, we were able to annotate a) what can be
> written and b) what can be read, this system could be leveraged to
> epic proportions of win.
>
Okay, apologies in advance for my long-windedness.  I hope this all
makes sense to everyone.

I should probably clarify that cloying strictly to flatness is not
what I'm proposing.  Reality has borne out the need for implications
and aliases in sanitising an unruly dataset with a complex
user-generated index, while arbitrary democratised group building has
improved some aspects of discovery.  However, I would consider these
features to be a lower priority than having a system at all.

So to break it down:
Tags - a concise vocabulary used for search.  In their default state
they are untyped and non-hierarchical.  They identify traits of a
package.  Suggest using lower-case and simple, descriptive naming
conventions. Highest priority.
Example: alien {{converter nogui package_management reads_tgz
reads_rpm reads_pkg reads_slp reads_lsb writes_tgz writes_rpm
writes_pkg writes_slp writes_lsb}}

Alias - a relationship between two tags establishing equivalence.
Query of the left term returns results of the right.  This type of
relationship helps reduce dictionary clutter. Low priority.
Example: sound = audio.  Attempting to add "sound" to a package will
instead add "audio" and searches for sound will return the results for
audio.

Implication - a relationship between two tags where the presence of
the left term necessarily requires the right.  This relationship
reduces menial work.  Low priority.
Example: mpd -> audio.  Adding "mpd" to the package will also add "audio".

Kent, your idea is pretty interesting and I rather like it.
Fortunately, it's completely possible within the context of the basic
flat layout, as I outlined with Alien above.  It probably looks ugly
to you-- this is no illusion; it's pretty ugly.  But it also grants us
the flexibility to get a basic system in place quickly and without a
lot of hassle.  We get 90% of the benefit up front, and can extend it
as necessary.

Unfortunately for "real" hierarchical methods, people still have
difficulty with even simple metadata systems.  Fetch some MP3s off the
internet and check their tags or look at search engine queries and
you'll find an entire class of people hampered by what is currently a
largely alien art.  In the end, this system needs to be usable by
people and by keeping it primarily flat, we ease the conceptual
overhead of its implementation and its use.  If it can't be
implemented on itch-scratching timescales, we have failed.  If people
can't use it with very little learning curve, we have failed.

A word on vocabulary:
As you've no doubt noticed, there seems to be a degree of combinatoric
explosion of tags in the method I propose.  In practical use, it's not
as bad as it looks.  For Gentoo, I'd recommend a basic "canonical"
list of general tags based on the current category system (subject to
discussion and addition/subtraction) and incorporate suggestions like
Kent's as they come up.  It's okay to control the vocabulary.  What
you find is that after the initial implementation, it grows fairly
slowly. (Even with reads_* and writes_* the number will probably be
south of 500 tags for a long time; the current categories dissolve
into about 175 tags from what I can see.)

Regards,
Wyatt

Re: [gentoo-dev] RFC: split up media-sound/ category

[Maciej Mrozowski]

[-- Attachment #1: Type: Text/Plain, Size: 808 bytes --]

On Tuesday 21 of June 2011 16:24:07 Michał Górny wrote:
> Hello,
> 
> As we discussed for a while, the media-sound/ category has grown very
> large and it may be a good idea to split it.
> 
> Right now, it contains audio players, editing software, converters,
> sound systems and a lot of other utilities related to sound. Splitting
> that up would make looking up software easier for users (e.g. if I want
> to take a look at what audio players we have, I don't need to see all
> other programs).
> 
> What do you think? What new category/-ies do you suggest?

I'd suggest to start thinking about dropping broken categories concept and 
introduce (not necessarily replace instantly) flat, but tagged package list 
(so that real vector search on tags can be utilized).

-- 
regards
MM

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Maciej Mrozowski]

[-- Attachment #1: Type: Text/Plain, Size: 1673 bytes --]

On Friday 24 of June 2011 09:55:19 Ciaran McCreesh wrote:
> On Fri, 24 Jun 2011 09:52:03 +0200
> 
> Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com> wrote:
> > You might not like it, but Gentoo categories have always been
> > directories, not words into metadata.xml.
> 
> So tags are in some way related to categories then?

IMHO the best approach is to forget about categories and:

- make package names unique identifiers (it's not that hard, renaming stuff in 
app-xemacs mostly) - categories would serve no purpose as id anymore (though 
may need to be provided as backward compatibility - but with symlinks to 
ebuilds/${PN} inside)

- move such packages into ${PORTDIR}/ebuilds directory (so that identity is 
ensured on filesystem level) - 'ebuilds' name doesn't seem to be reserved 
anywhere so good candidate imho.
To those concerned with directory lookup speed (in order to find package by 
name) - generated package index file provided in ${PORTDIR}

- extend their metadata.xml (no ebuild variables please) with tags in accepted 
format. We should provide dictionary for available tags - necessary in order 
to avoid randomly added system tags - tag could be extended when needed - 
similar policy to global USE flags for instance

- package manager needs to be make aware of tags of course in order to allow 
package list (not tree anymore) searching and filtering
(virtual package tree can be generated from tag - by number of tag occurrences 
in packages - for instance all packages with tag "kde" could be "shown" 
somewhere within "kde" tag subtree etc)

- no tag related symlinks please

-- 
regards
MM

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Kent Fredric]

On 25 June 2011 23:55, Maciej Mrozowski <reavertm@gmail.com> wrote:
>
> IMHO the best approach is to forget about categories and:
>
> - make package names unique identifiers (it's not that hard, renaming stuff in
> app-xemacs mostly) - categories would serve no purpose as id anymore (though
> may need to be provided as backward compatibility - but with symlinks to
> ebuilds/${PN} inside)


I think something else that may be important to consider if one is
eliminating category directories is how we'll replace the utility
currently provided by category/metadata.xml

Some things are simply grossly impractical to maintain individual
metadata.xml for reliably due to volume ( ie: dev-perl/* , last I
looked, the metadata.xml in there presently is largely copy-pasted
between dists )

Perhaps we need a new way to apply metadata to a whole host of packages?

Also, categories have extra use for simple convenience of their native
groupings, ie: I've been known to set USE flags/KEYWORDS that apply to
an entire category.  Trying to make useflags apply to all packages
with a given tagset would be comparatively messy.

categories also make it easy to do Naïve iteration of packages
efficiently, ie: for the most part, if you want to iterate all
perl-modules, you just need to iterate dev-perl and perl-core , and
that is all, you're not bogged down by stepping into all the other
categories, loading all their files and working out whether or not
they're perl related. ( Yes, I am aware this has its own caveats, but
if you know of these caveats and they're acceptable to your task, then
its fine )

the 'virtuals' category also is a bundle of fun. I really do not want
to see virtuals identified only by whatever their unique-idenitifier
might be and the tag 'virtual'. Yuck.




-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] RFC: split up media-sound/ category

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 958 bytes --]

On Sat, 25 Jun 2011 13:55:40 +0200
Maciej Mrozowski <reavertm@gmail.com> wrote:

> On Friday 24 of June 2011 09:55:19 Ciaran McCreesh wrote:
> > On Fri, 24 Jun 2011 09:52:03 +0200
> > 
> > Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com> wrote:
> > > You might not like it, but Gentoo categories have always been
> > > directories, not words into metadata.xml.
> > 
> > So tags are in some way related to categories then?
> 
> IMHO the best approach is to forget about categories and:
> 
> - make package names unique identifiers (it's not that hard, renaming
> stuff in app-xemacs mostly) - categories would serve no purpose as id
> anymore (though may need to be provided as backward compatibility -
> but with symlinks to ebuilds/${PN} inside)

And we'll all be doing a lot of ugly ${PN/python-/} and so on. Simplify
things, not make harder just because of some wannabe tag fashion.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 541 bytes --]

On Sun, 26 Jun 2011 00:22:39 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> Perhaps we need a new way to apply metadata to a whole host of
> packages?
> 
> Also, categories have extra use for simple convenience of their native
> groupings, ie: I've been known to set USE flags/KEYWORDS that apply to
> an entire category.  Trying to make useflags apply to all packages
> with a given tagset would be comparatively messy.

Yep, we dropped old-style virtuals and now we want a new mess.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: split up media-sound/ category

[Wyatt Epp]

On Sat, Jun 25, 2011 at 08:22, Kent Fredric <kentfredric@gmail.com> wrote:
> I think something else that may be important to consider if one is
> eliminating category directories is how we'll replace the utility
> currently provided by category/metadata.xml
>
> Some things are simply grossly impractical to maintain individual
> metadata.xml for reliably due to volume ( ie: dev-perl/* , last I
> looked, the metadata.xml in there presently is largely copy-pasted
> between dists )
>
Looking at the category/metadata.xml, it's a multilingual dictionary
entry and little else.  So make a dictionary of tags (categories).
And what does the latter half have to do with tagging things?  Where's
the maintenance?  There's the overhead of tagging it once, I'll grant.
 And then?  Tags are unlikely to change all that frequently once
they've been added (they don't need to).

> Perhaps we need a new way to apply metadata to a whole host of packages?
>
> Trying to make useflags apply to all packages
> with a given tagset would be comparatively messy.
>
Why do you think that?  The directory-like notation doesn't even need
to be discarded:
perl_module/* ssl

> categories also make it easy to do Naïve iteration of packages
> efficiently, ie: for the most part, if you want to iterate all
> perl-modules, you just need to iterate dev-perl and perl-core , and
> that is all, you're not bogged down by stepping into all the other
> categories, loading all their files and working out whether or not
> they're perl related. ( Yes, I am aware this has its own caveats, but
> if you know of these caveats and they're acceptable to your task, then
> its fine )
>
Or just iterate over the perl_module tag.

> the 'virtuals' category also is a bundle of fun. I really do not want
> to see virtuals identified only by whatever their unique-idenitifier
> might be and the tag 'virtual'. Yuck.
>
In the first place, it's still no different: mysql (the virtual) pulls
in db-mysql (or "charles" or whatever name sounds good) whatever else
is available.  Or, as I mentioned before, while unique identifiers are
really terribly simple, we are fully capable of working around the
lack of that feature.  What prevents virtual/mysql from pulling in
database/mysql?

Regards,
Wyatt

[gentoo-dev] Re: RFC: split up media-sound/ category

[Duncan]

Maciej Mrozowski posted on Sat, 25 Jun 2011 13:55:40 +0200 as excerpted:

> On Friday 24 of June 2011 09:55:19 Ciaran McCreesh wrote:

>> So tags are in some way related to categories then?
> 
> IMHO the best approach is to forget about categories and:
> 
> - make package names unique identifiers (it's not that hard, renaming
> stuff in app-xemacs mostly) - categories would serve no purpose as id
> anymore (though may need to be provided as backward compatibility - but
> with symlinks to ebuilds/${PN} inside)

What a beautiful bikeshed we're debating! =:^p

> - move such packages into ${PORTDIR}/ebuilds directory (so that identity
> is ensured on filesystem level) - 'ebuilds' name doesn't seem to be
> reserved anywhere so good candidate imho.
> To those concerned with directory lookup speed (in order to find package
> by name) - generated package index file provided in ${PORTDIR}

Alternatively, just use first letter subdivisions.  Perhaps grouping them 
as ac, df, etc, or whatever granularity seems appropriate, if desired.  
That's a common method of eliminating large-dir issues with otherwise 
flat listings.

> - extend their metadata.xml (no ebuild variables please) with tags in
> accepted format. We should provide dictionary for available tags -
> necessary in order to avoid randomly added system tags - tag could be
> extended when needed - similar policy to global USE flags for instance

Keep in mind that there has historically been extremely high resistance 
to "xml-ifying" anything critical to operational package management, by 
certain highly respected and politically influential gentoo devs.  
There's a reason metadata.xml contains only "ancillary" data, while the 
most important operational data (depends, inherits, src_uri, etc) remains 
as variables within the ebuilds and/or eclasses.

I never tracked who was so stridently opposed and it may well be that 
they've retired now, but there's some people who simply don't consider xml 
a sufficiently robust solution in terms of parsing dependencies AND easy 
error-free human parsability.

FWIW, I agree that it'd be a step backward in terms of human editability 
ease and thus I'd find it a sad day were that to happen, but my feelings 
aren't particularly strong on the issue.

But if packages are indeed uniquely and canonically identified by name 
only and tags are kept as ancillary to the core merge process as 
metadata.xml is now, there shouldn't be a problem with it.

Just a warning that "here be dragons" for anyone thinking about going 
down that path.  Consider reading up in the list archive for past debates 
on the subject.

> - package manager needs to be make aware of tags of course in order to
> allow package list (not tree anymore) searching and filtering (virtual
> package tree can be generated from tag - by number of tag occurrences in
> packages - for instance all packages with tag "kde" could be "shown"
> somewhere within "kde" tag subtree etc)

As long as it's kept out of critical operational functionality... but 
this seems to be getting pretty close, if the "package manager needs to 
be [made] aware of tags".  This would have been unlikely to have gotten 
thru at one point... but as I said, it's possible that opposition isn't 
any longer a factor.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Maciej Mrozowski]

[-- Attachment #1: Type: Text/Plain, Size: 3921 bytes --]

On Saturday 25 of June 2011 17:19:47 Duncan wrote:
> Maciej Mrozowski posted on Sat, 25 Jun 2011 13:55:40 +0200 as excerpted:
> > On Friday 24 of June 2011 09:55:19 Ciaran McCreesh wrote:
> >> So tags are in some way related to categories then?
> > 
> > IMHO the best approach is to forget about categories and:
> > 
> > - make package names unique identifiers (it's not that hard, renaming
> > stuff in app-xemacs mostly) - categories would serve no purpose as id
> > anymore (though may need to be provided as backward compatibility - but
> > with symlinks to ebuilds/${PN} inside)
> 
> What a beautiful bikeshed we're debating! =:^p

No bikeshedding, just any mean necessary (I'd be fine with anything) in order 
to phase out categories from being necessary for critical package manager 
operations.

> > - move such packages into ${PORTDIR}/ebuilds directory (so that identity
> > is ensured on filesystem level) - 'ebuilds' name doesn't seem to be
> > reserved anywhere so good candidate imho.
> > To those concerned with directory lookup speed (in order to find package
> > by name) - generated package index file provided in ${PORTDIR}
> 
> Alternatively, just use first letter subdivisions.  Perhaps grouping them
> as ac, df, etc, or whatever granularity seems appropriate, if desired.
> That's a common method of eliminating large-dir issues with otherwise
> flat listings.

Using directory structure as a way to enhance performance is sign of bad 
design. Simple
find /usr/portage/ebuilds -type d -maxdepth 1 | sort > ebuilds.index should be 
sufficient. One can even extract tags in that file and list them after package 
name for faster searching.

> > - extend their metadata.xml (no ebuild variables please) with tags in
> > accepted format. We should provide dictionary for available tags -
> > necessary in order to avoid randomly added system tags - tag could be
> > extended when needed - similar policy to global USE flags for instance
> 
> Keep in mind that there has historically been extremely high resistance
> to "xml-ifying" anything critical to operational package management, by
> certain highly respected and politically influential gentoo devs.
> There's a reason metadata.xml contains only "ancillary" data, while the
> most important operational data (depends, inherits, src_uri, etc) remains
> as variables within the ebuilds and/or eclasses.

Yes, and the reason is metadata.xml can contain only version invariant data 
and should not contain anything that's required for ebuild.sh. So inherits, 
src_uris, dependencies - cannot be placed there. Assuming package names are 
unique identifiers, tags are not necessary to be available for ebuild.sh so 
metadata.xml is the best place.

> I never tracked who was so stridently opposed and it may well be that
> they've retired now, but there's some people who simply don't consider xml
> a sufficiently robust solution in terms of parsing dependencies AND easy
> error-free human parsability.
[...]

Let's not diverge this purely technical topic to "who thought what on what 
based on sth" or "there are some people who don't consider xml..." and let 
them speak on technical matters if they want.

> FWIW, I agree that it'd be a step backward in terms of human editability
> ease and thus I'd find it a sad day were that to happen, but my feelings
> aren't particularly strong on the issue.
> 
> But if packages are indeed uniquely and canonically identified by name
> only and tags are kept as ancillary to the core merge process as
> metadata.xml is now, there shouldn't be a problem with it.

No, tags are no supposed to be critical for package manager operations.
Package manager needs to be aware of them in order to provide useful 
searching, but that's about it.

I think we'd just need some simplest proof of concept implementation...

-- 
regards
MM

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Ulrich Mueller]

>>>>> On Sat, 25 Jun 2011, Maciej Mrozowski wrote:

> Assuming package names are unique identifiers, tags are not
> necessary to be available for ebuild.sh so metadata.xml is the best
> place.

But we know that package names are _not_ unique. There are many cases
in the Portage tree where two or even more packages have the same
name. Categories are there to avoid such collisions.

With multiple overlays/repositories instead of one monolithic Portage
tree, the collision issue gets even worse if you have a flat
namespace.

Ulrich

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Maciej Mrozowski]

[-- Attachment #1: Type: Text/Plain, Size: 1875 bytes --]

On Saturday 25 of June 2011 19:29:58 Ulrich Mueller wrote:
> >>>>> On Sat, 25 Jun 2011, Maciej Mrozowski wrote:
> > Assuming package names are unique identifiers, tags are not
> > necessary to be available for ebuild.sh so metadata.xml is the best
> > place.
> 
> But we know that package names are _not_ unique. There are many cases
> in the Portage tree where two or even more packages have the same
> name. Categories are there to avoid such collisions.

But we also know, that making package names unique is first step to take as I 
already noted in my first post in this thread. It's not that current package 
naming scheme should be an unfixable obstacle preventing us from getting rid 
of pointless categories (yes, every pkgmove in tree renders categories concept 
broken by design, sorry to state this fact brutally).

As far as app-xemacs is concerned (and probably why you commented here), it 
should be sufficient to prepend "xemacs-" to package names from app-xemacs 
category in order to make them distinguished from the rest.
It would be elegant and correct - after all when you "emerge ocaml" you don't 
expect to be installing objective caml mode for Emacs, but ocaml interpreter 
itself.

> With multiple overlays/repositories instead of one monolithic Portage
> tree, the collision issue gets even worse if you have a flat
> namespace.

Every not Gentoo-based distro can live with unique package names, somehow 
Gentoo is not able to? Colour me surprised.

Btw, in above, I specifically proposed those unique packages to be placed in 
${PORTDIR}/ebuilds/ because when 'ebuilds' is considered like a fake category' 
- existing atom syntax can be used and so can be current package manager 
implementation (even with not entirely converted package tree, except 
uniqueness is not checked in such case).

-- 
regards
MM

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Kent Fredric]

On 26 June 2011 05:29, Ulrich Mueller <ulm@gentoo.org> wrote:
>>>>>> On Sat, 25 Jun 2011, Maciej Mrozowski wrote:
>
>> Assuming package names are unique identifiers, tags are not
>> necessary to be available for ebuild.sh so metadata.xml is the best
>> place.
>
> But we know that package names are _not_ unique. There are many cases
> in the Portage tree where two or even more packages have the same
> name. Categories are there to avoid such collisions.
>
> With multiple overlays/repositories instead of one monolithic Portage
> tree, the collision issue gets even worse if you have a flat
> namespace.
>
> Ulrich
>
>

At present, this exists because we use categories as a the primary way
for a *user* to find the package. Our current collision avoidance
strategy is targeted at not confusing our *user*.

However, in the proposed strategy, package names themselves are not
*users* primary interface. "tags" and other metadata are intended as
the users primary interface.

Package names themselves can be thusly arbitrary , and could be a SHA
sum or something obscure, as long as all internals and dependencies
used the same arbitrary name, things would work as intended.

There is one remaining downside to the flat topology however, and that
is it may hamper our move to git.

I was thinking that what could be done is have seperate submodules or
whathave you for various categories to somewhat ease the load of "A
full checkout of the tree going back for all time can be bloody huge
and slow" , but without categorical subtrees that approach will be
less viable.

Although, this what currently seems like a disadvantage could be used
to an advantage perhaps, with the possible idea of "meta trees" of
some description. If we relinquish the hold we have on symlinks, a few
interesting options become available.

Different Herds could have their own subtrees in

/projects/herd/<x>

and a tool could be used to symlink the contents of herd specific
subtrees to the ebuilds folder.

/ebuild/<pn>   --> /projects/herd/<herdname>/<pn>

And the tool can inform herds when they add a new package that
conflicts with the name of an existing one so it can be disambiguated
before the tree propagates to users.

Continuing on that line of thought and you get even more interesting
ideas, like introducing a "merge mask" file, which allows people to
work on stuff in the herd tree , and indicate that their
files/packages are not fit for integration with the main-tree yet,
somewhat bridging the gap we presently have between "Development"
overlays and the current main tree.

This could in turn make collaboration even easier, as dev branches
will be able to go nuts with all sorts of random contributions, and
when its deemed fit for public consumption and testing remove the
package from the merge mask and its there.

</early morning coffee fuelled idea session>

-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Wyatt Epp]

On Sat, Jun 25, 2011 at 21:47, Kent Fredric <kentfredric@gmail.com> wrote:
> Package names themselves can be thusly arbitrary , and could be a SHA
> sum or something obscure, as long as all internals and dependencies
> used the same arbitrary name, things would work as intended.
>
I mentioned this idea of internally referencing packages by a hash in
the other thread.  As long as we're clear that the most common
operation (emerge -av ${PN}) is still exposed to the user, it's
perfectly valid.  I want to be very sure we're clear in our
understanding that tags are for discovery in cases where the user is
not sure what is available (like categories).

As for the latter part, the size of a git repo becoming umanageable
over time had not occurred to me, I'm afraid-- would it work to use
shallow clones?  Otherwise, the herd-wise division is probably
acceptable.  Need to think about that one more.

Regards,
Wyatt

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Kent Fredric]

On 26 June 2011 15:49, Wyatt Epp <wyatt.epp@gmail.com> wrote:
> As for the latter part, the size of a git repo becoming umanageable
> over time had not occurred to me, I'm afraid-- would it work to use
> shallow clones?  Otherwise, the herd-wise division is probably
> acceptable.  Need to think about that one more.


  --depth <depth>
           Create a shallow clone with a history truncated to the specified
           number of revisions. A shallow repository has a number of
           limitations (you cannot clone or fetch from it, nor push from nor
           into it), but is adequate if you are only interested in the recent
           history of a large project with a long history, and would want to
           send in fixes as patches.

It would be ok perhaps for non-contributing users to use shallow
clones, but in my understanding, shallow clones limit you to doing
what you could do with a tar file of the specified revision, which
basically makes it impractical for people who are developing on it,
and would mean every new developer would get a progressively longer
time in order to do a complete check out.

( Unless of course we had some sort of periodic refresh where history
was discarded/rebased into nonexistence , but that is really the same
problem with different faces )

-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Jesús J. Guerrero Botella]

I am really amazed that someone didn't want to use links (a solution
with next to zero work involved) because they are not available in
fat32 (as if fat32 was relevant at all for us) but then people is
suggesting that we should put everything into a flat folder and use
tags. Well, good look getting more than 65k files into a fat32 folder.
Isn't that incongruence?

Plus, if you truly do that, this should be called xmltagage, beucase
it won't any longer resemble the bsd ports system at all.
-- 
Jesús Guerrero Botella

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Kent Fredric]

2011/6/26 Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com>:
> I am really amazed that someone didn't want to use links (a solution
> with next to zero work involved) because they are not available in
> fat32 (as if fat32 was relevant at all for us) but then people is
> suggesting that we should put everything into a flat folder and use
> tags.

Well, the difference being "Symlinks are only really surplus utilities
that make it easier for users" and "Some degree of backcompat" instead
of "Core Functionality that will be required to make the code work".

In theory, if you have the "most recent" versions of your package
manager, after this change takes place, you will not need any symlinks
at all, and functionality should still be retained if they were blown
out completely.





-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Patrick Lauer]

On 06/25/11 21:42, Maciej Mrozowski wrote:
> On Saturday 25 of June 2011 19:29:58 Ulrich Mueller wrote:
>>>>>>> On Sat, 25 Jun 2011, Maciej Mrozowski wrote:
>>> Assuming package names are unique identifiers, tags are not
>>> necessary to be available for ebuild.sh so metadata.xml is the best
>>> place.
>>
>> But we know that package names are _not_ unique. There are many cases
>> in the Portage tree where two or even more packages have the same
>> name. Categories are there to avoid such collisions.
> 
> But we also know, that making package names unique is first step to take as I 
> already noted in my first post in this thread. It's not that current package 
> naming scheme should be an unfixable obstacle preventing us from getting rid 
> of pointless categories (yes, every pkgmove in tree renders categories concept 
> broken by design, sorry to state this fact brutally).

I disagree. If I put postgresql in x11-libs that's just wrong, and then
you fix it with a package move. Doesn't mean the category system is
broken, just means that it was in the wrong place at the wrong time.

> 
> As far as app-xemacs is concerned (and probably why you commented here), it 
> should be sufficient to prepend "xemacs-" to package names from app-xemacs 
> category in order to make them distinguished from the rest.
> It would be elegant and correct - after all when you "emerge ocaml" you don't 
> expect to be installing objective caml mode for Emacs, but ocaml interpreter 
> itself.

Please don't do that. It leads to the kind of funny package names that
some legacy distros carry around, which make it exquisitely hard to
guess what their intent is.

Right now "emerge openoffice" does what I want. Don't change that to
"openoffice-core-core" or other aneurisms that can be found in the wild.
Do not try to make me do extra work, that's rude and inconsiderate :)

> 
>> With multiple overlays/repositories instead of one monolithic Portage
>> tree, the collision issue gets even worse if you have a flat
>> namespace.
> 
> Every not Gentoo-based distro can live with unique package names, somehow 
> Gentoo is not able to? Colour me surprised.

Most other distros also live without the option of downgrades, without a
working package search infrastructure and without working init scripts.
Do we want to define ourselves through the features we don't have?!

And actually we do have unique package names, just that we don't
obfuscate with random distractions but with a mostly working category
system. So think what you are trying to fix, and no, XML is not the
answer :)

> 
> Btw, in above, I specifically proposed those unique packages to be placed in 
> ${PORTDIR}/ebuilds/ because when 'ebuilds' is considered like a fake category' 
> - existing atom syntax can be used and so can be current package manager 
> implementation (even with not entirely converted package tree, except 
> uniqueness is not checked in such case).
> 
How about we remove slots too, that's only confusing, so you get a
distinct unique package postgres-8.3, postgres-8.4, postgres-9.0 - and a
meta package "postgres" that depends on any of those. As an upside we
roughly double the amount of packages we have, and our dependencies get
so much more ... OMG ... nooo ... what a nightmare.

So again, what are you trying to fix, and what makes you think it was
broken to start with?

-- 
Patrick Lauer         http://service.gentooexperimental.org

Gentoo Council Member and Evangelist
Part of Gentoo Benchmarks, Forensics, PostgreSQL, KDE herds

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Kent Fredric]

On 26 June 2011 21:53, Patrick Lauer <patrick@gentoo.org> wrote:
>
> I disagree. If I put postgresql in x11-libs that's just wrong, and then
> you fix it with a package move. Doesn't mean the category system is
> broken, just means that it was in the wrong place at the wrong time.
>
>>
>> As far as app-xemacs is concerned (and probably why you commented here), it
>> should be sufficient to prepend "xemacs-" to package names from app-xemacs
>> category in order to make them distinguished from the rest.
>> It would be elegant and correct - after all when you "emerge ocaml" you don't
>> expect to be installing objective caml mode for Emacs, but ocaml interpreter
>> itself.
>
> Please don't do that. It leads to the kind of funny package names that
> some legacy distros carry around, which make it exquisitely hard to
> guess what their intent is.
>
> Right now "emerge openoffice" does what I want. Don't change that to
> "openoffice-core-core" or other aneurisms that can be found in the wild.
> Do not try to make me do extra work, that's rude and inconsiderate :)
>
>>
> And actually we do have unique package names, just that we don't
> obfuscate with random distractions but with a mostly working category
> system. So think what you are trying to fix, and no, XML is not the
> answer :)
>

I have to agree for the most part, I mean, ultimately were' going to
have a bit of "convention" for the sake of convenience.

If you want to see what a flat portage system would look like, just do
ls /usr/bin/

A lot of thing that are perl modules will inevitably, in a flat
topology, be prefixed with "perl-" , like so many other distros do (
and its ugly ,  debian's   liblocal-libperl  for local::lib is an
example of this catastrophe )

Why do we need to devine a /new/ convention for giving packages a
unique name when we have one that is *presently* working just fine. (
for the purpose of uniquely defining a package that is, its failing in
a few places for reasons related to discoverability, but I'm no longer
arguing that ).

the difference between

dev-perl-local-lib

and

dev-perl/local-lib

is really not that big in practice, but one sucks infinitely more than
the other.

With this "new" system however, pkgmoves will be a thing of the past,
even if we keep the legacy category system around.

pkgmoves take place in my understanding, largely because the category
they're in makes them proves hard to find for discoverability
purposes.

In the new system, categories are no longer part of the discovery
system, and exist soley as a method to keep the files organised with
distinct names.

ie: Currently, were' proposing moving some things around  , splitting
media-sound into several sub categories, ie:

sound-players (and maybe sound-radio)
sound-editors
sound-plugins
sound-libs

"Flattening" does not in any way solve this issue in the least,
because in the case of a flattening, the difference would be primarily
that you have a load of packages with unuseful prefixes instead of a
directory, so you'd be doing

pkgmove media-sound-amarok   sound-players-amarok

and you would still have to go and update all the dependencies etc.

And I think you'll agree that that sounds a little daft.

So in the new system, you put a package in a category that makes some
degree of sense, and then leave it there, for all eternity, and rely
on tags to make them discoverable instead of moving things around or
renaming them.

> Right now "emerge openoffice" does what I want. Don't change that to
> "openoffice-core-core" or other aneurisms that can be found in the wild.
> Do not try to make me do extra work, that's rude and inconsiderate :)

I'd hope that with a new tag based system, you could "emerge
openoffice" and get told "Yeah, I know that once upon a time there was
only 1 openoffice, but now there's more than one, did you mean
openoffice-org or libreoffice?"

Same goes for the various forms of virtualbox ( did you mean
virtualbox from source, virtualbox binary, the opensource edition or
the commercial edition etc )  java ( did you mean 'sun-jdk?
blackdown-jdk? '  etc )  flash ( did you mean adobe-flash? ) xine (
did you mean xine-ui or xine-lib? , or perhaps even gxine ? )  and a
host of other things that are presently like this.



-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Rich Freeman]

On Sun, Jun 26, 2011 at 5:53 AM, Patrick Lauer <patrick@gentoo.org> wrote:
> So again, what are you trying to fix, and what makes you think it was
> broken to start with?

Well, I think there are things worth improving.  However, I'm not sure
that we should consider implementation of tagging a reason to
re-design the entire portage system.

I think the simplest way to go about tagging is to just stick tags in
metadata.xml and not change the way anything else works fundamentally.
 I could see a little room for allowing for future expansion, such as
defining namespaces in the xml.  Then you could have a set of
description tags, and later a set of file tags (a la portage file
search).

Lack of tools to do something with the tags isn't a problem - they'll
happen.  Probably the first place to do something with tags would be
packages.gentoo.org - if I want a quick list of all the text editors
on the system I don't mind doing a quick web search.

Sure, we can later expand things to provide a metadata index of some
kind and then command-line tools to search tags, or maybe a program
that sets up a tree full of symlinks or whatever after doing a sync.
I don't see why these have to be fully worked out to implement the
concept.

I think we should avoid changing the fundamental design of portage,
such as removing categories or allowing tags to be used as
dependencies/etc.  At least, not right now.  If we set up namespaces
for tags that might allow for such a thing in the future.

Also, there is no reason that tools couldn't help maintain the tags -
certainly if we used a tag namespace to list the files installed by a
package we'd want to automate that.  I don't think we should try to
tackle that out of the gates - for starters the list is
version-specific and not package-specific, so you'd either need to
figure out how to sanely build and maintain a union of everything, or
tag tags with package versions or something.

The main driver behind tags seems to be searchability, and I think
that is something we could easily improve.  I don't think we should
hold that up over an initiative to completely re-architect Gentoo...

Rich

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Kent Fredric]

On 26 June 2011 23:40, Rich Freeman <rich0@gentoo.org> wrote:
>
> I think we should avoid changing the fundamental design of portage,
> such as removing categories or allowing tags to be used as
> dependencies/etc.  At least, not right now.  If we set up namespaces
> for tags that might allow for such a thing in the future.

At this time I vehemently oppose the idea of using tags for dependencies.

If there was even a good reason to do this, I would probably want to
see the system working really well first.

> The main driver behind tags seems to be searchability, and I think
> that is something we could easily improve.

+1

> I don't think we should
> hold that up over an initiative to completely re-architect Gentoo...

+1

-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 382 bytes --]

On Sun, 26 Jun 2011 22:13:24 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> With this "new" system however, pkgmoves will be a thing of the past,
> even if we keep the legacy category system around.

Nope. Package names can still change, and I really dislike the idea of
keeping some magical hashes or deprecated names in @world.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Jorge Manuel B. S. Vicetto]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 26-06-2011 12:23, Kent Fredric wrote:
> On 26 June 2011 23:40, Rich Freeman <rich0@gentoo.org> wrote:
>>
>> I think we should avoid changing the fundamental design of portage,
>> such as removing categories or allowing tags to be used as
>> dependencies/etc.  At least, not right now.  If we set up namespaces
>> for tags that might allow for such a thing in the future.
> 
> At this time I vehemently oppose the idea of using tags for dependencies.
> 
> If there was even a good reason to do this, I would probably want to
> see the system working really well first.
> 
>> The main driver behind tags seems to be searchability, and I think
>> that is something we could easily improve.
> 
> +1
> 
>> I don't think we should
>> hold that up over an initiative to completely re-architect Gentoo...
> 
> +1

I agree with the above.

I've done my best to stay out of this discussion until now, but I see
people going so far "out of the box", that I fear if we were to do this
we would end up "breaking the box".

- From past discussions about tags, I'd also like to recall a few ideas
that have been completely ignored on this discussion:

 * tags should only be important for searching or classifying packages
 * tags should exist beyond portage / mostly use an on-line system
 * tags should be usable and maintainable by users
 * users should have a way to contribute directly to tags

That's why in the past the discussion pointed to using an online system
that users could contribute directly without having to wait on
developers answering their requests, why it was seen as a manual system
and why it didn't affect the package managers or tree.
Being able to combine tags on metadata.xml for the tree and overlays
with an on-line tagging system and have the tools use that data when
searching seems an interesting option.

- -- 
Regards,

Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / KDE / Elections / RelEng
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCAAGBQJOBy3dAAoJEC8ZTXQF1qEPrJkP/2HhRwwj4GuhRyz7bXi3g9/U
osmGEKj1f7dcLLCGNiS9AQ6aiCcOREXtq2uWkiMJHhILqfhm1bTLI/2OF7Vcmslh
ImE5KzDlBfITb2Q6KxBppDfCwgYzclVKqMtyaeLq84O1PSmUn1tEaWaHzXlkVHI8
Uiae5rYSq/ikfbt1wQXXwQ3LBcgGbjfr8AWGvXTdnB9drGoNCezEfQaizuTuN6ib
1DmVj9MiHiQ+9+ixSCzJGJ7XryfEX1qEdWkn+rGqmN9CXXBQjjXOvHTVhIHbixQA
6+LWrVL/qjrzCUZ6Nsapgb5SbK+BQZoHD01oLs2Em7gum2+K7Z+gIiKVN5rUncaQ
ExcIyOdnyZb8dyXYR/purSTEuTJv2vM6z6/EbV3XF4NsKfY8DQRq5x1h8+6ihMzE
G/UT5Hqg8px0hbaWe4fHGNUzPaX8meu791Kw+GJd27Z2BJaZjP8VK5NSk0S3Y7XM
SE0PNMdgLPBfcTK2Qw5WisH0DPwzfFWUoulCfvHGcZ/dfYC3SuideCfouHgkE8rT
K7Y45c0h0A9QhEDSjRG3NFQzNU8tFfYvM8uy0GZr81SVMA1qFLvlVWR51kyGs/of
a92wNdW0HqyN6nXTRLGcj8lFnv23Cqy0yi7Ya59Vkh1O0PMhIzl4GyghfnuszwYe
sXzbYFYbsOMfY1tI/XCZ
=OJ8M
-----END PGP SIGNATURE-----

[gentoo-dev] Re: RFC: split up media-sound/ category

[Duncan]

Kent Fredric posted on Sun, 26 Jun 2011 17:43:27 +1200 as excerpted:

> On 26 June 2011 15:49, Wyatt Epp <wyatt.epp@gmail.com> wrote:
>> As for the latter part, the size of a git repo becoming umanageable
>> over time had not occurred to me, I'm afraid-- would it work to use
>> shallow clones?  Otherwise, the herd-wise division is probably
>> acceptable.  Need to think about that one more.
> 
> 
>   --depth <depth>
>            Create a shallow clone with a history truncated to the
>            specified number of revisions. A shallow repository has a
>            number of limitations (you cannot clone or fetch from it, nor
>            push from nor into it), but is adequate if you are only
>            interested in the recent history of a large project with a
>            long history, and would want to send in fixes as patches.
> 
> It would be ok perhaps for non-contributing users to use shallow clones,
> but in my understanding, shallow clones limit you to doing what you
> could do with a tar file of the specified revision, which basically
> makes it impractical for people who are developing on it,
> and would mean every new developer would get a progressively longer time
> in order to do a complete check out.

Not substantially so, no.

FWIW, git scales VERY well in this regard, provided it's used for text-
based content (sources) as originally intended.  (It's not so hot at 
binary blob management, but it's not designed for that.  Fortunately, 
gentoo's usage would be nearly 100% text-based.)

What git does over time is compress the diffs into a series of packages 
(tarballs or whatever, I don't know the internals), and text compresses 
REALLY well.  Then new checkouts grab the compressed packages, with only 
the last little bit being uncompressed.  Existing users can run garbage-
collection periodically to collect and compress their existing history 
into the packages as well.

So for example, du says my kernel git tree totals 1.6 GB, including the 
active checkout and two separate (dirty) build trees.  The bare git tree 
(history repo without working tree) itself is 891 MB.  So the bare repo 
is only 54% of the total, and I've not actually garbage-collected in some 
time.  If I had, the ratio would be closer to 50%, meaning the entire 
kernel git history repo compresses to roughly the size of the working 
tree, and only roughly doubles the size of a single decompressed working 
tarball.

Over time that'll certainly grow a bit, but it really does scale well.  
The kernel has been in git for enough time now that there's quite some 
history built up, and that it only roughly doubles the size of a single 
decompressed working tree snapshot, while making available at my 
fingertips the entire history since original checkin, is impressive 
indeed.

It's all down to how well the sources and diffs compress.  If there were 
significant binary blobs in there (the kernel tree does have a few bits 
of firmware, the tux logo, etc), it would compress far less effectively.  
But gentoo's tree is pretty much all text as well, fortunately. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] RFC: split up media-sound/ category

[Zac Medico]

On 06/24/2011 12:52 AM, Jesús J. Guerrero Botella wrote:
> 2011/6/24 Zac Medico <zmedico@gentoo.org>:
>> On 06/22/2011 11:15 PM, Jesús J. Guerrero Botella wrote:
>>> Symlinks are clean, and portage has
>>> always been file-oriented so I see no problem with using them for
>>> this. All we need is to deference the symlink to find the real name of
>>> the package and put it in world instead of the symlinked name, so the
>>> rest of packages won't even need to be retouched to fix the
>>> dependencies. I don't really know if it's that simple as it sounds,
>>> but it's an idea.
>>
>> For some reason, using symlinks to represent tags seems like an odd
>> approach to me. I think it would be much more sensible to put them in
>> metadata.xml or an ebuild variable. If for some reason you want symlinks
>> representing the tags (I don't know why you would), you can always use a
>> script to generate symlinks from metadata.xml files.
> 
> You might not like it, but Gentoo categories have always been
> directories, not words into metadata.xml. Most portage tools rely on
> that. Not a strong argument, I know that. But someone used this
> argument when someone else wanted to put portage into a database
> instead of an fs-based tree. That was long ago, admittedly, don't know
> if that conversation ever came up again.

I see, so you want to optimize the tree layout for use with simple shell
commands. For a shell-friendly alternative to metadata.xml, I suppose
that we could instead use a plain text file called "tags" in the same
directory as metadata.xml. If it listed one tag per line, you could use
a simple shell command like this to search for packages with a specific tag:

   find /usr/portage -name tags | xargs grep <tag name>

> I've personally never bothered to learn how to use external tools
> anyway, so I just navigate the tree using command line tools when I
> need to know something about a given package. I am sure I am not alone
> in that regard. I guess I could also "nano metadata.xml", ugh!

Well, I just assumed that most people tend to use programs like eix,
esearch, or some kind of web interface to perform searches. It hadn't
occurred to me that people would resort to ad-hoc shell commands for
this. Sometimes I'll use shell wildcard tricks like `echo
/usr/portage/*/gcc` if I'm not sure about the category of a package, or
grep tricks like `grep -r gcc /usr/portage/metadata/cache`, but other
than that I tend to use domain-specific search tools like esearch.

> Some portage GUIs also use this categorization scheme, like portato or
> porthole (not that they are important at all, but they illustrate the
> trend).
> 
> Maybe it's just my mind model is archaic, but I can't really agree
> with tagging for massive trees. I wouldn't drop all my 40 thousand
> songs into a single folder and rely on tagging to keep them at hand.
> Portage has way more files so I don't see how tagging would be better
> for it than it would be for my music collection. I might be too much
> influenced by *nix (and DOS) OSes at this stage to be able to see the
> advantages of tagging (besides the decorative function), I admit.

Well, I imagine that the vast majority of people would be inclined to
use domain-specific search toola rather than shell commands for searches
involving tags.
-- 
Thanks,
Zac

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Jesús J. Guerrero Botella]

2011/6/26 Kent Fredric <kentfredric@gmail.com>:
> 2011/6/26 Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com>:
>> I am really amazed that someone didn't want to use links (a solution
>> with next to zero work involved) because they are not available in
>> fat32 (as if fat32 was relevant at all for us) but then people is
>> suggesting that we should put everything into a flat folder and use
>> tags.
>
> Well, the difference being "Symlinks are only really surplus utilities
> that make it easier for users" and "Some degree of backcompat" instead
> of "Core Functionality that will be required to make the code work".
>
> In theory, if you have the "most recent" versions of your package
> manager, after this change takes place, you will not need any symlinks
> at all, and functionality should still be retained if they were blown
> out completely.

The real question is why something that has been into POSIX since ever
can't be used for something that has also always been fs-based since
it was born (that's "portage").

I an not saying that this other system doesn't have it's advantages. I
am just saying that if I wanted a db-like-but-not-db-based system I'd
probably reinvent any .deb or .rpm based distro, rather than retouch
something based (even marginally) on BSD ports.

That still doesn't answer my question anyway: both features (symlinks
and +65k files on a single dir) are incompatible with fat32. And
someone said fat32 compatibility is a feature we want (still can't
guess why, but well, be consequent...). Obviously, we want fat32
compatibility when it comes to arguing against symlinks, which have
always been with us by the way, but that's not important when we talk
about other things that are not compatible with fat32.

Links seem to look not so elegant today, but we use them everyday for
eselect, /etc and even in the handbook. but Oh!, They are not that
good for portage.
-- 
Jesús Guerrero Botella

Re: [gentoo-dev] RFC: split up media-sound/ category

[Jesús J. Guerrero Botella]

2011/6/27 Zac Medico <zmedico@gentoo.org>:
> On 06/24/2011 12:52 AM, Jesús J. Guerrero Botella wrote:
>> 2011/6/24 Zac Medico <zmedico@gentoo.org>:
>>> On 06/22/2011 11:15 PM, Jesús J. Guerrero Botella wrote:
>>>> Symlinks are clean, and portage has
>>>> always been file-oriented so I see no problem with using them for
>>>> this. All we need is to deference the symlink to find the real name of
>>>> the package and put it in world instead of the symlinked name, so the
>>>> rest of packages won't even need to be retouched to fix the
>>>> dependencies. I don't really know if it's that simple as it sounds,
>>>> but it's an idea.
>>>
>>> For some reason, using symlinks to represent tags seems like an odd
>>> approach to me. I think it would be much more sensible to put them in
>>> metadata.xml or an ebuild variable. If for some reason you want symlinks
>>> representing the tags (I don't know why you would), you can always use a
>>> script to generate symlinks from metadata.xml files.
>>
>> You might not like it, but Gentoo categories have always been
>> directories, not words into metadata.xml. Most portage tools rely on
>> that. Not a strong argument, I know that. But someone used this
>> argument when someone else wanted to put portage into a database
>> instead of an fs-based tree. That was long ago, admittedly, don't know
>> if that conversation ever came up again.
>
> I see, so you want to optimize the tree layout for use with simple shell
> commands. For a shell-friendly alternative to metadata.xml, I suppose
> that we could instead use a plain text file called "tags" in the same
> directory as metadata.xml. If it listed one tag per line, you could use
> a simple shell command like this to search for packages with a specific tag:
>
>   find /usr/portage -name tags | xargs grep <tag name>

I still don't understand why

A) you need to build a project, a glep, whatever the course of action
is, I am bad at bureaucracy.
B) you need to code the solution, to fix What?
C) "ls $PORTDIR/whatever-category" is a command that's way simpler
than the one you posted.

XML seems to be the trend, but we should really think a moment, what's
what we are trying to fix?

We just needed to add some categories or rename them when someone
started this thread, but now, even when we know we are lacking dev
power in some areas we start arguing that the base concept of our OS
(portage) is wrong, and that we should redo it completely by putting
every ebuild into a directory and tagging them. Again, that's not
"port-age". Read on ports:

http://en.wikipedia.org/wiki/FreeBSD_Ports

I don't even use tags for my music collections and now I am going to
be forced to use them to operate my OS. We might even end having
something like

<portage>
<ebuild>
.......................
</ebuild>
<ebuild>
.......................
</ebuild>
..
.
.
.
.

</portage>

 :P

Or

<fs>
<dir>boot</dir>
<dir>usr</dir>
<dir>lib</dir>
<dir>home</dir>
</fs>

genpatches could also remove symlink stuff from the kernel, since it's
taking precious memory that could be used for the /proc.xml file. Yes,
feeling sarcastic, but I am really trying to understand what's what we
need to fix today.

-- 
Jesús Guerrero Botella

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Wyatt Epp]

2011/6/27 Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com>:
> That still doesn't answer my question anyway: both features (symlinks
> and +65k files on a single dir) are incompatible with fat32. And
> someone said fat32 compatibility is a feature we want (still can't
> guess why, but well, be consequent...). Obviously, we want fat32
> compatibility when it comes to arguing against symlinks, which have
> always been with us by the way, but that's not important when we talk
> about other things that are not compatible with fat32.

I'm not sure where you're getting 65k files.  Unless I misinterpreted
everything everyone else was saying, every package would still have
its own directory.  There are fewer than 20k even with a bunch of
overlays installed.  Regardless, you might check the other (other)
thread; I think we're probably going to go quick and
not-necessarily-dirty with sets to get 99% of what we're looking for
almost trivially.

2011/6/27 Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com>:
> C) "ls $PORTDIR/whatever-category" is a command that's way simpler
> than the one you posted.
>
It's also fundamentally broken because one package can only be in one
category and their expansion has not historically been speedy.  Tags
are a non-exclusive one-to-many relationship.  So a package can have
as many tags as it needs, and users will be able to leverage tags
alone or in combinations to find things they want or need.

> I don't even use tags for my music collections

That's very curious, and I wouldn't mind talking about why that is
off-list (not quite joking; that's really interesting).

So to sum it up, we're fixing package navigation and discovery because
Gentoo is about choice.  Even 15,000 packages is too many to have to
play "guess the category", and it's cruel to expect users (including
ourselves) to know everything in the tree at all times.  It's in all
our best interest to make it easy to know what choices are available
so we can get back to more important things.  Tags help further this
ideal.

[gentoo-dev] Re: RFC: split up media-sound/ category

[Duncan]

Jesús J. Guerrero Botella posted on Mon, 27 Jun 2011 16:19:57 +0200 as
excerpted:

> someone said fat32 compatibility is a feature we want (still can't guess
> why, but well, be consequent...).

I believe that "someone" that mentioned fat32 compatibility in the 
context of symlinks was me.  

But "we want" is rather strong for what I intended.  More, "it's a factor 
to keep in mind", and "if we decide we do NOT want to keep fat32 
compatibility, we should be sure the feature we're implementing that 
breaks it is worth the tradeoff."

Maybe it is worth the tradeoff.  Maybe it isn't.  But either way, we 
shouldn't break that compatibility accidentally, simply because we didn't 
think of it as a factor.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: RFC: split up media-sound/ category

[Jesús J. Guerrero Botella]

2011/6/27 Wyatt Epp <wyatt.epp@gmail.com>:
> 2011/6/27 Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com>:
>> That still doesn't answer my question anyway: both features (symlinks
>> and +65k files on a single dir) are incompatible with fat32. And
>> someone said fat32 compatibility is a feature we want (still can't
>> guess why, but well, be consequent...). Obviously, we want fat32
>> compatibility when it comes to arguing against symlinks, which have
>> always been with us by the way, but that's not important when we talk
>> about other things that are not compatible with fat32.
>
> I'm not sure where you're getting 65k files.  Unless I misinterpreted
> everything everyone else was saying, every package would still have
> its own directory.  There are fewer than 20k even with a bunch of
> overlays installed.  Regardless, you might check the other (other)
> thread; I think we're probably going to go quick and
> not-necessarily-dirty with sets to get 99% of what we're looking for
> almost trivially.

Well, someone suggested a flat directory, which I understand as having
all the ebuilds in a single directory, that's also why they are
talking about the need to make ebuild names unique, to avoid file
names collision.

> 2011/6/27 Jesús J. Guerrero Botella <jesus.guerrero.botella@gmail.com>:
>> C) "ls $PORTDIR/whatever-category" is a command that's way simpler
>> than the one you posted.
>>
> It's also fundamentally broken because one package can only be in one
> category and their expansion has not historically been speedy.  Tags
> are a non-exclusive one-to-many relationship.  So a package can have
> as many tags as it needs, and users will be able to leverage tags
> alone or in combinations to find things they want or need.

I wouldn't call that broken. Again, symlinks can perfectly solve that
with little extra work. We use them for that same purpose in lots of
things. For example, eselect sets symlinks to select the active mesa
implementation from between all the installed ones. And we have been
using symlinks to make libs and paths available with different names
in linux fs's since ever.

I am not saying that my idea is better than the rest. I am just
wondering why the heck symlinks are not an option when linux has
supported them natively since almost the day zero.

>> I don't even use tags for my music collections
>
> That's very curious, and I wouldn't mind talking about why that is
> off-list (not quite joking; that's really interesting).

Feel free to mail me if you want extra clarification. But to sum up, I
just keep everything in a estructured directory tree. I could easily
use easytag to automatically tag everything with little or not extra
effort, automatically, but I rarely resort to tags. I don't use
behemoths like amarok or the likes. I use mplayer or moc, usually. And
locate is the only tool I need to find quickly a song in less than one
second.

> So to sum it up, we're fixing package navigation and discovery because
> Gentoo is about choice.  Even 15,000 packages is too many to have to
> play "guess the category", and it's cruel to expect users (including
> ourselves) to know everything in the tree at all times.  It's in all
> our best interest to make it easy to know what choices are available
> so we can get back to more important things.  Tags help further this
> ideal.

That's fine by me, as long as some sense is retained in the underlying
fs estructure and tags are an added value, not a substitute for what's
already in there. What I wouldn't like is to get 140k ebuilds dumped
into a single directory.


-- 
Jesús Guerrero Botella

[gentoo-dev] Re: RFC: split up media-sound/ category

[Paul Arthur]

On 2011-06-27, Jesús J Guerrero Botella
<jesus.guerrero.botella@gmail.com> wrote:

> 2011/6/27 Wyatt Epp <wyatt.epp@gmail.com>:
>
>> 2011/6/27 Jesús J. Guerrero Botella
>> <jesus.guerrero.botella@gmail.com>:
>>
>>> That still doesn't answer my question anyway: both features
>>> (symlinks and +65k files on a single dir) are incompatible with
>>> fat32. And someone said fat32 compatibility is a feature we want
>>> (still can't guess why, but well, be consequent...). Obviously,
>>> we want fat32 compatibility when it comes to arguing against
>>> symlinks, which have always been with us by the way, but that's
>>> not important when we talk about other things that are not
>>> compatible with fat32.
>>
>> I'm not sure where you're getting 65k files.  Unless I
>> misinterpreted everything everyone else was saying, every package
>> would still have its own directory.  There are fewer than 20k
>> even with a bunch of overlays installed.  Regardless, you might
>> check the other (other) thread; I think we're probably going to go
>> quick and not-necessarily-dirty with sets to get 99% of what we're
>> looking for almost trivially.
>
> Well, someone suggested a flat directory, which I understand as
> having all the ebuilds in a single directory, that's also why they
> are talking about the need to make ebuild names unique, to avoid
> file names collision.

Packages, not ebuilds. Getting rid of categories puts all of the
packages in the same directory, so you need unique package names.
So you might make app-arch/par parchive, dev-util/par palm-par, and
app-text/par par. par-1.52.ebuild would still live in $PORTDIR/par/

-- 
Please do not ask the Staff to use red text to ask Users not to hypothetically
appeal hypothetical future bans.
    --Cessna on RPGNet

Re: [gentoo-dev] RFC: split up media-sound/ category

[Zac Medico]

On 06/27/2011 07:23 AM, Jesús J. Guerrero Botella wrote:
> I still don't understand why
> 
> A) you need to build a project, a glep, whatever the course of action
> is, I am bad at bureaucracy.
> B) you need to code the solution, to fix What?

Some people requested a "tags" feature. I'm not sure if "fix" is the
best word. Tags will provide a new way to search for packages, that's all.

> C) "ls $PORTDIR/whatever-category" is a command that's way simpler
> than the one you posted.

There are lots of possible approaches. See Ciaran's "Are tags just
sets?" approach that is very similar to your suggested symlink approach,
except that it represents a tag using a text file containing a list of
packages instead of a directory containing symlinks.

> XML seems to be the trend, but we should really think a moment, what's
> what we are trying to fix?

Again, maybe "fix" isn't the best word. I believe that the goal it to
provide a new method of searching that is based on tags.

> We just needed to add some categories or rename them when someone
> started this thread, but now, even when we know we are lacking dev
> power in some areas we start arguing that the base concept of our OS
> (portage) is wrong, and that we should redo it completely by putting
> every ebuild into a directory and tagging them.

I would advise against going down the "redo it completely" path, since
it's relatively easy to implement a tagging mechanism that will coexist
with the existing category framework.

> Again, that's not "port-age". Read on ports:
> 
> http://en.wikipedia.org/wiki/FreeBSD_Ports

Interestingly, the wiki page that you linked has a link to a project to
add tags to the ports collection:

  http://www.tobez.org/port-tags/

> I don't even use tags for my music collections and now I am going to
> be forced to use them to operate my OS.

Again, I would advise against going down the "redo it completely" path
that this statement implies, given that it's relatively easy to
implement a tagging mechanism that will coexist with the existing
category framework.
-- 
Thanks,
Zac