[gentoo-dev] Are tags just sets?

[gentoo-dev] Are tags just sets?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 714 bytes --]

Here's a completely different way of doing tags:

First, standardise sets. We probably want to go with a format along the
lines of:

    eapi = 4
    description = Monkeys

    dev-monkey/howler
    dev-monkey/spider
    >=dev-monkey/spanky-2.0
    dev-monkey/squirrel

where eapi has to be on the first line.

Second, make a bunch of sets named kde-tag, editors-tag, xml-tag,
monkeys-tag etc.

Third, make tools that allow browsing, searching etc able to list
things by tag (or sets in general).

Advantages: dead easy to implement, backwards compatible, we need sets
anyway.

Disadvantages: doesn't use some horribly convoluted system of XML,
wikis and web 2.0.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 953 bytes --]

On Sun, 26 Jun 2011 08:02:57 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> First, standardise sets. We probably want to go with a format along
> the lines of:
> 
>     eapi = 4
>     description = Monkeys

A 'type' field would be useful as well, to support various kinds of
package sets (much like portage handles currently).

>     dev-monkey/howler
>     dev-monkey/spider
>     >=dev-monkey/spanky-2.0
>     dev-monkey/squirrel

We'd either want to add || ( ) here, or somehow explicitly specify that
this is a one-of set.

> Second, make a bunch of sets named kde-tag, editors-tag, xml-tag,
> monkeys-tag etc.

I'd go with prefix, not suffix. Should be easier group them together
then.

> Disadvantages: doesn't use some horribly convoluted system of XML,
> wikis and web 2.0.

And introduces another dedicated file format the PM has to implement
from scratch.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1100 bytes --]

On Sun, 26 Jun 2011 10:41:24 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> A 'type' field would be useful as well, to support various kinds of
> package sets (much like portage handles currently).

I'm highly doubtful that there's any real need for different kinds of
repository-provided sets. We especially don't want sets to be code...

> >     dev-monkey/howler
> >     dev-monkey/spider
> >     >=dev-monkey/spanky-2.0
> >     dev-monkey/squirrel
> 
> We'd either want to add || ( ) here, or somehow explicitly specify
> that this is a one-of set.

No, that's something that's determined by how the set's used, not by
what's in the set. There's no such thing as a "one-of" set; a set is
just a list of package dep specs.

> > Disadvantages: doesn't use some horribly convoluted system of XML,
> > wikis and web 2.0.
> 
> And introduces another dedicated file format the PM has to implement
> from scratch.

It's a simple text file. It takes fewer lines of code to parse it
from scratch than it does to get the results out of an XML parser.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1107 bytes --]

On Sun, 26 Jun 2011 09:43:41 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Sun, 26 Jun 2011 10:41:24 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > A 'type' field would be useful as well, to support various kinds of
> > package sets (much like portage handles currently).
> 
> I'm highly doubtful that there's any real need for different kinds of
> repository-provided sets. We especially don't want sets to be code...

Simple things like getting a list of packages which own a particular
file (for rebuilds) or grepping a variable are useful to users.
For example, the x11 overlay provides a set to rebuild the xorg server
modules after an update.

> > We'd either want to add || ( ) here, or somehow explicitly specify
> > that this is a one-of set.
> 
> No, that's something that's determined by how the set's used, not by
> what's in the set. There's no such thing as a "one-of" set; a set is
> just a list of package dep specs.

Hm, true. I guess noone will want to merge 'a random package matching
a tag' :P.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

On Sun, 26 Jun 2011 10:54:44 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> > I'm highly doubtful that there's any real need for different kinds
> > of repository-provided sets. We especially don't want sets to be
> > code...
> 
> Simple things like getting a list of packages which own a particular
> file (for rebuilds) or grepping a variable are useful to users.

That's something done by sets as provided by the package mangler, not
something done by repository-specified sets.

> For example, the x11 overlay provides a set to rebuild the xorg server
> modules after an update.

That's just a list of package specs. The user then says "only the ones
of these that I have installed" by the way the set is used.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1165 bytes --]

On Sun, 26 Jun 2011 10:00:43 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Sun, 26 Jun 2011 10:54:44 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > > I'm highly doubtful that there's any real need for different kinds
> > > of repository-provided sets. We especially don't want sets to be
> > > code...
> > 
> > Simple things like getting a list of packages which own a particular
> > file (for rebuilds) or grepping a variable are useful to users.
> 
> That's something done by sets as provided by the package mangler, not
> something done by repository-specified sets.

So we should provide separate copies of the same sets for each package
mangler?

> > For example, the x11 overlay provides a set to rebuild the xorg
> > server modules after an update.
> 
> That's just a list of package specs. The user then says "only the ones
> of these that I have installed" by the way the set is used.

Well, I think a simple specification saying 'all installed packages
which install to /usr/lib/foo' is much simpler to write and maintain
than a random number of package names.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

On Sun, 26 Jun 2011 14:48:57 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> > That's something done by sets as provided by the package mangler,
> > not something done by repository-specified sets.
> 
> So we should provide separate copies of the same sets for each package
> mangler?

You should avoid providing sets that are so complicated that they rely
upon executing fancy code that maps to package manager APIs that are in
no way stable, documented or guaranteed to work two weeks from now.

> Well, I think a simple specification saying 'all installed packages
> which install to /usr/lib/foo' is much simpler to write and maintain
> than a random number of package names.

Sure, and a *user* can have a set like that, specified on the command
line. That's not something a repository should be doing though. Sets as
provided by the repository are a subset of sets available to the user.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Wyatt Epp]

On Sun, Jun 26, 2011 at 03:02, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> Here's a completely different way of doing tags:
>
You know, that's not a bad way of going about it.  Truth be told, I
had sort of forgotten sets exists because they're a bit cumbersome at
the moment.  But it's cheap and dead simple and gets us our 90%
immediately.  Actually, it gets 100%, even, if you can include a set
as part of another set (implication) and symlinks function as aliases.
 Very clever; I like it.

> where eapi has to be on the first line.
>
Looks fine but just to be clear, why is having the eapi necessary?

> Second, make a bunch of sets named kde-tag, editors-tag, xml-tag,
> monkeys-tag etc.
>
Don't even need the "-tag" part, really.  But yes, a couple hundred
sets are in order.  And some tool-glue.

> Disadvantages: doesn't use some horribly convoluted system of XML,
> wikis and web 2.0.
>
That's not a disadvantage at all.  Thank you for noticing the third path.

While I still don't really believe categories to be necessary, this
will be a fine intermediate step.

Cheers,
Wyatt

Re: [gentoo-dev] Are tags just sets?

[Kent Fredric]

On 26 June 2011 19:02, Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> Here's a completely different way of doing tags:
>
> First, standardise sets. We probably want to go with a format along the
> lines of:
>
>    eapi = 4
>    description = Monkeys
>
>    dev-monkey/howler
>    dev-monkey/spider
>    >=dev-monkey/spanky-2.0
>    dev-monkey/squirrel
>
> where eapi has to be on the first line.
>

I initially didn't like the idea somewhat, but then I figured the
amount of impact and retooling required to make this work is virtually
zero, its not complicated, and its text based.

So why don't we just implement it, even if it sucks balls, the amount
of downsides it has are zero really, it doesn't affect how portage
currently works at all, so if we prove it to suck or decide it needs
replacing, we can throw it out and put something else in with very
little pain.

So +1.

( Yes, I understand the concerns of Yet Another format, I myself would
suggest JSON for a plethora of reasons were it up to me, and all
though it is /mostly/ just a list of package specs, those "first
lines" with the = in them make this more "format" than just a text
file, but I think we should see whether or not the concept works FIRST
before debating whether or not we've bikeshedded the right format to
put it in ).


-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

[gentoo-dev] Re: Are tags just sets?

[Duncan]

Ciaran McCreesh posted on Sun, 26 Jun 2011 08:02:57 +0100 as excerpted:

> [M]ake a bunch of sets named kde-tag, editors-tag, xml-tag,
> monkeys-tag etc.

++

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Are tags just sets?

[Maciej Mrozowski]

[-- Attachment #1: Type: Text/Plain, Size: 2102 bytes --]

Hello,

On Sunday 26 of June 2011 09:02:57 Ciaran McCreesh wrote:
> Here's a completely different way of doing tags:

As far as sets are concerned, how about PROPERTIES=set?
https://bugs.gentoo.org/show_bug.cgi?id=272488

It's been proposed years ago. Is there a need to reinvent sets format every 
time it's bought up?

> First, standardise sets. We probably want to go with a format along the
> lines of:
> 
>     eapi = 4
>     description = Monkeys
> 
>     dev-monkey/howler
>     dev-monkey/spider
> 
>     >=dev-monkey/spanky-2.0
> 
>     dev-monkey/squirrel
> 
> where eapi has to be on the first line.
> 
> Second, make a bunch of sets named kde-tag, editors-tag, xml-tag,
> monkeys-tag etc.

I see major disadvantage with this approach. It's painful to maintain.
Imagine hundreds of different tags, with each package having at least two 
tags. You certainly don't expect anyone to be able to maintain that.
Also those files cannot be generated since there needs to be some original 
source of tags information to generate such 'sets' from.

I don't need to remind one needs to keep those files synchronized with tree 
changes (package renames, removals) while tags in metadata.xml automatically 
ravel with package.

Tag is a property or attribute of package and should be implemented as 
additional package information, metadata.xml is the most natural choice.

> Third, make tools that allow browsing, searching etc able to list
> things by tag (or sets in general).
> 
> Advantages: dead easy to implement, backwards compatible, we need sets
> anyway.

PROPERTIES=set have the same advantages - they can also be pulled within 
dependency tree by other packages.

> Disadvantages: doesn't use some horribly convoluted system of XML,
> wikis and web 2.0.

Using already proven technologies and tools is barely disadvantage. I think 
last thing we need is yet another obscure format nothing widely usable 
understands.

Sets concept is completely orthogonal to tags concept, please do not mix 
unrelated things.

-- 
regards
MM

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Kent Fredric]

On 27 June 2011 03:12, Maciej Mrozowski <reavertm@gmail.com> wrote:
>
> I see major disadvantage with this approach. It's painful to maintain.
> Imagine hundreds of different tags, with each package having at least two
> tags. You certainly don't expect anyone to be able to maintain that.
> Also those files cannot be generated since there needs to be some original
> source of tags information to generate such 'sets' from.

This problem is in my mind somewhat twofold.

There are 2 ways to approach applying a tag.

1. Opening each package and setting its tags ( package -> tags )
2. Opening a "tag" file and setting its packages ( tag -> package )

And both approaches are very handy.

1. Is useful because it makes it easy to apply a singular tag surgically.
2. Is useful because it makes it easy to apply one tag to a raft of packages.

However, if one wishes to implement something that works without
touching anything remotely part of portage itself, and one wants a
fast and easy proof of concept , number 2 is the preferable approach.
( Because you can make a basic example set of tags easily simply with
a bit of "find > file" for each tag you wish to create  )

Ultimately I think we'll see both forms emerging, long term we'll
probably edit metadata.xml, and the data will be aggregated to form a
singular fast-to-read tag index for each tag.

But we can develop the other half of the system, the "Work with an
existing tag index of sorts" /now/ and get a working proof of concept
without needing to derail ourselves bikeshedding how the portage side
of things will work.

( And I think we can expect to see tools emerge for maintaining
package tags, whether it be surgically or low-orbit-ion-cannon grade
precision )

-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Are tags just sets?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1751 bytes --]

On Sun, 26 Jun 2011 17:12:27 +0200
Maciej Mrozowski <reavertm@gmail.com> wrote:
> On Sunday 26 of June 2011 09:02:57 Ciaran McCreesh wrote:
> > Here's a completely different way of doing tags:
> 
> As far as sets are concerned, how about PROPERTIES=set?
> https://bugs.gentoo.org/show_bug.cgi?id=272488
> 
> It's been proposed years ago. Is there a need to reinvent sets format
> every time it's bought up?

The problems with PROPERTIES=set remain exactly the same as they were
when it was first proposed.

> I see major disadvantage with this approach. It's painful to maintain.
> Imagine hundreds of different tags, with each package having at least
> two tags. You certainly don't expect anyone to be able to maintain
> that.

Uh, I don't see how that's in any way difficult to maintain.

> Tag is a property or attribute of package

That one's highly debatable.

> PROPERTIES=set have the same advantages - they can also be pulled
> within dependency tree by other packages.

Yes, but PROPERTIES=set has all of the problems it had when it was
first proposed, and is the wrong way to implement sets.

> > Disadvantages: doesn't use some horribly convoluted system of XML,
> > wikis and web 2.0.
> 
> Using already proven technologies and tools is barely disadvantage. I
> think last thing we need is yet another obscure format nothing widely
> usable understands.

Good, so you'll be happy going with what Unix has been using for
decades then.

> Sets concept is completely orthogonal to tags concept, please do not
> mix unrelated things.

Depends upon what you think the "tags concept" is. We've already
established that everyone has a different idea of what tags are anyway.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Maciej Mrozowski]

[-- Attachment #1: Type: Text/Plain, Size: 2764 bytes --]

On Monday 27 of June 2011 07:49:24 Ciaran McCreesh wrote:
> On Sun, 26 Jun 2011 17:12:27 +0200

> Maciej Mrozowski <reavertm@gmail.com> wrote:
> > On Sunday 26 of June 2011 09:02:57 Ciaran McCreesh wrote:
> > > Here's a completely different way of doing tags:
> > As far as sets are concerned, how about PROPERTIES=set?
> > https://bugs.gentoo.org/show_bug.cgi?id=272488

> > It's been proposed years ago. Is there a need to reinvent sets format
> > every time it's bought up?

> The problems with PROPERTIES=set remain exactly the same as they were
> when it was first proposed.

Which is?
No, "been there, done that, won't work" is not sufficient. Please elaborate.

> > I see major disadvantage with this approach. It's painful to maintain.
> > Imagine hundreds of different tags, with each package having at least
> > two tags. You certainly don't expect anyone to be able to maintain
> > that.

> Uh, I don't see how that's in any way difficult to maintain.

No, it's not difficult, it's pain in ass to keep a hundred files with a few 
thousands lines each up2date with tree changes (pkgmove, cvs remove). Yes, it 
could be automated by some crafty cvs hook on server. However cvs hooks should 
be the last resort and not a tool to deal with consequences of broken design.

> > Tag is a property or attribute of package
> That one's highly debatable.

It's not debatable for those familiar with object oriented design concept.
It may be debatable for database designers what's the best way to implement 
many to many association because they have means to auto-update referenced 
keys - we don't have those means so proposal to "move tags to separate table" 
isn't practical.

> > PROPERTIES=set have the same advantages - they can also be pulled
> > within dependency tree by other packages.

> Yes, but PROPERTIES=set has all of the problems it had when it was
> first proposed, and is the wrong way to implement sets.

> > > Disadvantages: doesn't use some horribly convoluted system of XML,
> > > wikis and web 2.0.

> > Using already proven technologies and tools is barely disadvantage. I
> > think last thing we need is yet another obscure format nothing widely
> > usable understands.

> Good, so you'll be happy going with what Unix has been using for
> decades then.

Indeed, with sets in bash (ebuild) format.

> > Sets concept is completely orthogonal to tags concept, please do not
> > mix unrelated things.

> Depends upon what you think the "tags concept" is. We've already
> established that everyone has a different idea of what tags are anyway.

I don't know what's everyone's idea behind the tag, I refer to:

http://en.wikipedia.org/wiki/Tag_(metadata)

-- 
regards
MM

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2263 bytes --]

On Mon, 27 Jun 2011 20:21:29 +0200
Maciej Mrozowski <reavertm@gmail.com> wrote:
> > The problems with PROPERTIES=set remain exactly the same as they
> > were when it was first proposed.
> 
> Which is?
> No, "been there, done that, won't work" is not sufficient. Please
> elaborate.

You can find and read the original discussion as well as I can. No
point in going over exactly the same material all over again since the
design hasn't been updated since.

> > Uh, I don't see how that's in any way difficult to maintain.
> 
> No, it's not difficult, it's pain in ass to keep a hundred files with
> a few thousands lines each up2date with tree changes (pkgmove, cvs
> remove). Yes, it could be automated by some crafty cvs hook on
> server. However cvs hooks should be the last resort and not a tool to
> deal with consequences of broken design.

If that were true, you'd be doing the same thing for package.mask...
*shrug* If you really think it's that bad, though, go with Brian's
proposal, and whilst you're at it, make package.mask and all those
other profile files that contain long lists of package names be created
the same way.

> > > Tag is a property or attribute of package
> > That one's highly debatable.
> 
> It's not debatable for those familiar with object oriented design
> concept.

Clearly it is. Have a look at posts in this thread. Some people insist
that tags are properties of ebuilds, some that they're properties of
packages, some that they're repository-level properties, and some that
they're external, user-level properties. 

For that matter, ebuilds aren't object oriented.

> > Good, so you'll be happy going with what Unix has been using for
> > decades then.
> 
> Indeed, with sets in bash (ebuild) format.

No point, just like there's no point in package.mask being an ebuild.

> > Depends upon what you think the "tags concept" is. We've already
> > established that everyone has a different idea of what tags are
> > anyway.
> 
> I don't know what's everyone's idea behind the tag, I refer to:
> 
> http://en.wikipedia.org/wiki/Tag_(metadata)

Sure, and that's sufficiently vague that all sorts of completely
different ideas could be called tags.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Rich Freeman]

On Mon, Jun 27, 2011 at 1:49 AM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Sun, 26 Jun 2011 17:12:27 +0200
> Maciej Mrozowski <reavertm@gmail.com> wrote:
>
>> Sets concept is completely orthogonal to tags concept, please do not
>> mix unrelated things.
>
> Depends upon what you think the "tags concept" is. We've already
> established that everyone has a different idea of what tags are anyway.

I too feel that tags should be distinct from sets, for a bunch of reasons.

Sets should really be something carefully controlled by the
repository.  While I'm fine with having tags in the repository also,
there is talk about giving users ways of supplying them as well.

Sets are generally used to tell the package manager to do something
with a lot of packages at once.  I'm not sure there is much of a need
to do this with tags, at least not in most of the use cases that have
been suggested.

Here is how I see tags being used:

1.  I want a WYSIWYG html editor.
2.  I search for tags like "editor" and "html" and "WYSIWYG" and maybe
even "text."
3.  I check out descriptions and homepages or whatever for a few
likely candidates, and install one or maybe two.

What I doubt I'd ever do is just install any package that has anything
to do with text/html editing.  When you search google you care about
the top 5-10 - not the whole set of results.

Maybe if we define multiple namespaces for tags we could move to using
tags as dependencies or whatever, and those tags would be distinct and
much more carefully defined and controlled.  However, I think this is
more far-out and not the immediate goal.

Sets might work, but they seem a bit like a hack...

Rich

Re: [gentoo-dev] Are tags just sets?

[Wyatt Epp]

On Mon, Jun 27, 2011 at 16:23, Rich Freeman <rich0@gentoo.org> wrote:
> I too feel that tags should be distinct from sets, for a bunch of reasons.
>
> Sets should really be something carefully controlled by the
> repository.  While I'm fine with having tags in the repository also,
> there is talk about giving users ways of supplying them as well.
>
Too late; /etc/portage/sets/

> Sets are generally used to tell the package manager to do something
> with a lot of packages at once.  I'm not sure there is much of a need
> to do this with tags, at least not in most of the use cases that have
> been suggested.
>
At the moment, yes, that's very true.  But that's a matter of lacking
tools, more than a necessarily orthogonal concept.  If you look at
sets (or categories), you find they describe attributes of packages.
For example, @world is "everything the user has merged".  The kde
overlay provides things like @kde-live, "kde packages built from
subversion" (it's more specific than ${PN} in this case, but generally
won't need to be).  I don't think anyone here believes this feature
exists without some tool support to glue it together.

> Maybe if we define multiple namespaces for tags we could move to using
> tags as dependencies or whatever, and those tags would be distinct and
> much more carefully defined and controlled.  However, I think this is
> more far-out and not the immediate goal.
>
I'd say that's rather unnecessary.  We should be wary of conflating
all metadata together in our heads: Tags are not a replacement for
structured key-value that we already have.  When we talk of tags,
we're talking about general purpose semantic descriptors that are only
loosely structured and benefit from emergent community standards.  We
already have the things that benefit from rigid definition.

> Sets might work, but they seem a bit like a hack...
>
Oh, absolutely.  But nearly anything is better than the current state
of affairs; if it falls apart, we find a different way.

Re: [gentoo-dev] Are tags just sets?

[Rich Freeman]

On Mon, Jun 27, 2011 at 5:06 PM, Wyatt Epp <wyatt.epp@gmail.com> wrote:
> On Mon, Jun 27, 2011 at 16:23, Rich Freeman <rich0@gentoo.org> wrote:
>> I too feel that tags should be distinct from sets, for a bunch of reasons.
>>
>> Sets should really be something carefully controlled by the
>> repository.  While I'm fine with having tags in the repository also,
>> there is talk about giving users ways of supplying them as well.
>>
> Too late; /etc/portage/sets/

That wasn't what I was thinking of.  Package masking is also something
we carefully control in the repository but users can override it FOR
THEIR OWN SYSTEMS.  With tags I think that there were concepts
floating around of letting anybody influence how packages are tagged.

With some of the offline suggestions there is really nothing stopping
anybody from making their own tagging solution outside of the
repository.  I could set up my own packages.g.o site and implement
user-supplied tagging if I wanted to.  That might not be a bad way to
get this started, as somebody else in one of these threads seemed to
be implying.

Rich

Re: [gentoo-dev] Are tags just sets?

[Wyatt Epp]

On Mon, Jun 27, 2011 at 17:23, Rich Freeman <rich0@gentoo.org> wrote:
> That wasn't what I was thinking of.  Package masking is also something
> we carefully control in the repository but users can override it FOR
> THEIR OWN SYSTEMS.  With tags I think that there were concepts
> floating around of letting anybody influence how packages are tagged.
>
Ah yes, I think it was Jorge that mentioned that.  And it's a good
idea, too: it streamlines the patch process for tag updates and it
gives us an idea of what sorts of things users find important for
discovering packages.  But because the proposed solution is so simple,
we can handle that later pretty easily as long as we agree on what
we're doing up front to actually enable this.

Cheers,
Wyatt

Re: [gentoo-dev] Are tags just sets?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1357 bytes --]

On Mon, 27 Jun 2011 16:23:54 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> Sets should really be something carefully controlled by the
> repository.  While I'm fine with having tags in the repository also,
> there is talk about giving users ways of supplying them as well.

Why? You can have carefully controlled sets for fancy things. But tag
sets don't need to be carefully controlled.

The way you give users control over tags using sets is to make sets in
overlays be merged with sets with the same name in the main tree, and
then allow users who feel like it to publish an overlay containing
their tags.

> Here is how I see tags being used:
> 
> 1.  I want a WYSIWYG html editor.
> 2.  I search for tags like "editor" and "html" and "WYSIWYG" and maybe
> even "text."
> 3.  I check out descriptions and homepages or whatever for a few
> likely candidates, and install one or maybe two.

$package_mangler search-tags editor html wysywig

> What I doubt I'd ever do is just install any package that has anything
> to do with text/html editing.

Not really a problem. Sets are usable for lots of things, not just
installing. They're useful in configuration files, for example -- you'd
probably never want to install every X driver either, but you might
want to set some options for every X driver.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Are tags just sets?

[Brian Harring]

On Sun, Jun 26, 2011 at 08:02:57AM +0100, Ciaran McCreesh wrote:
> Here's a completely different way of doing tags:
> 
> First, standardise sets. We probably want to go with a format along the
> lines of:
> 
>     eapi = 4
>     description = Monkeys
> 
>     dev-monkey/howler
>     dev-monkey/spider
>     >=dev-monkey/spanky-2.0
>     dev-monkey/squirrel
> 
> where eapi has to be on the first line.
> 
> Second, make a bunch of sets named kde-tag, editors-tag, xml-tag,
> monkeys-tag etc.

This is a bit inverted- tagging is fundamentally pkg specific.  If we 
did as you proposed and I wanted to find out all tags/descriptions for 
a pkg, the PM would have to scan every tags file there.

Additionally, as others have indicated, it sucks have this data tucked 
away in another chunk of the tree, away from where the package data 
actually is (in cat/pkg/*).

> Advantages: dead easy to implement, backwards compatible, we need sets
> anyway.
> 
> Disadvantages: doesn't use some horribly convoluted system of XML,
> wikis and web 2.0.

Counter proposal; use what you're proposing as a cache.  metadata.xml 
is the proper place for this- we store use.local data there for 
example, and cache the results of it in profiles/use.local.desc.  Via 
this vcs concerns are addressed, data locality is preserved, and 
multiple modes of lookup are sanely supported.

Roughly, and this is definitely a rough sketch:

<tags>
 <atom val=dev-util/diffball>tag1 tag2 tag2 tag3</atom>
 <atom val=">=dev-util/diffball-1.0">awesomeness</atom>
</tags>

From there, jammed into profiles, a master description list in 'tag: 
description' style format.  Identifying the eapi is easy enough- just 
inspect the atom's that wind up in the tags list.  Easy enough.

Either way... thing everyone admits the current tree format has it's 
flaws; strikes me it's better to fit to the standards/conventions of 
the repo format as it is now.

<proceed w/ xml bashing>
~harring

Re: [gentoo-dev] Are tags just sets?

[Kent Fredric]

On 28 June 2011 15:26, Brian Harring <ferringb@gmail.com> wrote:
> This is a bit inverted- tagging is fundamentally pkg specific.  If we
> did as you proposed and I wanted to find out all tags/descriptions for
> a pkg, the PM would have to scan every tags file there.
>
> Additionally, as others have indicated, it sucks have this data tucked
> away in another chunk of the tree, away from where the package data
> actually is (in cat/pkg/*).
>
>> Advantages: dead easy to implement, backwards compatible, we need sets
>> anyway.
>>
>> Disadvantages: doesn't use some horribly convoluted system of XML,
>> wikis and web 2.0.
>
> Counter proposal; use what you're proposing as a cache.  metadata.xml
> is the proper place for this- we store use.local data there for
> example, and cache the results of it in profiles/use.local.desc.  Via
> this vcs concerns are addressed, data locality is preserved, and
> multiple modes of lookup are sanely supported.
>
> Roughly, and this is definitely a rough sketch:
>
> <tags>
>  <atom val=dev-util/diffball>tag1 tag2 tag2 tag3</atom>
>  <atom val=">=dev-util/diffball-1.0">awesomeness</atom>
> </tags>
>
> From there, jammed into profiles, a master description list in 'tag:
> description' style format.  Identifying the eapi is easy enough- just
> inspect the atom's that wind up in the tags list.  Easy enough.
>
> Either way... thing everyone admits the current tree format has it's
> flaws; strikes me it's better to fit to the standards/conventions of
> the repo format as it is now.
>
> <proceed w/ xml bashing>
> ~harring
>
>

I agree whole heartedly, I do, really.  Ultimately the best design
will involve:

A. Storing tag data in metadata.xml   ( package -> tag association )
B. Developing a tool that aggregates the contents of metadata.xml to
produce a cache for the data going the other way ( tag -> package )

People searching for packages that match a tag will thusly be able to
read this cached data ( Our primary use case )  and people who want to
know what tags a specific package have will read/augment metadata.xml

I Believe both these parts are required for the design to be successful.

However, both parts have a bit of bike-shedding involved in them,
part A of the system requires changes to exisiting structures, and
part A requires bike shedding about part B's format.

For the sake of simplicity, I'm ( and I believe others ) are simply
suggesting we develop part B first.

Yes, the initial complications in creating the tag indexes will be
somewhat large, but it does mean we can create an early proof of
concept that implements a somewhat "usable" tag-search for users
without having to really touch portage itself.

Once we get that part sorted out and agree upon the general format and
requirements of this index/cache, we can then decide how we'll fit it
in to metadata.xml




-- 
Kent

perl -e  "print substr( \"edrgmaM  SPA NOcomil.ic\\@tfrken\", \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Re: [gentoo-dev] Are tags just sets?

[Brian Harring]

On Tue, Jun 28, 2011 at 03:43:21PM +1200, Kent Fredric wrote:
> A. Storing tag data in metadata.xml   ( package -> tag association )
> B. Developing a tool that aggregates the contents of metadata.xml to
> produce a cache for the data going the other way ( tag -> package )
> 
> People searching for packages that match a tag will thusly be able to
> read this cached data ( Our primary use case )  and people who want to
> know what tags a specific package have will read/augment metadata.xml
> 
> I Believe both these parts are required for the design to be successful.
> 
> However, both parts have a bit of bike-shedding involved in them,
> part A of the system requires changes to exisiting structures, and
> part A requires bike shedding about part B's format.
> 
> For the sake of simplicity, I'm ( and I believe others ) are simply
> suggesting we develop part B first.

It's sub hour or so to look at the existing `egencache 
--update-use-local-desc` (specifically GenUseLocalDesc class) and 
implement such a cache.  It's not hard.

Either way, if you want this, the part that will take time is 
integrating it into emerge itself for searching; now if you want to 
write that code twice, go nuts, but someone lazier would do both at 
the same time to make sure they structured the API so it could 
support in one pass, rather than deploying an API, than having to 
shoehorn the cpv argument in.

If people want it, cut a patch and post it for feedback either way- 
bearing in mind that from where I'm sitting, deploying it half-assed 
requiring people to maintain a bunch of text files isn't a viable 
option ;)

~brian

Re: [gentoo-dev] Are tags just sets?

[Peter Volkov]

В Пнд, 27/06/2011 в 20:26 -0700, Brian Harring пишет:
> > Second, make a bunch of sets named kde-tag, editors-tag, xml-tag,
> > monkeys-tag etc.

I'd like avoid editing multiple files. Much better will be keep tags
with package.

> Counter proposal; use what you're proposing as a cache.  metadata.xml 
> is the proper place for this

++

> Roughly, and this is definitely a rough sketch:
> 
> <tags>
>  <atom val=dev-util/diffball>tag1 tag2 tag2 tag3</atom>
>  <atom val=">=dev-util/diffball-1.0">awesomeness</atom>
> </tags>

Since use dependencies eapi should be provided somewhere.

Also I like metadata.xml proposal since it'll be easy to use
per-category metadata.xml for all ebuilds to inherit.

--
Peter.

Re: [gentoo-dev] Are tags just sets?

[Wyatt Epp]

On Tue, Jun 28, 2011 at 07:53, Peter Volkov <pva@gentoo.org> wrote:
> В Пнд, 27/06/2011 в 20:26 -0700, Brian Harring пишет:
>> > Second, make a bunch of sets named kde-tag, editors-tag, xml-tag,
>> > monkeys-tag etc.
>
> I'd like avoid editing multiple files. Much better will be keep tags
> with package.
>
<snip />
> Also I like metadata.xml proposal since it'll be easy to use
> per-category metadata.xml for all ebuilds to inherit.
>
You weren't planning on doing this all _manually_, were you? ;)  With
proper tool support (or even just a couple quick scripts), the
single/multiple file argument becomes pretty much moot.

Another thing I think should be reiterated is that leveraging sets in
this matter gets us implication and aliasing for free (as long as sets
can be included inside of other sets).  I'm not sure it would be so
easy to accomplish otherwise.

Regards,
Wyatt

Re: [gentoo-dev] Are tags just sets?

[Maciej Mrozowski]

[-- Attachment #1: Type: Text/Plain, Size: 3012 bytes --]

On Tuesday 28 of June 2011 05:26:29 Brian Harring wrote:
> On Sun, Jun 26, 2011 at 08:02:57AM +0100, Ciaran McCreesh wrote:
> > Here's a completely different way of doing tags:
> > 
> > First, standardise sets. We probably want to go with a format along the
> > 
> > lines of:
> >     eapi = 4
> >     description = Monkeys
> >     
> >     dev-monkey/howler
> >     dev-monkey/spider
> >     
> >     >=dev-monkey/spanky-2.0
> >     
> >     dev-monkey/squirrel
> > 
> > where eapi has to be on the first line.
> > 
> > Second, make a bunch of sets named kde-tag, editors-tag, xml-tag,
> > monkeys-tag etc.
> 
> This is a bit inverted- tagging is fundamentally pkg specific.  If we
> did as you proposed and I wanted to find out all tags/descriptions for
> a pkg, the PM would have to scan every tags file there.
> 
> Additionally, as others have indicated, it sucks have this data tucked
> away in another chunk of the tree, away from where the package data
> actually is (in cat/pkg/*).
> 
> > Advantages: dead easy to implement, backwards compatible, we need sets
> > anyway.
> > 
> > Disadvantages: doesn't use some horribly convoluted system of XML,
> > wikis and web 2.0.
> 
> Counter proposal; use what you're proposing as a cache.  metadata.xml
> is the proper place for this- we store use.local data there for
> example, and cache the results of it in profiles/use.local.desc.  Via
> this vcs concerns are addressed, data locality is preserved, and
> multiple modes of lookup are sanely supported.
> 
> Roughly, and this is definitely a rough sketch:
> 
> <tags>
>  <atom val=dev-util/diffball>tag1 tag2 tag2 tag3</atom>
>  <atom val=">=dev-util/diffball-1.0">awesomeness</atom>
> </tags>
> 
> From there, jammed into profiles, a master description list in 'tag:
> description' style format.  Identifying the eapi is easy enough- just
> inspect the atom's that wind up in the tags list.  Easy enough.
> 
> Either way... thing everyone admits the current tree format has it's
> flaws; strikes me it's better to fit to the standards/conventions of
> the repo format as it is now.
> 
> <proceed w/ xml bashing>
> ~harring

Of course, nobody said metadata.xml would be the place that's to be 
immediately accessed by search tools. But as a data definition is concerned, 
metadata.xml is the right place for tags, not some obscure 'package.mask'-
style flat file.

Whether this information is to be processed for easier usage - by generating 
flat file with "cat/pkg tag1 tag2 ..." in each line for instance - is 
different story and nice to have probably.

@Ciaran
profiles/package.mask is quite convenient being flat file (although 
package.mask.d seems nicer, different story) as masking/unmasking groups of 
packages (KDE releases come to my mind) is a common use case.
Sure, it would benefit from auto-updating (pkgmove mostly as package.mask 
editing is otherwise closely tied to package removals anyway).

-- 
regards
MM

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]