[gentoo-portage-dev] qfile assumes category names contain a hyphen

[gentoo-portage-dev] qfile assumes category names contain a hyphen

[Amit Dor-Shifer]

Hi all.

While working on my overlay, I stumbled on an issue where qfile refused 
to acknowledge an installed file as being part of my package.

Looking into q's implementation (portage-utils-0.1.29), I see:

amit0 portage-utils-0.1.29 # grep -A 2 next_entry ./libq/vdb_get_next_dir.c
next_entry:
        ret = readdir(dir);
        if (ret == NULL) {
--
                goto next_entry;
        if (strchr(ret->d_name, '-') == NULL)
                if ((strcmp(ret->d_name, "virtual")) != 0)
                        goto next_entry;

I encountered this since I used a new category, which only contained a 
single word. Adding a hyphen and a 2nd token solved my issue, and now 
qfile knows the file's association.

Is this assumption, that category should be "stringA-stringB" documented 
somewhere?

10x,
Amit

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Alec Warner]

On Mon, Mar 16, 2009 at 9:34 AM, Amit Dor-Shifer <amitds@oversi.com> wrote:
> Hi all.
>
> While working on my overlay, I stumbled on an issue where qfile refused to
> acknowledge an installed file as being part of my package.
>
> Looking into q's implementation (portage-utils-0.1.29), I see:
>
> amit0 portage-utils-0.1.29 # grep -A 2 next_entry ./libq/vdb_get_next_dir.c
> next_entry:
>       ret = readdir(dir);
>       if (ret == NULL) {
> --
>               goto next_entry;
>       if (strchr(ret->d_name, '-') == NULL)
>               if ((strcmp(ret->d_name, "virtual")) != 0)
>                       goto next_entry;
>
> I encountered this since I used a new category, which only contained a
> single word. Adding a hyphen and a 2nd token solved my issue, and now qfile
> knows the file's association.
>
> Is this assumption, that category should be "stringA-stringB" documented
> somewhere?

It is a legacy assumption that we had prior to the introduction of the
'virtual' category into gentoo-x86.

>
> 10x,
> Amit
>
>
>
>

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Ned Ludd]

On Mon, 2009-03-16 at 18:34 +0200, Amit Dor-Shifer wrote:
> Hi all.
> 
> While working on my overlay, I stumbled on an issue where qfile refused 
> to acknowledge an installed file as being part of my package.
> 
> Looking into q's implementation (portage-utils-0.1.29), I see:
> 
> amit0 portage-utils-0.1.29 # grep -A 2 next_entry ./libq/vdb_get_next_dir.c
> next_entry:
>         ret = readdir(dir);
>         if (ret == NULL) {
> --
>                 goto next_entry;
>         if (strchr(ret->d_name, '-') == NULL)
>                 if ((strcmp(ret->d_name, "virtual")) != 0)
>                         goto next_entry;
> 
> I encountered this since I used a new category, which only contained a 
> single word. Adding a hyphen and a 2nd token solved my issue, and now 
> qfile knows the file's association.
> 
> Is this assumption, that category should be "stringA-stringB" documented 
> somewhere?


We made that assumption for portage-utils as they can be used on a
device which has no $PORTDIR at all. So when there is no categories file
that exists we fell back to the rules that have been working well for
the past %d years.  

We changed that behavior however a while ago. I thought this was in the
tree. But I guess not if you are hitting it.

http://sources.gentoo.org/viewcvs.py/gentoo-projects/portage-utils/libq/vdb_get_next_dir.c?r1=1.2&r2=1.3



-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Mike Frysinger]

On Monday 16 March 2009 14:35:15 Ned Ludd wrote:
> On Mon, 2009-03-16 at 18:34 +0200, Amit Dor-Shifer wrote:
> > Hi all.
> >
> > While working on my overlay, I stumbled on an issue where qfile refused
> > to acknowledge an installed file as being part of my package.
> >
> > Looking into q's implementation (portage-utils-0.1.29), I see:
> >
> > amit0 portage-utils-0.1.29 # grep -A 2 next_entry
> > ./libq/vdb_get_next_dir.c next_entry:
> >         ret = readdir(dir);
> >         if (ret == NULL) {
> > --
> >                 goto next_entry;
> >         if (strchr(ret->d_name, '-') == NULL)
> >                 if ((strcmp(ret->d_name, "virtual")) != 0)
> >                         goto next_entry;
> >
> > I encountered this since I used a new category, which only contained a
> > single word. Adding a hyphen and a 2nd token solved my issue, and now
> > qfile knows the file's association.
> >
> > Is this assumption, that category should be "stringA-stringB" documented
> > somewhere?
>
> We made that assumption for portage-utils as they can be used on a
> device which has no $PORTDIR at all. So when there is no categories file
> that exists we fell back to the rules that have been working well for
> the past %d years.
>
> We changed that behavior however a while ago. I thought this was in the
> tree. But I guess not if you are hitting it.
>
> http://sources.gentoo.org/viewcvs.py/gentoo-projects/portage-utils/libq/vdb
>_get_next_dir.c?r1=1.2&r2=1.3

we should do a new release already
-mike

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Ned Ludd]

On Mon, 2009-03-16 at 17:05 -0400, Mike Frysinger wrote:
> On Monday 16 March 2009 14:35:15 Ned Ludd wrote:
> > On Mon, 2009-03-16 at 18:34 +0200, Amit Dor-Shifer wrote:
> > > Hi all.
> > >
> > > While working on my overlay, I stumbled on an issue where qfile refused
> > > to acknowledge an installed file as being part of my package.
> > >
> > > Looking into q's implementation (portage-utils-0.1.29), I see:
> > >
> > > amit0 portage-utils-0.1.29 # grep -A 2 next_entry
> > > ./libq/vdb_get_next_dir.c next_entry:
> > >         ret = readdir(dir);
> > >         if (ret == NULL) {
> > > --
> > >                 goto next_entry;
> > >         if (strchr(ret->d_name, '-') == NULL)
> > >                 if ((strcmp(ret->d_name, "virtual")) != 0)
> > >                         goto next_entry;
> > >
> > > I encountered this since I used a new category, which only contained a
> > > single word. Adding a hyphen and a 2nd token solved my issue, and now
> > > qfile knows the file's association.
> > >
> > > Is this assumption, that category should be "stringA-stringB" documented
> > > somewhere?
> >
> > We made that assumption for portage-utils as they can be used on a
> > device which has no $PORTDIR at all. So when there is no categories file
> > that exists we fell back to the rules that have been working well for
> > the past %d years.
> >
> > We changed that behavior however a while ago. I thought this was in the
> > tree. But I guess not if you are hitting it.
> >
> > http://sources.gentoo.org/viewcvs.py/gentoo-projects/portage-utils/libq/vdb
> >_get_next_dir.c?r1=1.2&r2=1.3
> 
> we should do a new release already
> -mike


Why yes.. Yes you should :)


-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 2011 bytes --]

On Monday 16 March 2009 18:49:04 Ned Ludd wrote:
> On Mon, 2009-03-16 at 17:05 -0400, Mike Frysinger wrote:
> > On Monday 16 March 2009 14:35:15 Ned Ludd wrote:
> > > On Mon, 2009-03-16 at 18:34 +0200, Amit Dor-Shifer wrote:
> > > > Hi all.
> > > >
> > > > While working on my overlay, I stumbled on an issue where qfile
> > > > refused to acknowledge an installed file as being part of my package.
> > > >
> > > > Looking into q's implementation (portage-utils-0.1.29), I see:
> > > >
> > > > amit0 portage-utils-0.1.29 # grep -A 2 next_entry
> > > > ./libq/vdb_get_next_dir.c next_entry:
> > > >         ret = readdir(dir);
> > > >         if (ret == NULL) {
> > > > --
> > > >                 goto next_entry;
> > > >         if (strchr(ret->d_name, '-') == NULL)
> > > >                 if ((strcmp(ret->d_name, "virtual")) != 0)
> > > >                         goto next_entry;
> > > >
> > > > I encountered this since I used a new category, which only contained
> > > > a single word. Adding a hyphen and a 2nd token solved my issue, and
> > > > now qfile knows the file's association.
> > > >
> > > > Is this assumption, that category should be "stringA-stringB"
> > > > documented somewhere?
> > >
> > > We made that assumption for portage-utils as they can be used on a
> > > device which has no $PORTDIR at all. So when there is no categories
> > > file that exists we fell back to the rules that have been working well
> > > for the past %d years.
> > >
> > > We changed that behavior however a while ago. I thought this was in the
> > > tree. But I guess not if you are hitting it.
> > >
> > > http://sources.gentoo.org/viewcvs.py/gentoo-projects/portage-utils/libq
> > >/vdb _get_next_dir.c?r1=1.2&r2=1.3
> >
> > we should do a new release already
>
> Why yes.. Yes you should :)

if you dont do it before me, i'll probably try and do it this weekend.  btw, i 
went through the bug reports and saw qcache crashes ... are those still 
relevant ?
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Ned Ludd]

On Mon, 2009-03-16 at 19:45 -0400, Mike Frysinger wrote:
> On Monday 16 March 2009 18:49:04 Ned Ludd wrote:
> > On Mon, 2009-03-16 at 17:05 -0400, Mike Frysinger wrote:
> > > On Monday 16 March 2009 14:35:15 Ned Ludd wrote:
> > > > On Mon, 2009-03-16 at 18:34 +0200, Amit Dor-Shifer wrote:
> > > > > Hi all.
> > > > >
> > > > > While working on my overlay, I stumbled on an issue where qfile
> > > > > refused to acknowledge an installed file as being part of my package.
> > > > >
> > > > > Looking into q's implementation (portage-utils-0.1.29), I see:
> > > > >
> > > > > amit0 portage-utils-0.1.29 # grep -A 2 next_entry
> > > > > ./libq/vdb_get_next_dir.c next_entry:
> > > > >         ret = readdir(dir);
> > > > >         if (ret == NULL) {
> > > > > --
> > > > >                 goto next_entry;
> > > > >         if (strchr(ret->d_name, '-') == NULL)
> > > > >                 if ((strcmp(ret->d_name, "virtual")) != 0)
> > > > >                         goto next_entry;
> > > > >
> > > > > I encountered this since I used a new category, which only contained
> > > > > a single word. Adding a hyphen and a 2nd token solved my issue, and
> > > > > now qfile knows the file's association.
> > > > >
> > > > > Is this assumption, that category should be "stringA-stringB"
> > > > > documented somewhere?
> > > >
> > > > We made that assumption for portage-utils as they can be used on a
> > > > device which has no $PORTDIR at all. So when there is no categories
> > > > file that exists we fell back to the rules that have been working well
> > > > for the past %d years.
> > > >
> > > > We changed that behavior however a while ago. I thought this was in the
> > > > tree. But I guess not if you are hitting it.
> > > >
> > > > http://sources.gentoo.org/viewcvs.py/gentoo-projects/portage-utils/libq
> > > >/vdb _get_next_dir.c?r1=1.2&r2=1.3
> > >
> > > we should do a new release already
> >
> > Why yes.. Yes you should :)
> 
> if you dont do it before me, i'll probably try and do it this weekend.  



I'd prefer it if you could do it this time. (thanks in advance)

> btw, i 
> went through the bug reports and saw qcache crashes ... are those still 
> relevant ?
> -mike

Yeah. tcort was the guy who wrote most of that. He's retired now.
I never really looked into it much but I think there are some NULL
values he did not check for in the metacache.

There is also a bug with atom parsing iirc on 32bit platforms. gradm was
the test case. Think we need to change from int to long.. Maybe another
with -rX parsing.

-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 649 bytes --]

On Tuesday 17 March 2009 12:59:58 Ned Ludd wrote:
> There is also a bug with atom parsing iirc on 32bit platforms. gradm was
> the test case. Think we need to change from int to long.

the code is documented as having 64bit limitations for any specific component.  
the last release doesnt have the updated work i did in qatom to handle the 
latest atom spec though, and that includes moving from 32bit to 64bit for 
components ...

> Maybe another with -rX parsing.

if you're thinking of the open bug, that's an eprefix specific extension.  
they turned the X in -rX into a floating point #.  which isnt supported 
currently.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Ned Ludd]

On Tue, 2009-03-17 at 13:27 -0400, Mike Frysinger wrote:
> On Tuesday 17 March 2009 12:59:58 Ned Ludd wrote:
> > There is also a bug with atom parsing iirc on 32bit platforms. gradm was
> > the test case. Think we need to change from int to long.
> 
> the code is documented as having 64bit limitations for any specific component.  
> the last release doesnt have the updated work i did in qatom to handle the 
> latest atom spec though, and that includes moving from 32bit to 64bit for 
> components ...

Sounds good. 

> 
> > Maybe another with -rX parsing.
> 
> if you're thinking of the open bug, that's an eprefix specific extension.  
> they turned the X in -rX into a floating point #.  which isnt supported 
> currently.
> -mike

I don't think that was it. But I can't recall well enough off the top of
my head the problem that somebody pointed out to me one day on irc while
I was probably too busy.

-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Ned Ludd]

On Tue, 2009-03-17 at 10:50 -0700, Ned Ludd wrote:
> On Tue, 2009-03-17 at 13:27 -0400, Mike Frysinger wrote:
> > On Tuesday 17 March 2009 12:59:58 Ned Ludd wrote:
> > > There is also a bug with atom parsing iirc on 32bit platforms. gradm was
> > > the test case. Think we need to change from int to long.
> > 
> > the code is documented as having 64bit limitations for any specific component.  
> > the last release doesnt have the updated work i did in qatom to handle the 
> > latest atom spec though, and that includes moving from 32bit to 64bit for 
> > components ...
> 
> Sounds good. 
> 
> > 
> > > Maybe another with -rX parsing.
> > 
> > if you're thinking of the open bug, that's an eprefix specific extension.  
> > they turned the X in -rX into a floating point #.  which isnt supported 
> > currently.
> > -mike
> 
> I don't think that was it. But I can't recall well enough off the top of
> my head the problem that somebody pointed out to me one day on irc while
> I was probably too busy.

Mike,
The error was pointed out to me again today on irc by jmbsvicetto and
hoffie, which reminded me of what I had forgot before in this thread. 

The problem was/is that qpkg is not handling -rX extensions properly. 


-- 
Ned Ludd <solar@gentoo.org>
Gentoo Linux

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1546 bytes --]

On Thursday 16 April 2009 19:05:46 Ned Ludd wrote:
> On Tue, 2009-03-17 at 10:50 -0700, Ned Ludd wrote:
> > On Tue, 2009-03-17 at 13:27 -0400, Mike Frysinger wrote:
> > > On Tuesday 17 March 2009 12:59:58 Ned Ludd wrote:
> > > > There is also a bug with atom parsing iirc on 32bit platforms. gradm
> > > > was the test case. Think we need to change from int to long.
> > >
> > > the code is documented as having 64bit limitations for any specific
> > > component. the last release doesnt have the updated work i did in qatom
> > > to handle the latest atom spec though, and that includes moving from
> > > 32bit to 64bit for components ...
> >
> > Sounds good.
> >
> > > > Maybe another with -rX parsing.
> > >
> > > if you're thinking of the open bug, that's an eprefix specific
> > > extension. they turned the X in -rX into a floating point #.  which
> > > isnt supported currently.
> >
> > I don't think that was it. But I can't recall well enough off the top of
> > my head the problem that somebody pointed out to me one day on irc while
> > I was probably too busy.
>
> The error was pointed out to me again today on irc by jmbsvicetto and
> hoffie, which reminded me of what I had forgot before in this thread.
>
> The problem was/is that qpkg is not handling -rX extensions properly.

you'll have to be more specific.  like i said, -rX extensions are a prefix 
extension and not part of the standard tree and/or spec.  i'm not going to 
implement every random thing that someone feels like adding.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Ned Ludd]

On Sat, 2009-04-18 at 12:55 -0400, Mike Frysinger wrote:
> On Thursday 16 April 2009 19:05:46 Ned Ludd wrote:
> > On Tue, 2009-03-17 at 10:50 -0700, Ned Ludd wrote:
> > > On Tue, 2009-03-17 at 13:27 -0400, Mike Frysinger wrote:
> > > > On Tuesday 17 March 2009 12:59:58 Ned Ludd wrote:
> > > > > There is also a bug with atom parsing iirc on 32bit platforms. gradm
> > > > > was the test case. Think we need to change from int to long.
> > > >
> > > > the code is documented as having 64bit limitations for any specific
> > > > component. the last release doesnt have the updated work i did in qatom
> > > > to handle the latest atom spec though, and that includes moving from
> > > > 32bit to 64bit for components ...
> > >
> > > Sounds good.
> > >
> > > > > Maybe another with -rX parsing.
> > > >
> > > > if you're thinking of the open bug, that's an eprefix specific
> > > > extension. they turned the X in -rX into a floating point #.  which
> > > > isnt supported currently.
> > >
> > > I don't think that was it. But I can't recall well enough off the top of
> > > my head the problem that somebody pointed out to me one day on irc while
> > > I was probably too busy.
> >
> > The error was pointed out to me again today on irc by jmbsvicetto and
> > hoffie, which reminded me of what I had forgot before in this thread.
> >
> > The problem was/is that qpkg is not handling -rX extensions properly.
> 
> you'll have to be more specific.  like i said, -rX extensions are a prefix 
> extension and not part of the standard tree and/or spec.  i'm not going to 
> implement every random thing that someone feels like adding.
> -mike


Heh. I don't think you understand the problem yet. Not a feature
request.. It's a real bug/regression. See the bug# that jmbsvicetto
filed this morn about it.

https://bugs.gentoo.org/266646

Re: [gentoo-portage-dev] qfile assumes category names contain a hyphen

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1625 bytes --]

On Saturday 18 April 2009 13:08:24 Ned Ludd wrote:
> On Sat, 2009-04-18 at 12:55 -0400, Mike Frysinger wrote:
> > On Thursday 16 April 2009 19:05:46 Ned Ludd wrote:
> > > On Tue, 2009-03-17 at 10:50 -0700, Ned Ludd wrote:
> > > > On Tue, 2009-03-17 at 13:27 -0400, Mike Frysinger wrote:
> > > > > On Tuesday 17 March 2009 12:59:58 Ned Ludd wrote:
> > > > > > Maybe another with -rX parsing.
> > > > >
> > > > > if you're thinking of the open bug, that's an eprefix specific
> > > > > extension. they turned the X in -rX into a floating point #.  which
> > > > > isnt supported currently.
> > > >
> > > > I don't think that was it. But I can't recall well enough off the top
> > > > of my head the problem that somebody pointed out to me one day on irc
> > > > while I was probably too busy.
> > >
> > > The error was pointed out to me again today on irc by jmbsvicetto and
> > > hoffie, which reminded me of what I had forgot before in this thread.
> > >
> > > The problem was/is that qpkg is not handling -rX extensions properly.
> >
> > you'll have to be more specific.  like i said, -rX extensions are a
> > prefix extension and not part of the standard tree and/or spec.  i'm not
> > going to implement every random thing that someone feels like adding.
>
> Heh. I don't think you understand the problem yet. Not a feature
> request.. It's a real bug/regression. See the bug# that jmbsvicetto
> filed this morn about it.
>
> https://bugs.gentoo.org/266646

yes, that is the "more information" that i required.  that certainly looks 
bad.  i'm guessing 0.1.29 works but 0.2 fails ?
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]