[gentoo-dev] SRC_URI component naming collision

[gentoo-dev] SRC_URI component naming collision

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 865 bytes --]

Two ways this one can occur.

Way the first: foo-1.0 has a file in SRC_URI called foo.pdf. Then
foo-1.1 comes along, and has a different foo.pdf.

Way the second: foo-1.0 has a file called examples-1.0.tar.bz2. bar-1.0
also has a file called examples-1.0.tar.bz2.

To avoid this, ensure that your packages use versioned SRC_URI
component names, and that the name part is something that's reasonably
likely to be unique (e.g. includes the package name).

Side note: if the packages in question are fetch restricted, you're
screwed, and will not be able to add them to the tree.

Current offenders shall be receiving bugs shortly, since That Which
Shall Not Be Named now checks for this.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Jochen Maes]

[-- Attachment #1: Type: text/plain, Size: 987 bytes --]

Ciaran McCreesh wrote:

>Two ways this one can occur.
>
>Way the first: foo-1.0 has a file in SRC_URI called foo.pdf. Then
>foo-1.1 comes along, and has a different foo.pdf.
>
>Way the second: foo-1.0 has a file called examples-1.0.tar.bz2. bar-1.0
>also has a file called examples-1.0.tar.bz2.
>
>To avoid this, ensure that your packages use versioned SRC_URI
>component names, and that the name part is something that's reasonably
>likely to be unique (e.g. includes the package name).
>
>  
>
this gives problems on a number off packages... i'm not pro...

>Side note: if the packages in question are fetch restricted, you're
>screwed, and will not be able to add them to the tree.
>
>Current offenders shall be receiving bugs shortly, since That Which
>Shall Not Be Named now checks for this.
>
>  
>


-- 
"Defer no time, delays have dangerous ends"
"Ne humanus crede"

Jochen Maes                     
Gentoo Linux
Gentoo Belgium
http://sejo.be
http://gentoo.be
http://gentoo.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Doug Goldstein]

[-- Attachment #1: Type: text/plain, Size: 274 bytes --]

Ciaran McCreesh wrote:
-snip-
> Current offenders shall be receiving bugs shortly, since That Which
> Shall Not Be Named now checks for this.
> 
The One Tool To Rule Them All? TOT4A -> TOTAL

-- 
Doug Goldstein <cardoe@gentoo.org>
http://dev.gentoo.org/~cardoe/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 1181 bytes --]

On Friday 24 February 2006 15:19, Ciaran McCreesh wrote:
> Two ways this one can occur.
>
> Way the first: foo-1.0 has a file in SRC_URI called foo.pdf. Then
> foo-1.1 comes along, and has a different foo.pdf.
>
> Way the second: foo-1.0 has a file called examples-1.0.tar.bz2. bar-1.0
> also has a file called examples-1.0.tar.bz2.
>
> To avoid this, ensure that your packages use versioned SRC_URI
> component names, and that the name part is something that's reasonably
> likely to be unique (e.g. includes the package name).
>
> Side note: if the packages in question are fetch restricted, you're
> screwed, and will not be able to add them to the tree.
>
> Current offenders shall be receiving bugs shortly, since That Which
> Shall Not Be Named now checks for this.

What about introducing a new variable in the ebuild file:
DIST_PREFIX that has as default value ${PN}. This should not break anything 
for unaware portage versions. For aware portage versions, the files would be 
retrieved from ${DISTDIR}/${DIST_PREFIX} instead of ${DISTDIR}

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Drake Wyrm]

[-- Attachment #1: Type: text/plain, Size: 1506 bytes --]

Paul de Vrieze <pauldv@gentoo.org> wrote:
> On Friday 24 February 2006 15:19, Ciaran McCreesh wrote:
[snip]
> > To avoid this, ensure that your packages use versioned SRC_URI
> > component names, and that the name part is something that's
> > reasonably likely to be unique (e.g. includes the package name).

Unfortunately, that's not always a possibility. I've bugged at least one
upstream about non-versioned sources. How does one work around that if
they won't fix it?

> What about introducing a new variable in the ebuild file: DIST_PREFIX
> that has as default value ${PN}. This should not break anything for
> unaware portage versions. For aware portage versions, the files would
> be retrieved from ${DISTDIR}/${DIST_PREFIX} instead of ${DISTDIR}

That introduces a problem of redundancy. In some cases, more than one
package will use a given set of sources, and it will need to be fetched
and stored once for each package. Perhaps defaulting DIST_PREFIX to ""
and allowing it to be used for workarounds in the odd cases could work.

Just for the sake of demonstration, the following should show a list of
the source files which are used multiple times on any given system and
how many times they are used:

	emerge -p --fetch-all-uri --emptytree world 2>&1 1>/dev/null |
		sed -e '/^$/d;s/.*\///' |
		sort | uniq -D | uniq -c | sort -n

-- 
There are problems in today's world that cannot be
solved by the level of thinking that created them.
  -- Albert Einstein

[-- Attachment #2: Type: application/pgp-signature, Size: 199 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 1120 bytes --]

On Saturday 25 February 2006 22:29, Drake Wyrm wrote:
> > What about introducing a new variable in the ebuild file: DIST_PREFIX
> > that has as default value ${PN}. This should not break anything for
> > unaware portage versions. For aware portage versions, the files would
> > be retrieved from ${DISTDIR}/${DIST_PREFIX} instead of ${DISTDIR}
>
> That introduces a problem of redundancy. In some cases, more than one
> package will use a given set of sources, and it will need to be fetched
> and stored once for each package. Perhaps defaulting DIST_PREFIX to ""
> and allowing it to be used for workarounds in the odd cases could work.

I don't think that there is redundancy often. For some packages there might 
be, but those could set the value. There are however many packages that come 
with files like patch.4.2.52.4 (for sys-libs/db-4.2.52). From the name it's 
impossible to know what it belongs to. And the fact that there is no 
collision is just by chance, not by name convention.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 1904 bytes --]

On Fri, 2006-02-24 at 14:19 +0000, Ciaran McCreesh wrote:
> Two ways this one can occur.

[snip]

Third way ... upstream is a provider of commercial software, and
releases different editions of the same software with identical
filenames.

> Side note: if the packages in question are fetch restricted, you're
> screwed, and will not be able to add them to the tree.

There's two more important issues here that need to be cleaned up.  

I have the QA team on bug #123926 asserting that they have the right to
tell me to remove packages from the tree, because of basename $SRC_URI
filename collisions.

To the best of my knowledge, there's no policy document in existence
empowering the QA team to order package maintainers to remove packages
from Portage.  I've asked the team to provide a copy if one exists, but
I haven't seen one yet.  The team have (twice now) instead stated that
the email at the top of this thread is their authority.

Also, I cannot find this SRC_URI rule (as being applied by the QA team)
in any official Gentoo policy document.  I'm very concerned about the
judgement of a QA team that is choosing to try and apply policies that
it hasn't documented, and which haven't been through a formal approval
process such as GLEPs.

I'll contact the council separately, and ask that they look at two
things:

a) What the QA team is and isn't empowered to do
b) The approval process that the QA team must follow before imposing
tree-wide changes on other developers.

Best regards,
Stu
-- 
Stuart Herbert                                         stuart@gentoo.org
Gentoo Developer                                  http://www.gentoo.org/
                                          http://blog.stuartherbert.com/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Mike Frysinger]

On Sunday 26 February 2006 14:45, Stuart Herbert wrote:
> Also, I cannot find this SRC_URI rule (as being applied by the QA team)
> in any official Gentoo policy document.

that's because it's common sense ... filename collisions just dont work
-mike
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] SRC_URI component naming collision

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 1002 bytes --]

On Sun, 2006-02-26 at 14:56 -0500, Mike Frysinger wrote:
> that's because it's common sense ... filename collisions just dont work
> -mike

This set of packages has been this way since October 2003, and if it was
a real problem for users, you'd see that reflected in bugzilla and in
the forums.  It isn't.

The issue is also covered in our NX Guide, which was last updated August
2004 [1].  The Guide has explicit instructions on how to switch from one
'edition' to another (which is the only circumstance where the filename
collisions problem can occur).

[1] http://www.gentoo.org/doc/en/nx-guide.xml

Best regards,
Stu
-- 
Stuart Herbert                                         stuart@gentoo.org
Gentoo Developer                                  http://www.gentoo.org/
                                          http://blog.stuartherbert.com/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2129 bytes --]

On Sun, 26 Feb 2006 19:45:41 +0000 Stuart Herbert <stuart@gentoo.org>
wrote:
| On Fri, 2006-02-24 at 14:19 +0000, Ciaran McCreesh wrote:
| > Two ways this one can occur.
| 
| [snip]
| 
| Third way ... upstream is a provider of commercial software, and
| releases different editions of the same software with identical
| filenames.

Then you must talk to upstream and get them to change their ways.

| I have the QA team on bug #123926 asserting that they have the right
| to tell me to remove packages from the tree, because of basename
| $SRC_URI filename collisions.

We don't *want* to remove the package from the tree. We want to get
the breakage fixed.

| To the best of my knowledge, there's no policy document in existence
| empowering the QA team to order package maintainers to remove packages
| from Portage.  I've asked the team to provide a copy if one exists,
| but I haven't seen one yet.  The team have (twice now) instead stated
| that the email at the top of this thread is their authority.

There's no policy document in existence that explicitly says that you
(by name) can add stuff to the tree either. Most of our policy is
undocumented, because it's impossible to cover every situation. The
number one rule, however, is to be sensible and not commit things that
cause breakages.

Again, we don't *want* to remove it. On the other hand, if you refuse
to work with us to get the problem fixed, we're going to have to do
something about it ourselves.

| Also, I cannot find this SRC_URI rule (as being applied by the QA
| team) in any official Gentoo policy document.

As I recall, pretty much nothing about digests at all is in any
official policy document. Nor is nearly anything else on any
development topic. However, that it is not explicitly forbidden does
not mean that it should be done.

Where in policy does it say that you shouldn't commit pictures of
teletubbies in SVG format in the tree?

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 3194 bytes --]

On Sun, 2006-02-26 at 20:00 +0000, Ciaran McCreesh wrote:
> Then you must talk to upstream and get them to change their ways.

Already covered in the (growing) discussion in bug #123926.  UPSTREAM
have previously been contacted over the issue, and have not changed
their release policy.

> We don't *want* to remove the package from the tree. We want to get
> the breakage fixed.

In practical, does-it-affect-the-user terms, it's not broken.  It's also
covered by our NX Guide in the desktop docs section, just to be sure.

> There's no policy document in existence that explicitly says that you
> (by name) can add stuff to the tree either. Most of our policy is
> undocumented, because it's impossible to cover every situation. The
> number one rule, however, is to be sensible and not commit things that
> cause breakages.

I'm a little rusty at this - it's six years since I ran the DDS4 test
team for HP - but isn't one of the internationally recognised
requirements of every recognised QA standard that exists that a QA
policy should be documented?

On a practical note, I don't understand how you expect developers to
follow an undocumented QA process.  Sorry, I just don't get that one.

> Again, we don't *want* to remove it. On the other hand, if you refuse
> to work with us to get the problem fixed, we're going to have to do
> something about it ourselves.

I've refused to do two things, and only two things:

a) rename the files, and mirror them ourselves, because legally I don't
believe we can do this for these packages, and
b) to remove the packages from the tree

Everything else is up for discussion.  I think it's unreasonable to say
that I'm refusing to work with you.

Bearing in mind the discussion that's happened in the bug, on IRC with
Halcy0n, and in this mailing list, please understand this: I don't
believe that the QA team has provided evidence that it has the authority
to do anything to these packages over this SRC_URI issue.  If the team
chooses to take unilateral action, I'll be left with no choice but to
file a formal complaint against the team as a consequence.

I'm happy (and have suggested earlier) that this issue needs to go to
the council to be resolved.

> As I recall, pretty much nothing about digests at all is in any
> official policy document. Nor is nearly anything else on any
> development topic. However, that it is not explicitly forbidden does
> not mean that it should be done.
> 
> Where in policy does it say that you shouldn't commit pictures of
> teletubbies in SVG format in the tree?

The issue at hand is that the QA team is, in this case, repeatedly
asking for something it doesn't have the authority to insist on.  I also
think you're being unreasonable in this specific case.

Best regards,
Stu
-- 
Stuart Herbert                                         stuart@gentoo.org
Gentoo Developer                                  http://www.gentoo.org/
                                          http://blog.stuartherbert.com/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 3531 bytes --]

On Sun, 26 Feb 2006 20:46:37 +0000 Stuart Herbert <stuart@gentoo.org>
wrote:
| On Sun, 2006-02-26 at 20:00 +0000, Ciaran McCreesh wrote:
| > Then you must talk to upstream and get them to change their ways.
| 
| Already covered in the (growing) discussion in bug #123926.  UPSTREAM
| have previously been contacted over the issue, and have not changed
| their release policy.

Ok, so given that this is a closed source application, if upstream
won't cooperate on something as simple as this, why do you expect them
to cooperate with you on bugs or security issues?

| I'm a little rusty at this - it's six years since I ran the DDS4 test
| team for HP - but isn't one of the internationally recognised
| requirements of every recognised QA standard that exists that a QA
| policy should be documented?

Utterly impractical. If you want to hire a dozen people to do this, go
right ahead. But then, if you're hiring a dozen people, there are far
more useful things they could be doing.

| On a practical note, I don't understand how you expect developers to
| follow an undocumented QA process.  Sorry, I just don't get that one.

We expect you to be sensible. When this fails, we point out issues and
work with the developer to get things fixed.

Over the past couple of weeks, QA has gotten somewhere around a couple
of hundred tree issues fixed. In every situation bar three, the
response has been "thanks for pointing this out". In another two, the
response was to fix the bug silently.

No-one is expecting perfection. QA is here first and foremost to find
issues, get them fixed and try to prevent the same breakage from being
repeated.

| Everything else is up for discussion.  I think it's unreasonable to
| say that I'm refusing to work with you.

You're repeatedly closing off the bug rather than suggesting
alternative ways of fixing the issue. There's been one possibility
mentioned in this thread already, but it can't go anywhere unless
someone with an affected package (which is you) is prepared to go to
the Portage team with a justification.

| Bearing in mind the discussion that's happened in the bug, on IRC with
| Halcy0n, and in this mailing list, please understand this: I don't
| believe that the QA team has provided evidence that it has the
| authority to do anything to these packages over this SRC_URI issue.
| If the team chooses to take unilateral action, I'll be left with no
| choice but to file a formal complaint against the team as a
| consequence.

See Daniel's post in the thread. The council has already agreed that QA
has authority.

| I'm happy (and have suggested earlier) that this issue needs to go to
| the council to be resolved.

Being done. The council will be asked to approve a more specific
description of QA's authority in the next meeting, since our existing
mandate is "listen to the QA team because they're working of valid
policy. if you dont, devrel will take action" (sic).

| The issue at hand is that the QA team is, in this case, repeatedly
| asking for something it doesn't have the authority to insist on.  I
| also think you're being unreasonable in this specific case.

We're asking you to work with us in fixing a tree breakage. That's our
goal here. We can't do this if you just go around closing off bugs and
refusing to cooperate.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 2699 bytes --]

On Sun, 2006-02-26 at 21:04 +0000, Ciaran McCreesh wrote:
> Ok, so given that this is a closed source application, if upstream
> won't cooperate on something as simple as this, why do you expect them
> to cooperate with you on bugs or security issues?

That's not the issue here.  The issue here is whether the QA team is
entitled to be requesting the removal of packages in this specific
instance.

There are never any guarantees that any UPSTREAM will co-operate with us
on bugs or security issues.  If we can't live with the issues, and we
can't fix them, the packages get dropped.  I've no problem with that.

> | Everything else is up for discussion.  I think it's unreasonable to
> | say that I'm refusing to work with you.
> 
> You're repeatedly closing off the bug rather than suggesting
> alternative ways of fixing the issue. 

I think, in this specific case, there are better things to spend the
time on.  I don't have a queue of users telling me that the way we
handle this today is a problem.  There's no evidence that, in this
specific case, there is a problem out in the real world.

> There's been one possibility
> mentioned in this thread already, but it can't go anywhere unless
> someone with an affected package (which is you) is prepared to go to
> the Portage team with a justification.

Hang on a moment.  It's not clear to me why I must go to the Portage
team for a change, when it's the QA team demanding change?  As the QA
team wants the change, why don't you go to the Portage team and ask them
to implement DEST_PREFIX?

Because (quite rightly) you'd rather the Portage team did other things
too.

> See Daniel's post in the thread. The council has already agreed that QA
> has authority.

Daniel also said that the QA team was supposed to be coming back to the
council with more information.

> | The issue at hand is that the QA team is, in this case, repeatedly
> | asking for something it doesn't have the authority to insist on.  I
> | also think you're being unreasonable in this specific case.
> 
> We're asking you to work with us in fixing a tree breakage. That's our
> goal here. We can't do this if you just go around closing off bugs and
> refusing to cooperate.

Please stop spreading FUD, and libelling my name here.  

Best regards,
Stu
-- 
Stuart Herbert                                         stuart@gentoo.org
Gentoo Developer                                  http://www.gentoo.org/
                                          http://blog.stuartherbert.com/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2815 bytes --]

On Sun, 26 Feb 2006 21:30:22 +0000 Stuart Herbert <stuart@gentoo.org>
wrote:
| On Sun, 2006-02-26 at 21:04 +0000, Ciaran McCreesh wrote:
| > Ok, so given that this is a closed source application, if upstream
| > won't cooperate on something as simple as this, why do you expect
| > them to cooperate with you on bugs or security issues?
| 
| That's not the issue here.  The issue here is whether the QA team is
| entitled to be requesting the removal of packages in this specific
| instance.

The issue is whether you have the right to leave broken packages in the
tree. I don't see any policy document granting you that right.

| There are never any guarantees that any UPSTREAM will co-operate with
| us on bugs or security issues.  If we can't live with the issues, and
| we can't fix them, the packages get dropped.  I've no problem with
| that.

Sure. And if upstream won't even cooperate to the extent of renaming a
file, how do you expect them to react when we require something less
trvial?

| > | Everything else is up for discussion.  I think it's unreasonable
| > | to say that I'm refusing to work with you.
| > 
| > You're repeatedly closing off the bug rather than suggesting
| > alternative ways of fixing the issue. 
| 
| I think, in this specific case, there are better things to spend the
| time on.  I don't have a queue of users telling me that the way we
| handle this today is a problem.  There's no evidence that, in this
| specific case, there is a problem out in the real world.

It's so bad a problem that you even had to document it in the user
guide and tell people to use some nasty hacked workaround.

| Hang on a moment.  It's not clear to me why I must go to the Portage
| team for a change, when it's the QA team demanding change?  As the QA
| team wants the change, why don't you go to the Portage team and ask
| them to implement DEST_PREFIX?

We don't have a legitimate demonstration package, and we're not going
to go and ask the Portage team to make code changes to support
hypothetical speculation. You're the only one with a test case here.

| > | The issue at hand is that the QA team is, in this case, repeatedly
| > | asking for something it doesn't have the authority to insist on.
| > | I also think you're being unreasonable in this specific case.
| > 
| > We're asking you to work with us in fixing a tree breakage. That's
| > our goal here. We can't do this if you just go around closing off
| > bugs and refusing to cooperate.
| 
| Please stop spreading FUD, and libelling my name here.  

You've closed that bug five times now without fixing it.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 2365 bytes --]

On Sun, 2006-02-26 at 21:40 +0000, Ciaran McCreesh wrote:
> The issue is whether you have the right to leave broken packages in the
> tree. I don't see any policy document granting you that right.

From a discussion in #-portage, I understand that ferringb has already
told the QA team that file clashes in distfiles is legitimate, and has
no interest in implementing DEST_PREFIX support to tackle the problem. 

Unfortunately, I don't have a mailing list post or an IRC log of that
discussion between ferringb and the QA team to reference here :(

> Sure. And if upstream won't even cooperate to the extent of renaming a
> file, how do you expect them to react when we require something less
> trvial?

It's still not the issue here, no matter how you try and re-introduce it
to the discussion.

> It's so bad a problem that you even had to document it in the user
> guide and tell people to use some nasty hacked workaround.

"so bad a problem"?  "nasty hacked workaround"?  

I don't understand why you feel the need to be so alarmist over this.

> We don't have a legitimate demonstration package, and we're not going
> to go and ask the Portage team to make code changes to support
> hypothetical speculation. You're the only one with a test case here.

I don't agree - you're the ones making a mountain out of a grain of sand
- but anyway.  I've had a chat in #-portage, and there's no support for
adding DEST_PREFIX into Portage at this time.

> | Please stop spreading FUD, and libelling my name here.  
> 
> You've closed that bug five times now without fixing it.

Yes I have.  

I carefully considered the QA team's concerns, and the proposed
solutions, and felt that - in this specific case - the bug doesn't have
enough merit.  And then the bug degenerated into the QA team being
repeatedly asked - and unable to provide - any evidence that they're
entitled to push for the package to be removed.

What's your excuse? :)

Best regards,
Stu
-- 
Stuart Herbert                                         stuart@gentoo.org
Gentoo Developer                                  http://www.gentoo.org/
                                          http://blog.stuartherbert.com/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1655 bytes --]

On Sun, 26 Feb 2006 22:13:32 +0000 Stuart Herbert <stuart@gentoo.org>
wrote:
| On Sun, 2006-02-26 at 21:40 +0000, Ciaran McCreesh wrote:
| > The issue is whether you have the right to leave broken packages in
| > the tree. I don't see any policy document granting you that right.
| 
| From a discussion in #-portage, I understand that ferringb has already
| told the QA team that file clashes in distfiles is legitimate, and has
| no interest in implementing DEST_PREFIX support to tackle the
| problem. 

Which is probably a good thing, since there's exactly one affected set
of packages...

| > | Please stop spreading FUD, and libelling my name here.  
| > 
| > You've closed that bug five times now without fixing it.
| 
| Yes I have.  
| 
| I carefully considered the QA team's concerns, and the proposed
| solutions, and felt that - in this specific case - the bug doesn't
| have enough merit.  And then the bug degenerated into the QA team
| being repeatedly asked - and unable to provide - any evidence that
| they're entitled to push for the package to be removed.

There are plenty of options you could have taken rather than repeatedly
closing the bug off. From the looks of the rest of this thread, you've
decided to go with one of these options. A comment on the bug saying
"I don't like the proposed fix. Here's how I'd solve it" would be fine.
Repeatedly closing the bug off and trying to pretend that there isn't a
problem when there is is not.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 579 bytes --]

On Sunday 26 February 2006 22:40, Ciaran McCreesh wrote:
> The issue is whether you have the right to leave broken packages in the
> tree. I don't see any policy document granting you that right.

The general consensus over the years has been that if something cannot be 
fixed due to portage problems, then we do what necessary to warn users 
about it, but keep the package. In this regard also look at various 
dependency cycles, and/or use flag dependencies.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 863 bytes --]

On Mon, 27 Feb 2006 11:02:57 +0100 Paul de Vrieze <pauldv@gentoo.org>
wrote:
| On Sunday 26 February 2006 22:40, Ciaran McCreesh wrote:
| > The issue is whether you have the right to leave broken packages in
| > the tree. I don't see any policy document granting you that right.
| 
| The general consensus over the years has been that if something
| cannot be fixed due to portage problems, then we do what necessary to
| warn users about it, but keep the package. In this regard also look
| at various dependency cycles, and/or use flag dependencies.

The general consensus has been to implement the best available
workaround, if one is doable, and just remove the thing where it's not.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 1575 bytes --]

Ciaran McCreesh wrote:
> On Mon, 27 Feb 2006 11:02:57 +0100 Paul de Vrieze <pauldv@gentoo.org>
> wrote:
> | On Sunday 26 February 2006 22:40, Ciaran McCreesh wrote:
> | > The issue is whether you have the right to leave broken packages in
> | > the tree. I don't see any policy document granting you that right.
> | 
> | The general consensus over the years has been that if something
> | cannot be fixed due to portage problems, then we do what necessary to
> | warn users about it, but keep the package. In this regard also look
> | at various dependency cycles, and/or use flag dependencies.
> 
> The general consensus has been to implement the best available
> workaround, if one is doable, and just remove the thing where it's not.

Where is this general consensus documented (other than an email sent out
a few days ago). I'd have to go with Paul on this assumption. I don't
see the problem with keeping a package such as stu's in portage as long
as it doesn't affect other users. Do you honesty expect that we will get
a sterile tree out of this? Please focus your QA efforts are more
important and visible issues. Going on a witch hunt to fix one problem
compared to the bigger issues we know we have is simply silly. This is
really starting to look like a power issue rather than a QA issue.

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

On Mon, 27 Feb 2006 10:46:23 -0600 Lance Albertson
<ramereth@gentoo.org> wrote:
| Where is this general consensus documented (other than an email sent
| out a few days ago). I'd have to go with Paul on this assumption. I
| don't see the problem with keeping a package such as stu's in portage
| as long as it doesn't affect other users. Do you honesty expect that
| we will get a sterile tree out of this? Please focus your QA efforts
| are more important and visible issues. Going on a witch hunt to fix
| one problem compared to the bigger issues we know we have is simply
| silly. This is really starting to look like a power issue rather than
| a QA issue.

You know, funnily enough, QA has filed a whole heap of bugs on the
conflicting digest issue. With every other maintainer for bugs we've
filed, the developer in question has worked with us to fix the issue,
and thanked us for pointing out the problem. The only reason this one
has gone so far is because of Stuart repeatedly closing the bug off and
refusing to discuss alternatives.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Daniel Ostrow]

> I'll contact the council separately, and ask that they look at two
> things:
> 
> a) What the QA team is and isn't empowered to do
> b) The approval process that the QA team must follow before imposing
> tree-wide changes on other developers.

According to prior council meeting logs:

15:14 <@vapier> QA works off of agreed policy.  policy was put forth by
developers and validated by council.
15:14 <@seemant> the fact is, QA can scream all they want -- but if
there's no *authority* behind them, devs might be inclined to feel free
to ignore QA
15:14 <@Azarah> and if they do, they get sorted by devrel in theory
15:15 <@vapier> so they have authority now.  listen to the QA team
because they're working of valid policy.  if you dont, devrel will take
action.

So the only question in my mind is what does it take for something to
become 'vaild policy'. I believe the QA team is working on a writeup to
present to the council to address that issue.

--Dan

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] SRC_URI component naming collision

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 591 bytes --]

On Fri, Feb 24, 2006 at 02:19:40PM +0000, Ciaran McCreesh wrote:
> Side note: if the packages in question are fetch restricted, you're
> screwed, and will not be able to add them to the tree.
Actually, there is a solution for this, and it's reasonable logical.
Don't use the same name that upstream does for the files.

Simply tell the user to download X and place it in $DISTDIR renaming it
to X-foo-bar, where's you've chosen X-foo-bar to avoid conflicts.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 241 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 611 bytes --]

On Sun, 2006-02-26 at 13:29 -0800, Robin H. Johnson wrote:
> Simply tell the user to download X and place it in $DISTDIR renaming it
> to X-foo-bar, where's you've chosen X-foo-bar to avoid conflicts.

That works for me.

Best regards,
Stu
-- 
Stuart Herbert                                         stuart@gentoo.org
Gentoo Developer                                  http://www.gentoo.org/
                                          http://blog.stuartherbert.com/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Daniel Ostrow]

On Sun, 2006-02-26 at 21:54 +0000, Stuart Herbert wrote:
> On Sun, 2006-02-26 at 13:29 -0800, Robin H. Johnson wrote:
> > Simply tell the user to download X and place it in $DISTDIR renaming it
> > to X-foo-bar, where's you've chosen X-foo-bar to avoid conflicts.
> 
> That works for me.
> 
> Best regards,
> Stu

That would work for fetch restricted packages, not nomirrored ones.

--Dan

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] SRC_URI component naming collision

[Stuart Herbert]

[-- Attachment #1: Type: text/plain, Size: 765 bytes --]

On Sun, 2006-02-26 at 17:02 -0500, Daniel Ostrow wrote:
> That would work for fetch restricted packages, not nomirrored ones.
> 
> --Dan

/me nods.  That's what we'll have to do.  Unfortunately, it leaves users
with a worse experience than before, but until I can find a way to reach
the QA team and help them see my pov as the package maintainer, what
else can I do? :(

Best regards,
Stu
-- 
Stuart Herbert                                         stuart@gentoo.org
Gentoo Developer                                  http://www.gentoo.org/
                                          http://blog.stuartherbert.com/

GnuGP key id# F9AFC57C available from http://pgp.mit.edu
Key fingerprint = 31FB 50D4 1F88 E227 F319  C549 0C2F 80BA F9AF C57C
--


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 775 bytes --]

On Sun, 26 Feb 2006 22:17:33 +0000 Stuart Herbert <stuart@gentoo.org>
wrote:
| On Sun, 2006-02-26 at 17:02 -0500, Daniel Ostrow wrote:
| > That would work for fetch restricted packages, not nomirrored ones.
| 
| /me nods.  That's what we'll have to do.  Unfortunately, it leaves
| users with a worse experience than before, but until I can find a way
| to reach the QA team and help them see my pov as the package
| maintainer, what else can I do? :(

It leaves them without a weird Portage error message, and it removes
the assumption that the user will study a particular document in great
detail.

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Daniel Ostrow]

On Sun, 2006-02-26 at 22:17 +0000, Stuart Herbert wrote:
> On Sun, 2006-02-26 at 17:02 -0500, Daniel Ostrow wrote:
> > That would work for fetch restricted packages, not nomirrored ones.
> > 
> > --Dan
> 
> /me nods.  That's what we'll have to do.  Unfortunately, it leaves users
> with a worse experience than before, but until I can find a way to reach
> the QA team and help them see my pov as the package maintainer, what
> else can I do? :(

Don't know why this didn't occur to me before but this is exactly what
the Java team does for IBM's jdk/jre. IBM releases all micro releases
under a tarball of the same name so end users already have to rename it
before placing it in ${DISTDIR}. The only difference there is IBM's
files were already fetch restricted.

In any event, even though I'm not part of the QA team, thanks for
accepting an alternate solution.

--Dan

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] SRC_URI component naming collision

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 962 bytes --]

Daniel Ostrow wrote:
> On Sun, 2006-02-26 at 22:17 +0000, Stuart Herbert wrote:
> 
>>On Sun, 2006-02-26 at 17:02 -0500, Daniel Ostrow wrote:
>>
>>>That would work for fetch restricted packages, not nomirrored ones.
>>>
>>>--Dan
>>
>>/me nods.  That's what we'll have to do.  Unfortunately, it leaves users
>>with a worse experience than before, but until I can find a way to reach
>>the QA team and help them see my pov as the package maintainer, what
>>else can I do? :(

It's only 'worse' than before if you assume that all your users read the
documentation, otherwise they end up futzing around with their distfiles
trying to figure out why the checksum is incorrect.  This way the ebuild
will die right off and the information is presented at the same time.
They can go get the ebuild and rename it ( hell you can even provide
them with the wget command to do so ).  As such, glad this is resolved
with at least some agreement on all sides :)

--Alec Warner

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 894 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 779 bytes --]

On Sunday 26 February 2006 22:29, Robin H. Johnson wrote:
> On Fri, Feb 24, 2006 at 02:19:40PM +0000, Ciaran McCreesh wrote:
> > Side note: if the packages in question are fetch restricted, you're
> > screwed, and will not be able to add them to the tree.
>
> Actually, there is a solution for this, and it's reasonable logical.
> Don't use the same name that upstream does for the files.
>
> Simply tell the user to download X and place it in $DISTDIR renaming it
> to X-foo-bar, where's you've chosen X-foo-bar to avoid conflicts.

Is there any valid reason that we can't have portage do this 
automatically. This particular way is very user-un-friendly.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 498 bytes --]

On Mon, 27 Feb 2006 11:05:00 +0100 Paul de Vrieze <pauldv@gentoo.org>
wrote:
| Is there any valid reason that we can't have portage do this 
| automatically. This particular way is very user-un-friendly.

There's exactly one set of packages affected, and they're closed source
and non-repackagable. I doubt it's high priority...

-- 
Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

On Mon, Feb 27, 2006 at 04:34:00PM +0000, Ciaran McCreesh wrote:
> On Mon, 27 Feb 2006 11:05:00 +0100 Paul de Vrieze <pauldv@gentoo.org>
> wrote:
> | Is there any valid reason that we can't have portage do this 
> | automatically. This particular way is very user-un-friendly.
> There's exactly one set of packages affected, and they're closed source
> and non-repackagable. I doubt it's high priority...
There's more than one set of packages with this.

There is only one set in the tree that don't use a workaround of some
sort (the NX stuff).

A quick hacked up grepping indicates that the following packages use the
trick of having the user rename the file after downloading it.
dev-java/ibm-jdk-bin 
dev-java/ibm-jre-bin
dev-java/jdbc2-oracle
dev-java/jdbc3-oracle
sci-chemistry/platon
There might be others, but I'm not looking too hard at the moment.

And I know I've used it in the past when upstream has been unreliable in
naming distfiles (eg they did thank me and add the major version portion
to the filename, but not the minor version, and still changed the
download once a week).

-- 
Robin Hugh Johnson
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 241 bytes --]

Re: [gentoo-dev] SRC_URI component naming collision

[Marius Mauch]

Paul de Vrieze wrote:
> On Sunday 26 February 2006 22:29, Robin H. Johnson wrote:
> 
>>On Fri, Feb 24, 2006 at 02:19:40PM +0000, Ciaran McCreesh wrote:
>>
>>>Side note: if the packages in question are fetch restricted, you're
>>>screwed, and will not be able to add them to the tree.
>>
>>Actually, there is a solution for this, and it's reasonable logical.
>>Don't use the same name that upstream does for the files.
>>
>>Simply tell the user to download X and place it in $DISTDIR renaming it
>>to X-foo-bar, where's you've chosen X-foo-bar to avoid conflicts.
> 
> 
> Is there any valid reason that we can't have portage do this 
> automatically. This particular way is very user-un-friendly.

Check your mail archives for the old discussions about distfile name 
mangling (short version: a lot of stuff relies on distfile-name == 
basename(src_uri), also if at all this would only be a long term 
solution due to compat issues involved).

Marius
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] SRC_URI component naming collision

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]

On Tuesday 28 February 2006 15:16, Marius Mauch wrote:
> Check your mail archives for the old discussions about distfile name
> mangling (short version: a lot of stuff relies on distfile-name ==
> basename(src_uri), also if at all this would only be a long term
> solution due to compat issues involved).

For this reason a package might just specify a DIST_PREFIX. This would 
indicate a subdirectory in DISTDIR where the package is stored. When the 
package is merged, the files are copied from this directory instead of 
the DISTDIR directory. As the copy is already made, this behaviour should 
not be problematic. This does however solve 99% of the file name 
problems. The DIST_PREFIX value could be anything, with empty being the 
feature disabled, allowing for the much-wanted sharing of files between 
packages.

Even better is that this will even work with portage versions that do not 
support this feature. They will just store everything in DIST_PREFIX as 
it is now.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]