[gentoo-portage-dev] Portage dependencies hell and PATCH to solve it

[gentoo-portage-dev] Portage dependencies hell and PATCH to solve it

[Javier Marcet]

[-- Attachment #1.1: Type: text/plain, Size: 3958 bytes --]

Hi guys,
a long long time ago I began having problems with portage which
seemingly only affected me. I talked to some guys on #gentoo-dev or
#gentoo-portage (I'm not sure what was the channel) who did take part in
portage development, but couldn't fix it.

In the 2.0.51_pre series the last version I remember not giving me
problems was 2.0.51_pre10, with it I can emerge anything, from source
or binary and whatever DEPENDS the ebuild has.

From pre11 on I often get errors which always look like (this is with
the last one released, pre17):

# emerge mozilla -p


Invalid package name: moznomail

These are the packages that I would merge, in order:

Calculating dependencies   
!!! Problem in net-www/mozilla dependencies.
!!! 73

This time I decided to chase the problem until I found it.
One thing common to all the errors I was getting with pre17 is that it
involved negated dependencies, e.g. the above error results from this
RDEPEND in mozilla's ebuild:

RDEPEND="java? ( virtual/jre )
	crypt? ( !moznomail ( >=app-crypt/gnupg-1.2.1 ) )"

As you may see it is the negated moznomail which has, allegedly, an
invalid package name.

Another source of errors is depending on inexistent packages (this was
the good side of the bug, it found errors within gcc ebuilds). The
ebuild hardened-gcc does not exist anymore within portage yet gcc's
ebuilds include it within their list of DEPENDs. It is included negated,
but it is included:


DEPEND="virtual/libc
	!uclibc? ( >=sys-libs/glibc-2.3.3_pre20040420-r1 )
	!uclibc? ( hardened? ( >=sys-libs/glibc-2.3.3_pre20040529 ) )
	( !sys-devel/hardened-gcc )
...

The error for the above is:

Invalid package name: sys-devel/hardened-gcc

These are the packages that I would merge, in order:

Calculating dependencies   
!!! Problem in sys-devel/gcc dependencies.
!!! 73

As you can see, it is quite identical to the one with the negated
moznomail. Indeed, now that I think of it, the cause seems the same,
the !, not that hardened-gcc does not exist anymnore.
Anyhow, I think it is something needing a change.

I attach two patches.

The first one is the hunk of difference between pre10 and pre11 which made
the difference between a clean portage which gives no problems, and a
hosed one which often complain about invalid package names.

The second one is for the current _pre17. Unlike the former one which
affected bin/emerge, thus not touching ebuild.sh, pre17's bug is within
pym/portage.py, hence it makes `ebuild` also fail in some situations
(which as of now I cannot consistently reproduce, I have traced it
enough).
Hopefully it solves each and every problem. So far I've been able to
do and 'emerge -u world --deep --empty-tree -p' without any error, which
I couldn't before due to packages like mozilla, mozilla-firefox or gcc,
to name a few.
Oddly enough, the change I propose for _pre17 has been on purpose
judging by the comments the culprit line is surrounded. I read the bug
description and that's not the problem I've always had at all.

What I don't understand, but puzzles me quite a bit, is how is it possible
that this only shows on my systems?
This has been present since 2.0.49_something, with a lot of portage
released versions making havoc. I tried fresh installs, without moving
portage's dir to customs paths or anything like that and even so the bug
hit me, on different machines. All of them set up by me, though.
It seems it's me rather than the system having the problems.

Suggestions, comments, explanations for this behavior will be highly
appreciated.

Thanks for your help & time.

P.S I'm sorry for not narrowing down the first patch a little bit more,
but I haven't yet learned Python, maybe I do in the coming months, but
ATM I'm happy enough hacking with C, C++ & Perl all day long :)


-- 
Captain's Log, star date 21:34.5...
Ö\x01
Javier Marcet <javier@marcet.info>

[-- Attachment #1.2: portage-2.0.51_pre11-dependencies_hell.fix --]
[-- Type: text/plain, Size: 753 bytes --]

--- bin/emerge	2004-08-15 03:19:17.000000000 +0200
+++ bin.orig/emerge	2004-08-15 03:19:08.000000000 +0200
@@ -829,18 +823,9 @@
 			merging=2
 		if merging==1:
 			mybigkey.append("merge")
-			self.mydbapi[myroot].cpv_inject(mykey)
 		else:
 			mybigkey.append("nomerge")
 			
-		virts = portage.db[myroot]["porttree"].dbapi.aux_get(mykey, ["PROVIDE"])[0].split()
-		cp = portage.dep_getkey(mykey)
-		for virt in virts:
-			virt = portage.dep_getkey(virt)
-			while cp in portage.virts[virt]:
-				portage.virts[virt].remove(cp)
-			portage.virts[virt].insert(0, cp)
-		
 		# whatever the case, we need to add the node to our digraph so
 		# that children can depend upon it.
 		self.digraph.addnode(string.join(mybigkey),myparent)

[-- Attachment #1.3: portage-2.0.51_pre17-dependencies_hell.fix --]
[-- Type: text/plain, Size: 726 bytes --]

--- pym/portage.py.orig	2004-08-05 13:24:41.000000000 +0200
+++ pym/portage.py	2004-08-15 07:04:55.954353680 +0200
@@ -3932,7 +3932,7 @@
 				if mydep!=None:
 					tmp=(len(mydep)>=1)
 					if deplist[mypos][0]=="!":
-						#tmp=not tmp
+						tmp=not tmp
 						# This is ad-hoc code. We should rewrite this later.. (See #52377)
 						# The reason is that portage uses fakedb when --update option now.
 						# So portage considers that a block package doesn't exist even if it exists.
@@ -3940,7 +3940,7 @@
 						# ==== start
 						# emerge checks if it's block or not, so we can always set tmp=False.
 						# but it's not clean..
-						tmp=False
+						#tmp=False
 						# ==== end
 					deplist[mypos]=tmp
 				else:

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-portage-dev] Portage dependencies hell and PATCH to solve it

[Jason Stubbs]

Apologies if this was received already

On Sunday 15 August 2004 14:59, Javier Marcet wrote:
> From pre11 on I often get errors which always look like (this is with
> the last one released, pre17):
>
> # emerge mozilla -p
>
>
> Invalid package name: moznomail
>
> These are the packages that I would merge, in order:
>
> Calculating dependencies
> !!! Problem in net-www/mozilla dependencies.
> !!! 73

http://bugs.gentoo.org/show_bug.cgi?id=57212

The _pre11 patch you provided will cause lots of problems with regards to 
virtuals. For example, if the default virtual/x11 is xfree and the user tries 
to emerge xorg-x11, portage will try to install both xfree and xorg-x11 
without those lines.

The _pre17 patch I'm not to sure about, but it is definately unrelated to the 
problem you are having - even if enabling the code does appear to fix it.

Anyway, it is fixed now and will be in portage-2.0.51_pre18.

Regards,
Jason Stubbs

--
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] Re: Portage dependencies hell and PATCH to solve it

[Javier Marcet]

[-- Attachment #1: Type: text/plain, Size: 2726 bytes --]

>> From pre11 on I often get errors which always look like (this is with
>> the last one released, pre17):
>>
>> # emerge mozilla -p
>>
>>
>> Invalid package name: moznomail
>>
>> These are the packages that I would merge, in order:
>>
>> Calculating dependencies
>> !!! Problem in net-www/mozilla dependencies.
>> !!! 73

>http://bugs.gentoo.org/show_bug.cgi?id=57212

Oh ****, this time you were already into it. As I said, I had similar
problems with some 2.0.49 versions already, and back them I didn't find
anybody else facing them.
I should have checked bugzilla anyway, sorry.

>The _pre11 patch you provided will cause lots of problems with regards to 
>virtuals. For example, if the default virtual/x11 is xfree and the user tries 
>to emerge xorg-x11, portage will try to install both xfree and xorg-x11 
>without those lines.

As I explained, it was not itself a patch, but only a hunk of
difference between pre10 and pre11 which made pre10 fail when applied.
I have no idea of Python, hence I thought the quickest way to find a
workaround --which could also be a hint for where to look for a proper
bug fix-- was going over every difference between both portage releases.

>The _pre17 patch I'm not to sure about, but it is definately unrelated to the 
>problem you are having - even if enabling the code does appear to fix it.

The procedure I followed to get to it was the same as for the pre11.
Anyhow, I began to see that it was not a proper fix when I started to
read pym/portage.py and bin/emerge more closely, and also after I saw
packages and virtuals blocking was not working at all.

>Anyway, it is fixed now and will be in portage-2.0.51_pre18.

Thanks a lot, I've tried the patch you provide on bug #57212 and it
definitely solves all my problems and without any side effect. Virtuals'
blocking, packages blocking within normal ebuilds, ... everything works
fine.
I'll be trying to see if I get again the 'invalid package name' when
doing an 'ebuild xxxxxxx.ebuild qmerge' after it has been compiled &
installed properly. Since your patch only targets bin/emerge, the ebuild
problem will remain. It is much less common to hit a package which shows
this behavior, though, at least with pre17, for other older versions it
was much more common.

BTW, do you think I should file a bug report about the hardened-gcc not
existing anymore but being present on current gcc ebuilds?

Thanks again :) This indeed was faaast, much faster than my tonight
chasing session (in a language unknown to me).


-- 
"Our nation must come together to unite."

George W. Bush
June 4, 2001
Remark made in Tampa, Florida.

Javier Marcet <javier@marcet.info>

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-portage-dev] =* support in package.{keywords,mask,unmask}

[Andy Lutomirski]

[-- Attachment #1: Type: text/plain, Size: 206 bytes --]

I like to use

=*gnome-base/ ~amd64

in my portage.keywords.  It looks like this is supposed to work, but it 
doesn't.  Here's a short patch to fix it.  Is this the right place to 
submit it?

Thanks,
Andy

[-- Attachment #2: portage.patch --]
[-- Type: text/plain, Size: 311 bytes --]

--- ./portage.py.orig	2004-08-08 10:46:45.009821264 -0700
+++ ./portage.py	2004-08-08 11:05:33.467269656 -0700
@@ -3541,7 +3541,11 @@
 		if not operator:
 			return []
 	else:
-		operator = None
+		if mydep[0:2] == '=*':
+			operator = '=*'
+			mycpv = mydep[2:]
+		else:
+			operator = None
 
 	mylist = []
 


[-- Attachment #3: Type: text/plain, Size: 45 bytes --]

--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] =* support in package.{keywords,mask,unmask}

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 648 bytes --]

On 08/08/04  Andy Lutomirski wrote:

> I like to use
> 
> =*gnome-base/ ~amd64
> 
> in my portage.keywords.  It looks like this is supposed to work, but
> it doesn't.  Here's a short patch to fix it.  Is this the right place
> to submit it?

No, please post bugs and patches on bugs.gentoo.org (under "Portage
Development"). Also I've no clue what that syntax is supposed to mean,
so please include a detailed description for it on your bugreport.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] =* support in package.{keywords,mask,unmask}

[Andy Lutomirski]

Done.  It's bug #59893.

--Andy

Marius Mauch wrote:

> On 08/08/04  Andy Lutomirski wrote:
> 
> 
>>I like to use
>>
>>=*gnome-base/ ~amd64
>>
>>in my portage.keywords.  It looks like this is supposed to work, but
>>it doesn't.  Here's a short patch to fix it.  Is this the right place
>>to submit it?
> 
> 
> No, please post bugs and patches on bugs.gentoo.org (under "Portage
> Development"). Also I've no clue what that syntax is supposed to mean,
> so please include a detailed description for it on your bugreport.
> 
> Marius
> 

--
gentoo-portage-dev@gentoo.org mailing list