* [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
@ 2006-11-03 7:29 Mike Frysinger
2006-11-03 8:23 ` Brian Harring
` (3 more replies)
0 siblings, 4 replies; 11+ messages in thread
From: Mike Frysinger @ 2006-11-03 7:29 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 1471 bytes --]
this is not a "implicit vs explicit" thread; if you want that discussion start
your own
we've said the relationship of DEPEND atoms in ebuilds should be independent
of the DEPEND atoms found in eclasses as logically ebuilds should not care
what it takes for eclasses to work and vice versa ... for the most part, this
is true ...
however, semi-recently, a change was made such that the implicit
RDEPEND=$DEPEND was dropped from ebuilds if an inherited class set RDEPEND in
any way ... that means if you have an ebuild at the moment that does:
DEPEND="foo"
and you dont inherit any eclasses, then you also get for free:
RDEPEND="foo"
if you decide to inherit eclasses though, you had better do some research as
any eclass that does even RDEPEND="" will change that behavior ... or if you
are an eclass writer and you decided to add RDEPEND to your eclass, you had
better do a reverse check and make sure that any ebuild that inherits your
eclass (directly or indirectly) does not utilize implicit RDEPEND behavior as
you would have just broken it ... awesome ;)
i posted a patch to fix this regression, but since it took so long before
anyone noticed, zmedico wants to see if anyone is opposed (i dont know why
they would be, but i cant think of everything)
so to recap, the fix here changes it back to the historically documented
behavior that the implicit RDEPEND happens in ebuilds regardless of what
eclasses do
-mike
[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
2006-11-03 7:29 [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior Mike Frysinger
@ 2006-11-03 8:23 ` Brian Harring
2006-11-03 13:02 ` Mike Frysinger
2006-11-03 8:43 ` Zac Medico
` (2 subsequent siblings)
3 siblings, 1 reply; 11+ messages in thread
From: Brian Harring @ 2006-11-03 8:23 UTC (permalink / raw
To: gentoo-dev; +Cc: swegener, dev-portage
[-- Attachment #1: Type: text/plain, Size: 5525 bytes --]
On Fri, Nov 03, 2006 at 02:29:45AM -0500, Mike Frysinger wrote:
> this is not a "implicit vs explicit" thread; if you want that discussion start
> your own
That discussion (dropping it to explicit, as has been discussed often
enough) should be started off again since fixing it isn't exactly a
matter of applying a patch and ignoring it.... so...
swegener? Feel free to chime in, you were one of the main folks
after killing implicit from what I recall.
Meanwhile...
> we've said the relationship of DEPEND atoms in ebuilds should be independent
> of the DEPEND atoms found in eclasses as logically ebuilds should not care
> what it takes for eclasses to work and vice versa ... for the most part, this
> is true ...
>
> however, semi-recently, a change was made such that the implicit
> RDEPEND=$DEPEND was dropped from ebuilds if an inherited class set RDEPEND in
> any way ... that means if you have an ebuild at the moment that does:
> DEPEND="foo"
> and you dont inherit any eclasses, then you also get for free:
> RDEPEND="foo"
>
> if you decide to inherit eclasses though, you had better do some research as
> any eclass that does even RDEPEND="" will change that behavior ... or if you
> are an eclass writer and you decided to add RDEPEND to your eclass, you had
> better do a reverse check and make sure that any ebuild that inherits your
> eclass (directly or indirectly) does not utilize implicit RDEPEND behavior as
> you would have just broken it ... awesome ;)
>
> i posted a patch to fix this regression, but since it took so long before
> anyone noticed, zmedico wants to see if anyone is opposed (i dont know why
> they would be, but i cant think of everything)
>
> so to recap, the fix here changes it back to the historically documented
> behavior that the implicit RDEPEND happens in ebuilds regardless of what
> eclasses do
Mildly screwed on this one from a cache standpoint;
1) the change went out unversioned, meaning that while it was
spreading, folks were getting a mix of it. Likely their are still
cache entries distributed with gentoo-x86 that are still old style
deps.
Nice.
2) Same issue via reverting it; it's an unversioned change, so portage
will *not* pick up on it and fix it. Additionally, cause I was being
anal about making --metadata as fast as possible, portage will *not*
pick up the change since mtimes won't differ, thus will not write the
new entry into the localized cache (cache.util.py, note the
is_eclass_data_valid check controlling write_it boolean).
#1 was stupid, #2 pretty royally gets us up the backside.
So... my suggestion. Decide what you're going to do long term now, no
more (bluntly) fucking around with this.
Got a few courses of action from where I'm sitting-
1) EAPI bump for the restored behaviour. This sucks since it forces
the entire tree to be bumped to force falling back to older behaviour
(this is why changes like this are supposed to be EAPI bumped in the
first place btw).
2) (several steps)
2.a) Revbump all afflicted portage versions with a patch
restoring original behaviour, leaving keywords the same (meaning
2.1-r3 goes in with stable keywords); pull all portage versions that
have the broken behaviour (>=2.1 basically).
2.b) Force mtime touches of _all_ eclasses to force both rsync
transfer within the mirror tier (can do this other ways), but more
importantly, to force the transfer of all eclass consumers to the
localized cache on folks machines. This at least gets them the old
style deps in their cache. It will *not* fix the rdeps for overlay
ebuilds that are afflicted, and does not fix it if the user triggers
regeneration locally.
3) ignore it, and laugh like nero till everyone has upgraded to a
fixed version of portage and enough churn in the tree has occured to
have forced the corrected cache entries locally.
#1 blows since either it requires changing EAPI in every ebuild (
easier, base/profile.bashrc, although I never intended profile bashrc
to influence ebuild metadata), or auditing the rdeps and fixing them,
bumping when after older behaviour. Additionally, requires leaving a
portage version behind at the older EAPI for upgrading.
#2 is fairly brutal initially, but should clear out the corruption
pretty quickly for majority of users; for overlay users and devs
however, it relies on folks upgrading to a fixed portage fairly
quickly so that rdeps are proper. Doubly so for devs, since their
portage installation is the first line of dependency verification
(folks running automated repoman/pcheck dependency scans being second
line).
One upshot though, this method will push the proper deps into the
localized cache for stage* users.
Downside is that it triggers a pretty heavy hit on rsync mirrors,
although the eclass touching can be stretched out over a few days to
incrementally push out the updated cache.
#3 means that broken rdeps linger for the next few months till folks
have upgraded, and the mess has gradually cleaned itself out. Will
linger for a long while in older ebuilds, although a forced cache
regen fixes it for folks who either wipe their localized cache, or
somehow trigger a regen.
So... I'd go with #2.
Pardon the essay, but getting the effects of it fixed isn't trivial,
and the long term lingering effects aren't either imo.
So... thoughts?
~harring
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
2006-11-03 7:29 [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior Mike Frysinger
2006-11-03 8:23 ` Brian Harring
@ 2006-11-03 8:43 ` Zac Medico
2006-11-03 9:32 ` Peter Volkov (pva)
2006-11-05 3:45 ` Zac Medico
2006-11-03 18:37 ` Diego 'Flameeyes' Pettenò
2006-11-03 18:45 ` Stephen Bennett
3 siblings, 2 replies; 11+ messages in thread
From: Zac Medico @ 2006-11-03 8:43 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 2799 bytes --]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Mike Frysinger wrote:
> this is not a "implicit vs explicit" thread; if you want that discussion start
> your own
>
> we've said the relationship of DEPEND atoms in ebuilds should be independent
> of the DEPEND atoms found in eclasses as logically ebuilds should not care
> what it takes for eclasses to work and vice versa ... for the most part, this
> is true ...
>
> however, semi-recently, a change was made such that the implicit
> RDEPEND=$DEPEND was dropped from ebuilds if an inherited class set RDEPEND in
> any way ... that means if you have an ebuild at the moment that does:
> DEPEND="foo"
> and you dont inherit any eclasses, then you also get for free:
> RDEPEND="foo"
The change occurred sometime between portage-2.0.51 and portage-2.0.52, and the
first stable version to have it was portage-2.0.53 (released in December of
2005). People didn't really start to notice the difference until after
portage-2.1 had been deployed on the master rsync mirror last June (it had been
running portage-2.0.51.22-r3 up until then).
> if you decide to inherit eclasses though, you had better do some research as
> any eclass that does even RDEPEND="" will change that behavior ... or if you
> are an eclass writer and you decided to add RDEPEND to your eclass, you had
> better do a reverse check and make sure that any ebuild that inherits your
> eclass (directly or indirectly) does not utilize implicit RDEPEND behavior as
> you would have just broken it ... awesome ;)
>
> i posted a patch to fix this regression, but since it took so long before
> anyone noticed, zmedico wants to see if anyone is opposed (i dont know why
> they would be, but i cant think of everything)
>
> so to recap, the fix here changes it back to the historically documented
> behavior that the implicit RDEPEND happens in ebuilds regardless of what
> eclasses do
> -mike
If we do this then I want to make sure everybody is prepared for the
consequences. Basically, it restricts implicit RDEPEND to the ebuild level,
making it independent of whatever RDEPEND may or may not be defined in the
inherited eclasses. That means that some ebuilds will get implicit RDEPEND
that they don't get currently. Also, some ebuilds will loose some implicit
RDEPEND that they current get from eclasses.
I've attached the patch which reverts the behavior. If we do this, I think that
we should do it in one big sweep, via a sys-apps/portage revbump and a
simultaneous application of the patch on the master rsync mirror (see Brian
Harring's email for more details).
Zac
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
iD8DBQFFSwFI/ejvha5XGaMRArjaAKDi8B++F71vt3D2QkG5vyRhZ0yWvgCghBVA
ZrVSROlN0E7X/xdkFbJ6NXo=
=yRsF
-----END PGP SIGNATURE-----
[-- Attachment #2: revert_to_2.0.51_rdepend.patch --]
[-- Type: text/plain, Size: 510 bytes --]
Index: bin/ebuild.sh
===================================================================
--- bin/ebuild.sh (revision 4913)
+++ bin/ebuild.sh (working copy)
@@ -1504,8 +1504,8 @@
#syntax from getting expanded :)
#check eclass rdepends also.
set -f
-if [ "${RDEPEND-unset}" == "unset" ] && [ "${E_RDEPEND-unset}" == "unset" ] ; then
- export RDEPEND="${DEPEND} ${E_DEPEND}"
+if [ "${RDEPEND-unset}" == "unset" ]; then
+ export RDEPEND=${DEPEND}
debug-print "RDEPEND: not set... Setting to: ${DEPEND}"
fi
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
2006-11-03 8:43 ` Zac Medico
@ 2006-11-03 9:32 ` Peter Volkov (pva)
2006-11-03 9:39 ` Zac Medico
2006-11-03 13:02 ` Mike Frysinger
2006-11-05 3:45 ` Zac Medico
1 sibling, 2 replies; 11+ messages in thread
From: Peter Volkov (pva) @ 2006-11-03 9:32 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 252 bytes --]
On 2006-11-03 at 00:43 -0800, Zac Medico wrote:
> Also, some ebuilds will loose some implicit RDEPEND that they current
> get from eclasses.
Why? I suppose more logical solution is to adjoin DEPEND from ebuild and
RDEPEND from eclass.
Peter.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
2006-11-03 9:32 ` Peter Volkov (pva)
@ 2006-11-03 9:39 ` Zac Medico
2006-11-03 13:02 ` Mike Frysinger
1 sibling, 0 replies; 11+ messages in thread
From: Zac Medico @ 2006-11-03 9:39 UTC (permalink / raw
To: gentoo-dev
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Peter Volkov (pva) wrote:
> On 2006-11-03 at 00:43 -0800, Zac Medico wrote:
>> Also, some ebuilds will loose some implicit RDEPEND that they current
>> get from eclasses.
>
> Why? I suppose more logical solution is to adjoin DEPEND from ebuild and
> RDEPEND from eclass.
>
> Peter.
You've misunderstood the meaning of "implicit RDEPEND" in my statement above (I
don't blame you, implicit RDEPEND can be a confusing topic). When I say
"implicit RDEPEND", I am talking about DEPEND that has been implicitly converted
to RDEPEND. Some ebuilds may currently have some implicit RDEPEND that
originated as DEPEND in an eclass. If we use the patch to revert that behavior,
those specific implicit RDEPEND atoms will go away. I hope this makes sense. :)
Zac
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
iD8DBQFFSw45/ejvha5XGaMRAntfAJ0X0K9U+CtyB4nhq73v8p5EBd5w8ACg8nc4
jN+Q4rWo+tfvoVL1YUY01E8=
=X/2/
-----END PGP SIGNATURE-----
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
2006-11-03 9:32 ` Peter Volkov (pva)
2006-11-03 9:39 ` Zac Medico
@ 2006-11-03 13:02 ` Mike Frysinger
1 sibling, 0 replies; 11+ messages in thread
From: Mike Frysinger @ 2006-11-03 13:02 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 353 bytes --]
On Friday 03 November 2006 04:32, Peter Volkov (pva) wrote:
> On 2006-11-03 at 00:43 -0800, Zac Medico wrote:
> > Also, some ebuilds will loose some implicit RDEPEND that they current
> > get from eclasses.
>
> I suppose more logical solution is to adjoin DEPEND from ebuild and
> RDEPEND from eclass.
that is the behavior that we'd be moving to
-mike
[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
2006-11-03 8:23 ` Brian Harring
@ 2006-11-03 13:02 ` Mike Frysinger
0 siblings, 0 replies; 11+ messages in thread
From: Mike Frysinger @ 2006-11-03 13:02 UTC (permalink / raw
To: Brian Harring; +Cc: gentoo-dev, swegener, dev-portage
[-- Attachment #1: Type: text/plain, Size: 125 bytes --]
On Friday 03 November 2006 03:23, Brian Harring wrote:
> so...
so... start a new thread exactly like i told you so :P
-mike
[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
2006-11-03 7:29 [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior Mike Frysinger
2006-11-03 8:23 ` Brian Harring
2006-11-03 8:43 ` Zac Medico
@ 2006-11-03 18:37 ` Diego 'Flameeyes' Pettenò
2006-11-03 18:45 ` Stephen Bennett
3 siblings, 0 replies; 11+ messages in thread
From: Diego 'Flameeyes' Pettenò @ 2006-11-03 18:37 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 631 bytes --]
On Friday 03 November 2006 08:29, Mike Frysinger wrote:
> so to recap, the fix here changes it back to the historically documented
> behavior that the implicit RDEPEND happens in ebuilds regardless of what
> eclasses do
Fine by me, that would probably solve quite a bit of problems (and although I
tried, I can't think of a way how restoring this will break stuff.. the worse
it can do is making pointless some extra DEPEND="${RDEPEND}" or vice-versa in
ebuilds).
--
Diego "Flameeyes" Pettenò - http://farragut.flameeyes.is-a-geek.org/
Gentoo/Alt lead, Gentoo/FreeBSD, Video, Sound, ALSA, PAM, KDE, CJK, Ruby ...
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
2006-11-03 7:29 [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior Mike Frysinger
` (2 preceding siblings ...)
2006-11-03 18:37 ` Diego 'Flameeyes' Pettenò
@ 2006-11-03 18:45 ` Stephen Bennett
3 siblings, 0 replies; 11+ messages in thread
From: Stephen Bennett @ 2006-11-03 18:45 UTC (permalink / raw
To: gentoo-dev
On Fri, 3 Nov 2006 02:29:45 -0500
Mike Frysinger <vapier@gentoo.org> wrote:
> so to recap, the fix here changes it back to the historically
> documented behavior that the implicit RDEPEND happens in ebuilds
> regardless of what eclasses do
Do it please. The current behaviour is retarded however you look at it.
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
2006-11-03 8:43 ` Zac Medico
2006-11-03 9:32 ` Peter Volkov (pva)
@ 2006-11-05 3:45 ` Zac Medico
2006-11-05 7:40 ` Mike Frysinger
1 sibling, 1 reply; 11+ messages in thread
From: Zac Medico @ 2006-11-05 3:45 UTC (permalink / raw
To: gentoo-dev
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Zac Medico wrote:
> If we do this then I want to make sure everybody is prepared for the
> consequences. Basically, it restricts implicit RDEPEND to the ebuild level,
> making it independent of whatever RDEPEND may or may not be defined in the
> inherited eclasses. That means that some ebuilds will get implicit RDEPEND
> that they don't get currently. Also, some ebuilds will loose some implicit
> RDEPEND that they current get from eclasses.
I've generated some data to show what the differences in RDEPEND will be after the
patch has been applied. The data includes a list of affected packages [1], and two
separate files (tab separated values) for those packages that will have an implicit
RDEPEND decrease [2] and those that will have an implicit RDEPEND increase [3]. The
format is:
${CATEGORY}/${PF} <tab> RDEPEND before patch <tab> RDEPEND after patch
Fortunately, I don't anticipate that this change will cause significant disruption.
My plan is to release a sys-apps/portage-2.1.1-r2 revbump masked by package.mask and
have all of the arch teams keyword it. The ebuild will have an ewarn message in
pkg_postinst() that instructs gentoo-x86 cvs users to clean out their existing cache
and regenerate it from scratch.
After all of the teams have keyworded portage-2.1.1-r2, I'll unmask it and apply the
patch on the master rsync mirror. The only extra load on the rsync mirrors will be
for the cache entries of approximately 2700 packages [1]. The combined total size of
these cache entries will be approximately 1.5 megabytes. For comparison, cache entry
updates average approximately 425 per day, but in those cases there is additional
overhead for changes in ebuilds, eclasses, and auxiliary files. For the ~2700 cache
entries that need to be regenerated after application of the patch, only the cache
entries alone will incur load on the rsync mirrors. The new version of portage will
include a patch that will cause it to automatically recognize the changed RDEPEND
after a sync (unlike earlier versions of portage that relied on mtime alone).
Anyone concerned about the impact of this patch on any packages that they maintain
should check the list of packages [1] to see if they are affected. Please do that
and let me know if you anticipate any problems.
Thanks,
Zac
[1] http://dev.gentoo.org/~zmedico/bug_153591/data/20061104/package_names.txt.bz2
[2] http://dev.gentoo.org/~zmedico/bug_153591/data/20061104/rdepend_decrease.csv.bz2
[3] http://dev.gentoo.org/~zmedico/bug_153591/data/20061104/rdepend_increase.csv.bz2
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
iD8DBQFFTV5Z/ejvha5XGaMRApbiAKCWDEU1gjK3NaH0xQmTt7CT8ITHsACgk56y
JdiNGWgX5BpX5cZ+sgxkEfo=
=g2g4
-----END PGP SIGNATURE-----
--
gentoo-dev@gentoo.org mailing list
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior
2006-11-05 3:45 ` Zac Medico
@ 2006-11-05 7:40 ` Mike Frysinger
0 siblings, 0 replies; 11+ messages in thread
From: Mike Frysinger @ 2006-11-05 7:40 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 639 bytes --]
On Saturday 04 November 2006 22:45, Zac Medico wrote:
> Fortunately, I don't anticipate that this change will cause significant
> disruption. My plan is to release a sys-apps/portage-2.1.1-r2 revbump
> masked by package.mask and have all of the arch teams keyword it. The
> ebuild will have an ewarn message in pkg_postinst() that instructs
> gentoo-x86 cvs users to clean out their existing cache and regenerate it
> from scratch.
thanks
> Anyone concerned about the impact of this patch on any packages that they
> maintain should check the list of packages [1] to see if they are affected.
i only see fixes for my packages ;)
-mike
[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2006-11-05 7:44 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-11-03 7:29 [gentoo-dev] [RFC] fixing up portage implicit RDEPEND behavior Mike Frysinger
2006-11-03 8:23 ` Brian Harring
2006-11-03 13:02 ` Mike Frysinger
2006-11-03 8:43 ` Zac Medico
2006-11-03 9:32 ` Peter Volkov (pva)
2006-11-03 9:39 ` Zac Medico
2006-11-03 13:02 ` Mike Frysinger
2006-11-05 3:45 ` Zac Medico
2006-11-05 7:40 ` Mike Frysinger
2006-11-03 18:37 ` Diego 'Flameeyes' Pettenò
2006-11-03 18:45 ` Stephen Bennett
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox