From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from lists.gentoo.org ([140.105.134.102] helo=robin.gentoo.org) by nuthatch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1GRv32-0006Ic-Po for garchives@archives.gentoo.org; Mon, 25 Sep 2006 18:19:21 +0000 Received: from robin.gentoo.org (localhost [127.0.0.1]) by robin.gentoo.org (8.13.8/8.13.6) with SMTP id k8PIISpc030196; Mon, 25 Sep 2006 18:18:28 GMT Received: from alnrmhc13.comcast.net (alnrmhc13.comcast.net [206.18.177.53]) by robin.gentoo.org (8.13.8/8.13.6) with ESMTP id k8PIGSIs014492 for ; Mon, 25 Sep 2006 18:16:29 GMT Received: from seldon (c-24-21-135-117.hsd1.or.comcast.net[24.21.135.117]) by comcast.net (alnrmhc13) with SMTP id <20060925181627b1300l9lh8e>; Mon, 25 Sep 2006 18:16:27 +0000 Date: Mon, 25 Sep 2006 11:16:25 -0700 From: Brian Harring To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] RFC about another *DEPEND variable Message-ID: <20060925181625.GA26618@seldon> References: <45126B07.6030403@gentoo.org> <200609230950.13347.vapier@gentoo.org> <20060923143020.GB12282@seldon> <200609232331.12953.vapier@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="W/nzBZO5zC0uMSeA" Content-Disposition: inline In-Reply-To: <200609232331.12953.vapier@gentoo.org> User-Agent: Mutt/1.5.13 (2006-08-11) X-Archives-Salt: 1698850e-be8e-48db-a2c7-8ca0d39a29ff X-Archives-Hash: f287a9b22efb2558c65b01fbd57c2886 --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 23, 2006 at 11:31:12PM -0400, Mike Frysinger wrote: > On Saturday 23 September 2006 10:30, Brian Harring wrote: > > dlopen? >=20 > we already said that this will need a new depend variable >=20 > > How does this fix openssl horkage? (bad soname handling) >=20 > it wont, but such things are broken regardless outside of Gentoo ... and= =20 > trying to accommodate something that happens every three blue moons at th= e=20 > cost of developer time is not worth it Bad soname handling is just *part* of what BINCOMPAT could do; it's=20 not the sole reason for it's existance, as such it's not quite right=20 dismissing it just because it addresses a rarity the NEEDED approach=20 doesn't. :) > > Also... what do we do for python/perl (*-updater scripts in general) > > where a change in a pkg state means we have to rebuild the revdeps? >=20 > my solution does not address this, but what you're proposing would over= =20 > address this ... how do you know when a package needs to be rebuilt (a pe= rl=20 > module) or rebuilding is a waste of time (a package installs perl scripts= =20 > that execute `perl` and nothing else) Original email stated that 'binding deps' would be required for that,=20 marking deps in some way such that it indicates they're sensitive to=20 changes in BINCOMPAT of the match. *IF* we actually had that in place it would enable detecting and=20 rebuilding c++ code whenever gcc pulls its next c++ abi change with=20 appropriate deps in place (iow, kill off the implicit system deps). Back to your example, if it's just a caller of it, it's not binding;=20 now if it were a perl module that sticks its modules into the perl=20 installation, yes, binding (it needs to rebuild to merge to the new=20 location). > > What you're suggesting works for strictly linkage; still think this > > shouuld work for the general problem rather then just one subset. >=20 > yes, i am addressing what i see to be the most critical issue and the eas= iest=20 > to break a user's system >=20 > > Clarifying my statement; we don't break our DEPEND down into "this is > > what is executed in building the package" (toolchain), >=20 > DEPEND >=20 > > vs "this is the=20 > > crap the binaries we build against need avail to be linked against", > > literally what winds up as -l args. >=20 > RDEPEND If that were the case, why do we have libraries listed in DEPEND then? DEPEND is (and always has been) "this is the crap I need merged to be=20 able to build an install image of myself", RDEPEND is (and always has=20 been) "this is the crap I need on the fs to actually run my=20 binaries/libs" and PDEPEND is around to cover up portages resolver,=20 but moreso the trees mostly whacked deps. This is why eradicator, solar, and you were discussing splitting link=20 depends out of DEPEND for saner CHOST/CTARGET support around a year=20 back also. Semantics at this point, but RDEPEND does *not* need to be merged to=20 build a pkg, only DEPEND; it's never been any other way. > > If punting the old lib (as I assumed), means we would potentially be > > making certain DEPEND atoms unusable if they're required in an > > execution context (rather then just winding up as a -l arg). >=20 > no ... lemme give a perfect example. >=20 > user has openssl-0.9.7j installed and they upgrade to openssl-0.9.8c ... = the=20 > old version provided SONAME files libcrypto.so.0.9.7 and libssl.so.0.9.7= =20 > while the new one provides SONAME files libcrypto.so.0.9.8 and=20 > libssl.so.0.9.8 >=20 > everything from openssl-0.9.7j is unmerged except for the two files=20 > libcrypto.so.0.9.7 and libssl.so.0.9.7. openssl-0.9.8c, being a differen= t=20 > ABI, does not provide these files thus there is no clash. >=20 > portage keeps track of libcrypto.so.0.9.7 and libssl.so.0.9.7 and once no= more=20 > packages have NEEDED entries for these, will silently clean them out >=20 > openssl is odd in that it encodes .x.y.z version into the ABI ... if we u= se=20 > the more common example with say expat, older versions install=20 > libexpat.so.0.1, libexpat.so.0.2, libexpat.so.0.3, etc... but the ABI SON= AME=20 > is still just libexpat.so.0. when the next major version of expat comes = out,=20 > the SONAME is bumped to libexpat.so.1 and portage needs to keep around th= e=20 > last libexpat.so.0 >=20 > > In that case, wouldn't mind a response to the "what about ctrl+c > > during the run?" The potential for orphaning there sucks; recording > > the old library in the new version sucks also since it complicates the > > merge process, that lib *must* be removed else it's a potential > > collision-protect minefield. >=20 > - portage merges new version 2 to $ROOT > - system now has version 1 and version 2 in $ROOT > - portage starts to clean out version 1 from $ROOT > - do not fully trim version 1's vdb until version 2 has been updated with= the=20 > new information >=20 > so ctrl+c at any point and so what ? you dont remove old files until new= =20 > files are fully placed and you can resume at any point Bleh, this is getting back to exactly my point that it's unbounded=20 resolution. To support this, every step of execution would require=20 scanning for dangling nodes to punt; aside from invalidating -p's=20 output it *still* is a collision-protect hit. It also involves changing vdb nodes from "installed and usable" to=20 "installed/usable" or "lingering", which makes things messier for 1) resolver 2) for --info 3) for has_version 4) for built_with_use 5) glsa scanning There are more, but the point is that there isn't any concept of "half=20 merged" packages for portage; introducing it gums things up pretty=20 heavily, beyond the support NEEDED requires to actually mangle the=20 half merged nodes. > > Finally, even if the lib is temporarily left behind, this solution > > doesn't gurantee the library actually would *work* still- it only can > > work if the lib is standalone from the rest of the pkg and doesn't > > rely on any external data from the pkg. >=20 > we're talking about preserving the system long enough to rebuild things; = we're=20 > not talking about keeping the system forever in a sane state. >=20 > i would guess that this corner case is not the norm and thus we can ignor= e it=20 > as the situation is still a lot better than what we have now: > $ foo > foo: error while loading shared libraries: libbar.so.1: cannot open shar= ed=20 > object file: No such file or directory >=20 > awesome So... NEEDED doesn't support bad soname handling, doesn't have a graph=20 based gurantee that the remaining bits actually would work (bits=20 above), and requires making merging *massively* slower (a NEEDED scan=20 per vdb change effectively), finally emerge -p results no longer being=20 accurate to what will be undertaken. Ways to speed it up, but frankly NEEDED just plain sucks. Its the=20 route towards openembeddeds resolution, rather then up front=20 calculation portage/rpm resolvers/apt all do. > > Basically trying to point out that what you're proposing only works in > > a subset of the cases revdep must deal with, and that revdep itself > > doesn't deal with *all* situations as is; hence BINCOMPAT as a way to > > try and shift it under maintainers control. >=20 > revdep-rebuild doesnt take into consideration any of the issues you raised >=20 > and forcing maintainers to always track BINCOMPAT is unwieldy ... a singl= e=20 > package provides multiple SONAMEs ? bitrot ? SONAME is dynamic based up= on=20 > architecture or USE flags ? Tracking BINCOMPAT should *not* be that hard. You hit a major=20 version? Bump the bincompat. Not that fricking hard really; if=20 dealing in a package that is pissy about it's so handling, well, you=20 ought to know your inducing wide spread breakage for a rename already-=20 so again, not that hard. The only case that is slightly funky is if the so doesn't change name,=20 but changes abi- BINCOMPAT at least enables notifying the manager that=20 a rebuild is needed, NEEDED based approach is incapable of it. Hell, automate a tool to determine if it's a BINCOMPAT bump via NEEDED=20 data (folks should be compiling the pkg anyways), point is it's mainly=20 common sense for maintainenance of it. The NEEDED approach doesn't really fix much frankly; from a complexity=20 standpoint, leaving it in revdep-rebuild is *far* better then trying=20 to integrate the crap you're requesting, both from an implementation=20 and from a long standing maintainence view (unbounded resolution is=20 *really* not the direction you want to go). The solution (imo) should be one that enables fixing the *class* of=20 problem without making matters worse. Yes, if the solution can be automated without flinging poo into the=20 code, I'm for it; that said I know what the implementation is going to=20 have to look like, and it's *nasty*. ~harring --W/nzBZO5zC0uMSeA Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (GNU/Linux) iD8DBQFFGBz5siLx3HvNzgcRAvvMAJ4z5Vr7JMfiQQdk7KBH6WMiCXnMGgCfXdDn 7GLx9ACt1WOXUSaLxS+Zk94= =326f -----END PGP SIGNATURE----- --W/nzBZO5zC0uMSeA-- -- gentoo-dev@gentoo.org mailing list