[gentoo-portage-dev] New preserve-libs feature

[gentoo-portage-dev] New preserve-libs feature

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 1444 bytes --]

If you haven't noticed, I just added a new 'preserve-libs' feature for
bug 62207 that moves shared object that are still used but would be
removed on an update into the new package instance (so on a update from
expat-1 to expat-2 the user would still have libexpat.so.0, owned by
expat-2). The actual match criteria are a bit stricter, but you get the
idea. I think this is an long overdue bugfix, but given past
discussions not everyone has the same opinion, though the last
discussion about it ended without a conclusion (at least none that I
could see).
So everyone who has valid objections to the _general idea_ of this
implementation (preserving old libraries to avoid some runtime linker
errors) speak up now. 

Just keep the following things in mind:
- I'm not claiming that it's a silver bullet to all possible runtime
linker problems, it's supposed to cover some of the common cases (like
the expat problem)
- I'm not claiming that the implementation is perfect yet
- This feature is currently optional, so I'm not forcing it down on
anyone (in fact it's completely hidden for now). Of course if people
start using it ebuild devs will have to deal with it sooner or later,
but lets discuss it here first.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] New preserve-libs feature

[Simon Stelling]

Marius Mauch wrote:
> So everyone who has valid objections to the _general idea_ of this
> implementation (preserving old libraries to avoid some runtime linker
> errors) speak up now. 

For how long are these libraries preserved? This might have a security
impact in cases like the recent openssl-case where you had to upgrade to
an incompatible ABI because the version using the old one was
vulnerable. Using preserve-libs it would leave the old lib around,
making it possible for programs to link against the wrong version and
ending up being vulnerable. I realize that the feature is meant to help
the transitional phase until all apps are built against the new ABI, but
how would you find these vulnerable apps currently? revdep-rebuild
wouldn't rebuild them since they are still functional.

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 developer
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] New preserve-libs feature

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

On Saturday 17 February 2007, Simon Stelling wrote:
> Using preserve-libs it would leave the old lib around,
> making it possible for programs to link against the wrong version and
> ending up being vulnerable.

generally, this is incorrect

the only way you could link against it is if you were to actually specify the 
full path to the library:
... /usr/lib/libfoo.so.3 ...

and since that's invalid usage, there is no real security impact
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-portage-dev] New preserve-libs feature

[Marius Mauch]

[-- Attachment #1: Type: text/plain, Size: 1525 bytes --]

On Sat, 17 Feb 2007 14:55:26 +0100
Simon Stelling <blubb@gentoo.org> wrote:

> Marius Mauch wrote:
> > So everyone who has valid objections to the _general idea_ of this
> > implementation (preserving old libraries to avoid some runtime
> > linker errors) speak up now. 
> 
> For how long are these libraries preserved? This might have a security
> impact in cases like the recent openssl-case where you had to upgrade
> to an incompatible ABI because the version using the old one was
> vulnerable. Using preserve-libs it would leave the old lib around,
> making it possible for programs to link against the wrong version and
> ending up being vulnerable. I realize that the feature is meant to
> help the transitional phase until all apps are built against the new
> ABI, but how would you find these vulnerable apps currently?
> revdep-rebuild wouldn't rebuild them since they are still functional.

Currently they are around as long as they are referenced by other
packages or until the package is unmerged. And yes, there should be a
way to tell revdep-rebuild/the user which packages should/need to be
rebuilt, but I haven't made my mind up yet on how to accomplish that
(in fact atm there is no separation between "native" and "imported"
libs in vdb, I'm aware that needs to be added).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] New preserve-libs feature

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]

On Sat, Feb 17, 2007 at 09:03:24AM -0500, Mike Frysinger wrote:
> On Saturday 17 February 2007, Simon Stelling wrote:
> > Using preserve-libs it would leave the old lib around,
> > making it possible for programs to link against the wrong version and
> > ending up being vulnerable.
> 
> generally, this is incorrect
> 
> the only way you could link against it is if you were to actually specify the 
> full path to the library:
> ... /usr/lib/libfoo.so.3 ...
> 
> and since that's invalid usage, there is no real security impact

Security impact is from a pkg potentially dragging along old libs; if 
you've got a stable pkg that gets an update once every blue moon, it 
can hold onto the lib for a *long* time while still using the lib; 
thus if a vuln. in the lib, said pkg still is screwed.

Other angle is someone intentionally forcing usage of a known bad 
library that is still dangling.  Corner case, but doable.

Bit curious how this is going to behave if via linked in libs, new loc 
and old get loaded alongside...

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] New preserve-libs feature

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1296 bytes --]

On Saturday 17 February 2007, Brian Harring wrote:
> Security impact is from a pkg potentially dragging along old libs; if
> you've got a stable pkg that gets an update once every blue moon, it
> can hold onto the lib for a *long* time while still using the lib;
> thus if a vuln. in the lib, said pkg still is screwed.

umm, no ... the package itself is updated against the new copy while the old 
copy exists for other packages that have already been built

> Other angle is someone intentionally forcing usage of a known bad
> library that is still dangling.  Corner case, but doable.

as i said, this is the "invalid" syntax:
... /usr/lib/libfoo.so.3 ...

besides, this is not a real concern ... if a user is purposefully relinking 
against files because it has a security issue, they have the ability to do a 
lot more than any bug exposed in the library

> Bit curious how this is going to behave if via linked in libs, new loc
> and old get loaded alongside...

this would require multiple libraries to be involved in the equation and the 
answer is undefined behavior which most certainly result in runtime 
failures ...

besides, just like the gcc-3.3 -> gcc-3.4 transition, if you dont run 
revdep-rebuild and things are breaking, it's your own fault
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-portage-dev] New preserve-libs feature

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1685 bytes --]

On Sat, Feb 17, 2007 at 09:39:58AM -0500, Mike Frysinger wrote:
> On Saturday 17 February 2007, Brian Harring wrote:
> > Security impact is from a pkg potentially dragging along old libs; if
> > you've got a stable pkg that gets an update once every blue moon, it
> > can hold onto the lib for a *long* time while still using the lib;
> > thus if a vuln. in the lib, said pkg still is screwed.
> 
> umm, no ... the package itself is updated against the new copy while the old 
> copy exists for other packages that have already been built

Suspect you're ignoring soname changes, which is about all this patch 
would address- for example, ssl's old form of breakage.  In that case, 
*yes* the package gets updated, anything recompiled should get the 
correct lib (assuming the code knows the appropriate linker args), but 
the old vuln. lib still will hang around as long as anything refs it.


> > Other angle is someone intentionally forcing usage of a known bad
> > library that is still dangling.  Corner case, but doable.
> 
> as i said, this is the "invalid" syntax:
> ... /usr/lib/libfoo.so.3 ...

Not to LD_PRELOAD :)

Haven't tried anything crazy, but suspect it can be abused to override 
to the old.

> > Bit curious how this is going to behave if via linked in libs, new loc
> > and old get loaded alongside...
> 
> this would require multiple libraries to be involved in the equation and the 
> answer is undefined behavior which most certainly result in runtime 
> failures ...

Point there was that instead of just bailing with "lib is missing", 
suspect it'll manage to run, then segfault at potentially crappy 
times.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] New preserve-libs feature

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 2235 bytes --]

On Saturday 17 February 2007, Brian Harring wrote:
> On Sat, Feb 17, 2007 at 09:39:58AM -0500, Mike Frysinger wrote:
> > On Saturday 17 February 2007, Brian Harring wrote:
> > > Security impact is from a pkg potentially dragging along old libs; if
> > > you've got a stable pkg that gets an update once every blue moon, it
> > > can hold onto the lib for a *long* time while still using the lib;
> > > thus if a vuln. in the lib, said pkg still is screwed.
> >
> > umm, no ... the package itself is updated against the new copy while the
> > old copy exists for other packages that have already been built
>
> Suspect you're ignoring soname changes, which is about all this patch
> would address- for example, ssl's old form of breakage.  In that case,
> *yes* the package gets updated, anything recompiled should get the
> correct lib

i'm not ignoring soname changes, those are exactly what i'm talking about

> (assuming the code knows the appropriate linker args)

there is no such thing ... it's always "-lfoo"

> the old vuln. lib still will hang around as long as anything refs it.

of course and this is the desired behavior ... people need to run 
revdep-rebuild, there's no two ways about it

> > > Other angle is someone intentionally forcing usage of a known bad
> > > library that is still dangling.  Corner case, but doable.
> >
> > as i said, this is the "invalid" syntax:
> > ... /usr/lib/libfoo.so.3 ...
>
> Not to LD_PRELOAD :)
>
> Haven't tried anything crazy, but suspect it can be abused to override
> to the old.

again, not in any scenario that actually matters ... so this too is a 
pointless line of thought to pursue as it has no real security impact

> > > Bit curious how this is going to behave if via linked in libs, new loc
> > > and old get loaded alongside...
> >
> > this would require multiple libraries to be involved in the equation and
> > the answer is undefined behavior which most certainly result in runtime
> > failures ...
>
> Point there was that instead of just bailing with "lib is missing",
> suspect it'll manage to run, then segfault at potentially crappy
> times.

this is really an outside case and not worth worrying over
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-portage-dev] New preserve-libs feature

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 3515 bytes --]

On Sat, Feb 17, 2007 at 10:09:35AM -0500, Mike Frysinger wrote:
> On Saturday 17 February 2007, Brian Harring wrote:
> > On Sat, Feb 17, 2007 at 09:39:58AM -0500, Mike Frysinger wrote:
> > > On Saturday 17 February 2007, Brian Harring wrote:
> > > > Security impact is from a pkg potentially dragging along old libs; if
> > > > you've got a stable pkg that gets an update once every blue moon, it
> > > > can hold onto the lib for a *long* time while still using the lib;
> > > > thus if a vuln. in the lib, said pkg still is screwed.
> > >
> > > umm, no ... the package itself is updated against the new copy while the
> > > old copy exists for other packages that have already been built
> >
> > Suspect you're ignoring soname changes, which is about all this patch
> > would address- for example, ssl's old form of breakage.  In that case,
> > *yes* the package gets updated, anything recompiled should get the
> > correct lib
> 
> i'm not ignoring soname changes, those are exactly what i'm talking about
> 
> > (assuming the code knows the appropriate linker args)
> 
> there is no such thing ... it's always "-lfoo"

The point is that it is *not* always -lfoo.  Commenting about packages 
that collapse, split libs apart, where -lorig becomes -lnew1 -lnew2.

Non-slotted package upgrade crossing a major version (assuming they're 
not being dumb asses), that scenario *does* occur, and it's not the 
same args.

Until all packages are upgraded (assuming an ugprade is available) to 
use the new layout, the old lingers.

Also, so help me, if you respond to valid critism with "it doesn't 
actually matter" as a retort, you're getting wedgied considering 
this is one of the scenarios this patch is supposed to address :p


> > > > Other angle is someone intentionally forcing usage of a known bad
> > > > library that is still dangling.  Corner case, but doable.
> > >
> > > as i said, this is the "invalid" syntax:
> > > ... /usr/lib/libfoo.so.3 ...
> >
> > Not to LD_PRELOAD :)
> >
> > Haven't tried anything crazy, but suspect it can be abused to override
> > to the old.
> 
> again, not in any scenario that actually matters ... so this too is a 
> pointless line of thought to pursue as it has no real security impact

Eh?  setuid comes to mind, or any other attempt to finagle privs via 
forcing the bad lib.  Combined with the scenario above (where a 
crappy pkg can hold the lib around for a *long* time), tend to think 
it matters.


> > > > Bit curious how this is going to behave if via linked in libs, new loc
> > > > and old get loaded alongside...
> > >
> > > this would require multiple libraries to be involved in the equation and
> > > the answer is undefined behavior which most certainly result in runtime
> > > failures ...
> >
> > Point there was that instead of just bailing with "lib is missing",
> > suspect it'll manage to run, then segfault at potentially crappy
> > times.
> 
> this is really an outside case and not worth worrying over

It's a change in behaviour; previously, would just flat out stop; now 
it'll run, but probably take a poo in your shoes.

Going from the potential of "it won't run" to "it eats itself in 
special way" is *not* something I'd blistfully write off as "not 
worth worring over", considering what this change intends to address.

Additional comment, introducing the change makes it so that glsa's 
aren't really as accurate- version based, rather then lib.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-portage-dev] New preserve-libs feature

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 2384 bytes --]

On Saturday 17 February 2007, Brian Harring wrote:
> On Sat, Feb 17, 2007 at 10:09:35AM -0500, Mike Frysinger wrote:
> > On Saturday 17 February 2007, Brian Harring wrote:
> > > (assuming the code knows the appropriate linker args)
> >
> > there is no such thing ... it's always "-lfoo"
>
> The point is that it is *not* always -lfoo.  Commenting about packages
> that collapse, split libs apart, where -lorig becomes -lnew1 -lnew2.

so why is this an issue ... the build system for a package either takes it 
into account or it doesnt, it'll behave exactly the same whether we preserve 
the ABI lib

we're preserving runtime libs here, not ones detectable by a linker process 
and thus a build system

> Also, so help me, if you respond to valid critism with "it doesn't
> actually matter" as a retort, you're getting wedgied considering
> this is one of the scenarios this patch is supposed to address :p

i guess come up with a valid criticism first and then i wont use that

> Eh?  setuid comes to mind

why ?  pretty much all LD_* variables are filtered by ldso in a set*id 
environment

> or any other attempt to finagle privs via forcing the bad lib.

considering the only privs you can finagle are your own, this is not an issue

> Combined with the scenario above (where a 
> crappy pkg can hold the lib around for a *long* time), tend to think
> it matters.

it doesnt

> It's a change in behaviour; previously, would just flat out stop; now
> it'll run, but probably take a poo in your shoes.
>
> Going from the potential of "it won't run" to "it eats itself in
> special way" is *not* something I'd blistfully write off as "not
> worth worring over", considering what this change intends to address.

let's review the original by copy & paste:
- I'm not claiming that it's a silver bullet to all possible runtime
linker problems, it's supposed to cover some of the common cases (like
the expat problem)

what you're talking about can never be fully solved by the methodology 
utilized here ... if you want the full solution, go implement your own 
behavior

> Additional comment, introducing the change makes it so that glsa's
> aren't really as accurate- version based, rather then lib.

considering current glsa integration (== 0), i'd say proper general 
infrastructure needs to be put into place first
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

[gentoo-portage-dev] Re: New preserve-libs feature

[Duncan]

Mike Frysinger <vapier@gentoo.org> posted
200702171009.35968.vapier@gentoo.org, excerpted below, on  Sat, 17 Feb
2007 10:09:35 -0500:

>> the old vuln. lib still will hang around as long as anything refs it.
>
> of course and this is the desired behavior ... people need to run
> revdep-rebuild, there's no two ways about it

Question:  With the old library still around, will revdep-rebuild even try
to rebuild anything linked against it?  Maybe I'm wrong, but I thought it
would only rebuild when the library was actually missing.  (There's also a
hint of that in another comment, but maybe I'm reading that wrong as well.)

(Yes, I know the sig delimiter below is bugged.  Bugged pan ATM.)
--
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: New preserve-libs feature

[Paul Varner]

On Sat, 2007-02-17 at 20:56 +0000, Duncan wrote:
> Question:  With the old library still around, will revdep-rebuild even try
> to rebuild anything linked against it?  Maybe I'm wrong, but I thought it
> would only rebuild when the library was actually missing.  (There's also a
> hint of that in another comment, but maybe I'm reading that wrong as well.)

Not by default.  However, if you specify the --library parameter with
the older library, it will rebuild the stuff linked to that library.

Regards,
Paul
-- 
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] Re: New preserve-libs feature

[Duncan]

Paul Varner <fuzzyray@gentoo.org> posted
1171777543.9946.1.camel@garath.homelinux.org, excerpted below, on  Sat, 17
Feb 2007 23:45:43 -0600:

> On Sat, 2007-02-17 at 20:56 +0000, Duncan wrote:
>> Question:  With the old library still around, will revdep-rebuild even try
>> to rebuild anything linked against it?  Maybe I'm wrong, but I thought []
>
> Not by default.  However, if you specify the --library parameter with
> the older library, it will rebuild the stuff linked to that library.

The light dawns... Thanks! =8^)

--
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: New preserve-libs feature

[Carsten Lohrke]

On Samstag, 17. Februar 2007, Duncan wrote:
> Question:  With the old library still around, will revdep-rebuild even try
> to rebuild anything linked against it?  Maybe I'm wrong, but I thought it
> would only rebuild when the library was actually missing.  (There's also a
> hint of that in another comment, but maybe I'm reading that wrong as well.)

The question isn't so much, if revdep-rebuild picks it up, the problem from my 
POV is that the information to rebuild against the new library shows up only 
once via ewarn in pkg_postinst and unexperienced users may not have 
configured elog facility and may miss to see the emerge output scrolling by, 
so the library and everything built against it remains as it is.

Therefore I consider the preserve-libs functionality one of the biggest 
security threats for Gentoo users. You may dismiss this, saying the problem 
sits in front of the keyboard, but I'm telling you this is careless and that 
we can do better:

echo "/path/to/preserved.so" >> /var/lib/portage/preserved_libs

stores the libraries, and Portage can each time emerge is run look up, if the 
file lists libraries, check, if those exist, if not remove the lines or 
otherwise warn the user about the possibly vulnerable libraries and tell him 
what to do.

Simple solution at low cost. Fine with this idea?


Carsten


-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: New preserve-libs feature

[Marius Mauch]

On Fri, 23 Feb 2007 14:22:05 +0100
Carsten Lohrke <carlo@gentoo.org> wrote:

> echo "/path/to/preserved.so" >> /var/lib/portage/preserved_libs
> 
> stores the libraries, and Portage can each time emerge is run look up, if the 
> file lists libraries, check, if those exist, if not remove the lines or 
> otherwise warn the user about the possibly vulnerable libraries and tell him 
> what to do.
> 
> Simple solution at low cost. Fine with this idea?

Yeah, I had planned a registration for those libs anyway to assist
revdep-rebuild, just haven't settled on a format yet (need to track
which package registered the lib).

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] Re: New preserve-libs feature

[Duncan]

Carsten Lohrke <carlo@gentoo.org> posted
200702231422.05809.carlo@gentoo.org, excerpted below, on  Fri, 23 Feb 2007
14:22:05 +0100:

> I consider the preserve-libs functionality one of the biggest
> security threats for Gentoo users. You may dismiss this, saying the
> problem sits in front of the keyboard, but I'm telling you this is
> careless and that we can do better:
> 
> echo "/path/to/preserved.so" >> /var/lib/portage/preserved_libs
> 
> stores the libraries, and Portage can each time emerge is run look up,
> if the file lists libraries, check, if those exist, if not remove the
> lines or otherwise warn the user about the possibly vulnerable libraries
> and tell him what to do.

+1 here!  During my own sysadmin-ings, I've wondered why there wasn't 
such a list on several occasions.  It would make things /so/ much 
simpler, at least from the sysadmin perspective.  (Of course, I realize 
that's /not/ the same thing as simpler from a portage perspective, but 
anyway, that's what's being discussed here. =8^)

If this is added, I think it's big enough to have it mentioned in the 
handbook as well.  Having that handy list all nicely centralized to one 
location would be a /big/ boon to security conscious Gentoo sysadmins 
everywhere, so it's easily worth mentioning in the handbook as one of the 
valuable tools portage provides.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-portage-dev@gentoo.org mailing list